2024-09-05T12:05:51.446 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-05T12:05:51.473 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T12:05:51.574 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431 branch: wip-bharath6-testing-2024-09-05-1413-squid description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7890431' kernel: client: branch: testing flavor: default kdb: 1 sha1: 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath6-testing-2024-09-05-1413-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: e856d6b2462509a377ef39149a05475ed006ebac subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: e856d6b2462509a377ef39149a05475ed006ebac kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath6-testing-2024-09-05-1413-squid sha1: e856d6b2462509a377ef39149a05475ed006ebac owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8220 sha1: e856d6b2462509a377ef39149a05475ed006ebac sleep_before_teardown: 0 subset: 990/1024 suite: fs:workload suite_branch: wip-bharath6-testing-2024-09-05-1413-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: e856d6b2462509a377ef39149a05475ed006ebac syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAcvzf7PJF84FreTgg8m0Kn3RUo3WWpPFdpnxyUoseX3eq/5FFPs3PntU2dLIzpOKNPk6N5G8PlgXdwevMfu8E= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOFWE4ApFjacwkp6c9khbTOcTiEGdbvZiv3HnmCKvf/znOmPq7JWHqJ9xIiUgPvejdWf2wbJtQGuMJn/5I77jk8= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMVCDiYl5LYViGm/k0IPiZjWPH0NaA30tJHOoSCv8TtiU3w/04ksN6VTb42nuSSKyVMeibkw1m8KL+6rCIiMGE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-05_11:20:21 tube: smithi user: hyelloji verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-05T12:05:51.575 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa; will attempt to use it 2024-09-05T12:05:51.575 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks 2024-09-05T12:05:51.575 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-05T12:05:51.577 INFO:teuthology.task.internal:Checking packages... 2024-09-05T12:05:51.598 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e856d6b2462509a377ef39149a05475ed006ebac' 2024-09-05T12:05:51.598 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-05T12:05:51.598 INFO:teuthology.packaging:ref: None 2024-09-05T12:05:51.598 INFO:teuthology.packaging:tag: None 2024-09-05T12:05:51.598 INFO:teuthology.packaging:branch: wip-bharath6-testing-2024-09-05-1413-squid 2024-09-05T12:05:51.598 INFO:teuthology.packaging:sha1: e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:05:51.599 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath6-testing-2024-09-05-1413-squid 2024-09-05T12:05:51.734 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-261.ge856d6b2 2024-09-05T12:05:51.735 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-05T12:05:51.742 INFO:teuthology.task.internal:no buildpackages task found 2024-09-05T12:05:51.742 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-05T12:05:51.754 INFO:teuthology.task.internal:Saving configuration 2024-09-05T12:05:51.775 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-05T12:05:51.781 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-05T12:05:51.802 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 12:00:51.605460', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGAcvzf7PJF84FreTgg8m0Kn3RUo3WWpPFdpnxyUoseX3eq/5FFPs3PntU2dLIzpOKNPk6N5G8PlgXdwevMfu8E='} 2024-09-05T12:05:51.824 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 12:00:51.603775', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOFWE4ApFjacwkp6c9khbTOcTiEGdbvZiv3HnmCKvf/znOmPq7JWHqJ9xIiUgPvejdWf2wbJtQGuMJn/5I77jk8='} 2024-09-05T12:05:51.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-05 12:00:51.606574', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMMVCDiYl5LYViGm/k0IPiZjWPH0NaA30tJHOoSCv8TtiU3w/04ksN6VTb42nuSSKyVMeibkw1m8KL+6rCIiMGE='} 2024-09-05T12:05:51.843 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-05T12:05:51.849 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-05T12:05:51.849 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-05T12:05:51.850 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-05T12:05:51.850 INFO:teuthology.run_tasks:Running task console_log... 2024-09-05T12:05:51.930 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc1c9e4fac0>, signals=[15]) 2024-09-05T12:05:51.930 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-05T12:05:51.937 INFO:teuthology.task.internal:Opening connections... 2024-09-05T12:05:51.937 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:05:51.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:05:52.015 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-05T12:05:52.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:05:52.088 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-09-05T12:05:52.089 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:05:52.163 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-05T12:05:52.170 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-09-05T12:05:52.188 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-09-05T12:05:52.188 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-09-05T12:05:52.247 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T12:05:52.248 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T12:05:52.249 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-09-05T12:05:52.274 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-05T12:05:52.293 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-05T12:05:52.293 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-05T12:05:52.350 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-05T12:05:52.350 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-05T12:05:52.350 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-05T12:05:52.350 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T12:05:52.351 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T12:05:52.351 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-05T12:05:52.382 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-09-05T12:05:52.401 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-09-05T12:05:52.401 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-09-05T12:05:52.459 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-09-05T12:05:52.459 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-09-05T12:05:52.459 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-05T12:05:52.460 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-05T12:05:52.461 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-09-05T12:05:52.482 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-05T12:05:52.496 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-05T12:05:52.502 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-05T12:05:52.502 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-09-05T12:05:52.505 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-05T12:05:52.507 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-09-05T12:05:52.524 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-05T12:05:52.531 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-05T12:05:52.531 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-09-05T12:05:52.562 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-05T12:05:52.564 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-09-05T12:05:52.783 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-05T12:05:52.855 INFO:teuthology.run_tasks:Running task kernel... 2024-09-05T12:05:52.870 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '8ad8b8b160987a14eb199bb67cc00e23a08d43f6'} 2024-09-05T12:05:52.870 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-05T12:05:52.870 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-05T12:05:52.871 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T12:05:52.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:05:52.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-05T12:05:53.058 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-09-05T12:05:53.059 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8ad8b8b160987a14eb199bb67cc00e23a08d43f6"... 2024-09-05T12:05:53.059 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-05T12:05:53.077 INFO:teuthology.orchestra.run.smithi071.stdout:5.14.0-503.el9.x86_64 2024-09-05T12:05:53.077 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-09-05T12:05:53.077 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-05T12:05:53.078 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum upgrade -y linux-firmware 2024-09-05T12:05:55.571 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-05T12:05:57.322 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-05T12:06:05.505 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.5 MB 00:04 2024-09-05T12:06:07.100 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 19 kB 00:00 2024-09-05T12:06:08.018 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-05T12:06:13.474 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-05T12:06:15.512 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-05T12:06:15.514 INFO:teuthology.orchestra.run.smithi071.stdout:Nothing to do. 2024-09-05T12:06:15.514 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-05T12:06:15.636 INFO:teuthology.task.kernel:Downloading kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-09-05T12:06:15.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8ad8b8b160987a14eb199bb67cc00e23a08d43f6 2024-09-05T12:06:15.786 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ 2024-09-05T12:06:15.786 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-05T12:06:15.788 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi071.front.sepia.ceph.com to complete... 2024-09-05T12:06:24.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05 12:06:24 URL:https://1.chacra.ceph.com/r/kernel/testing/8ad8b8b160987a14eb199bb67cc00e23a08d43f6/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g8ad8b8b16098-1.x86_64.rpm [359646365/359646365] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-05T12:06:24.961 INFO:teuthology.orchestra.run.smithi071.stderr:FINISHED --2024-09-05 12:06:24-- 2024-09-05T12:06:24.961 INFO:teuthology.orchestra.run.smithi071.stderr:Total wall clock time: 9.1s 2024-09-05T12:06:24.961 INFO:teuthology.orchestra.run.smithi071.stderr:Downloaded: 1 files, 343M in 8.9s (38.7 MB/s) 2024-09-05T12:06:24.963 INFO:teuthology.task.kernel:Installing kernel 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 on client.0... 2024-09-05T12:06:24.963 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-05T12:06:27.108 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying... ######################################## 2024-09-05T12:06:27.240 INFO:teuthology.orchestra.run.smithi071.stdout:Preparing... ######################################## 2024-09-05T12:06:27.367 INFO:teuthology.orchestra.run.smithi071.stdout:Updating / installing... 2024-09-05T12:06:35.326 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-6.11.0_rc5_g8ad8b8b16098-1 ######################################## 2024-09-05T12:07:15.661 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi071.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-05T12:07:15.661 DEBUG:teuthology.orchestra.run.smithi071:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-05T12:07:15.760 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/System.map-6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.760 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/config-6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.760 INFO:teuthology.orchestra.run.smithi071.stdout:/boot/vmlinuz-6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.760 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.761 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/System.map 2024-09-05T12:07:15.761 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/config 2024-09-05T12:07:15.761 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel 2024-09-05T12:07:15.761 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch 2024-09-05T12:07:15.761 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86 2024-09-05T12:07:15.762 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto 2024-09-05T12:07:15.762 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-05T12:07:15.762 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-05T12:07:15.762 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-05T12:07:15.762 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-05T12:07:15.763 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-05T12:07:15.763 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-05T12:07:15.763 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-05T12:07:15.763 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-05T12:07:15.763 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-05T12:07:15.763 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-05T12:07:15.764 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-05T12:07:15.764 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-05T12:07:15.764 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-05T12:07:15.764 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-05T12:07:15.764 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-05T12:07:15.765 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-05T12:07:15.765 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-05T12:07:15.765 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-05T12:07:15.765 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-05T12:07:15.765 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-05T12:07:15.766 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel 2024-09-05T12:07:15.766 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu 2024-09-05T12:07:15.766 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce 2024-09-05T12:07:15.766 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-05T12:07:15.766 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm 2024-09-05T12:07:15.766 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-05T12:07:15.767 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-05T12:07:15.767 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/arch/x86/kvm/kvm.ko 2024-09-05T12:07:15.767 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto 2024-09-05T12:07:15.767 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ansi_cprng.ko 2024-09-05T12:07:15.767 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/anubis.ko 2024-09-05T12:07:15.768 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/arc4.ko 2024-09-05T12:07:15.768 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx 2024-09-05T12:07:15.768 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_memcpy.ko 2024-09-05T12:07:15.768 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_pq.ko 2024-09-05T12:07:15.768 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-05T12:07:15.769 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_tx.ko 2024-09-05T12:07:15.769 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/async_xor.ko 2024-09-05T12:07:15.769 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/async_tx/raid6test.ko 2024-09-05T12:07:15.769 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authenc.ko 2024-09-05T12:07:15.769 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/authencesn.ko 2024-09-05T12:07:15.770 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blake2b_generic.ko 2024-09-05T12:07:15.770 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_common.ko 2024-09-05T12:07:15.770 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/blowfish_generic.ko 2024-09-05T12:07:15.770 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/camellia_generic.ko 2024-09-05T12:07:15.770 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast5_generic.ko 2024-09-05T12:07:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast6_generic.ko 2024-09-05T12:07:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cast_common.ko 2024-09-05T12:07:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ccm.ko 2024-09-05T12:07:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cmac.ko 2024-09-05T12:07:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crc32_generic.ko 2024-09-05T12:07:15.771 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_engine.ko 2024-09-05T12:07:15.772 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/crypto_user.ko 2024-09-05T12:07:15.772 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/cts.ko 2024-09-05T12:07:15.772 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/des_generic.ko 2024-09-05T12:07:15.772 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/dh_generic.ko 2024-09-05T12:07:15.772 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecc.ko 2024-09-05T12:07:15.773 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/ecdh_generic.ko 2024-09-05T12:07:15.773 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/echainiv.ko 2024-09-05T12:07:15.773 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/essiv.ko 2024-09-05T12:07:15.773 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/fcrypt.ko 2024-09-05T12:07:15.773 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/khazad.ko 2024-09-05T12:07:15.774 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4.ko 2024-09-05T12:07:15.774 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/lz4hc.ko 2024-09-05T12:07:15.774 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/md4.ko 2024-09-05T12:07:15.774 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/michael_mic.ko 2024-09-05T12:07:15.774 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcbc.ko 2024-09-05T12:07:15.775 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/pcrypt.ko 2024-09-05T12:07:15.775 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/rmd160.ko 2024-09-05T12:07:15.775 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/seed.ko 2024-09-05T12:07:15.775 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/serpent_generic.ko 2024-09-05T12:07:15.775 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tcrypt.ko 2024-09-05T12:07:15.776 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/tea.ko 2024-09-05T12:07:15.776 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_common.ko 2024-09-05T12:07:15.776 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/twofish_generic.ko 2024-09-05T12:07:15.776 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/vmac.ko 2024-09-05T12:07:15.776 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/wp512.ko 2024-09-05T12:07:15.776 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xcbc.ko 2024-09-05T12:07:15.777 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xor.ko 2024-09-05T12:07:15.777 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/crypto/xxhash_generic.ko 2024-09-05T12:07:15.777 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers 2024-09-05T12:07:15.777 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi 2024-09-05T12:07:15.777 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-05T12:07:15.778 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/acpi_pad.ko 2024-09-05T12:07:15.778 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/ec_sys.ko 2024-09-05T12:07:15.778 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/platform_profile.ko 2024-09-05T12:07:15.778 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbs.ko 2024-09-05T12:07:15.778 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/sbshc.ko 2024-09-05T12:07:15.778 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/acpi/video.ko 2024-09-05T12:07:15.779 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata 2024-09-05T12:07:15.779 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/acard-ahci.ko 2024-09-05T12:07:15.779 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ahci_platform.ko 2024-09-05T12:07:15.779 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/ata_generic.ko 2024-09-05T12:07:15.779 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/libahci_platform.ko 2024-09-05T12:07:15.780 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_acpi.ko 2024-09-05T12:07:15.780 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ali.ko 2024-09-05T12:07:15.780 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_amd.ko 2024-09-05T12:07:15.780 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_artop.ko 2024-09-05T12:07:15.780 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atiixp.ko 2024-09-05T12:07:15.780 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_atp867x.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd640.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cmd64x.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_cypress.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_efar.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt366.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt37x.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it8213.ko 2024-09-05T12:07:15.781 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_it821x.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_jmicron.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_marvell.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_mpiix.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_netcell.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ninja32.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87410.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_ns87415.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_oldpiix.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_opti.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_optidma.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pcmcia.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-05T12:07:15.782 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_piccolo.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_rdc.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sch.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_serverworks.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sil680.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sis.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_sl82c105.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_triflex.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pata_via.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/pdc_adma.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_inic162x.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_mv.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_nv.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_promise.ko 2024-09-05T12:07:15.783 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_qstor.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sil24.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sis.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_svw.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_sx4.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_uli.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_via.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ata/sata_vsc.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/auxdisplay/ks0108.ko 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap 2024-09-05T12:07:15.784 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/bcma/bcma.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/aoe/aoe.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/brd.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/drbd/drbd.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/floppy.ko 2024-09-05T12:07:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/loop.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/nbd.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/null_blk/null_blk.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/pktcdvd.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/rbd.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/virtio_blk.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/xen-blkfront.ko 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram 2024-09-05T12:07:15.786 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/block/zram/zram.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hangcheck-timer.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/via-rng.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-05T12:07:15.787 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/lp.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/mwave/mwave.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/ppdev.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tlclk.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-05T12:07:15.788 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/uv_mmtimer.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/char/virtio_console.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto 2024-09-05T12:07:15.789 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/ccp/ccp.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-aes.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/padlock-sha.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca 2024-09-05T12:07:15.790 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dca/dca.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/amd64_edac.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/e752x_edac.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/edac_mce_amd.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3000_edac.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i3200_edac.ko 2024-09-05T12:07:15.791 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5100_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i5400_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7300_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i7core_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/i82975x_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/ie31200_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/sb_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/edac/x38_edac.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/edd.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio 2024-09-05T12:07:15.792 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-ich.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-appleir.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-aureal.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-axff.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-dr.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elecom.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-elo.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-emsff.ko 2024-09-05T12:07:15.793 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gaff.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gt683r.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-gyration.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-holtekff.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-hyperv.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-icade.ko 2024-09-05T12:07:15.794 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-keytouch.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-kye.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lcpower.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-led.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-lenovo.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-multitouch.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-ortek.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-penmount.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-petalynx.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-picolcd.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-pl.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-plantronics.ko 2024-09-05T12:07:15.795 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-primax.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-rmi.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-common.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-05T12:07:15.796 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-roccat.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-saitek.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-samsung.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sjoy.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sony.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-speedlink.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-steelseries.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-sunplus.ko 2024-09-05T12:07:15.797 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tivo.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-tmff.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-topseed.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-twinhan.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-uclogic.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-waltop.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-wiimote.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-xinmo.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zpff.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/hid-zydacron.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/uhid.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hid/wacom.ko 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv 2024-09-05T12:07:15.798 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_balloon.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_utils.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hv/hv_vmbus.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/abituguru3.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7414.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ad7418.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adc128d818.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1025.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1026.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1029.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm1031.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adm9240.ko 2024-09-05T12:07:15.799 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ads7828.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7410.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7411.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7462.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7470.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7475.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/adt7x10.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/amc6821.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/applesmc.ko 2024-09-05T12:07:15.800 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asb100.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asc7621.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/atxp1.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/coretemp.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/dme1737.ko 2024-09-05T12:07:15.801 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds1621.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ds620.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc1403.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/emc6w201.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71805f.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f71882fg.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/f75375s.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fam15h_power.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/fschmd.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g760a.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/g762.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl518sm.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/gl520sm.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5500_temp.ko 2024-09-05T12:07:15.802 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/i5k_amb.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmaem.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ibmpex.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina209.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ina2xx.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/it87.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k10temp.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/k8temp.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lineage-pem.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm63.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm73.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm75.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm77.ko 2024-09-05T12:07:15.803 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm78.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm80.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm83.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm85.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm87.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm90.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm92.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm93.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95234.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95241.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/lm95245.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc2945.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4151.ko 2024-09-05T12:07:15.804 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4215.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4222.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4245.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4260.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/ltc4261.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max16065.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1619.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max1668.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max197.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6639.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6650.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/max6697.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/mcp3021.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6683.ko 2024-09-05T12:07:15.805 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775-core.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct6775.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/nct7802.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87360.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pc87427.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pcf8591.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-05T12:07:15.806 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/powr1220.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5627.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch5636.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht15.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sht21.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/shtc1.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/sis5595.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47b397.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m1.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/smsc47m192.ko 2024-09-05T12:07:15.807 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/thmc50.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp102.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp103.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp401.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/tmp421.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via-cputemp.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/via686a.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt1211.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/vt8231.ko 2024-09-05T12:07:15.808 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627ehf.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83627hf.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83781d.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83791d.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83792d.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83793.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83795.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l785ts.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/hwmon/w83l786ng.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-05T12:07:15.809 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-05T12:07:15.810 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-dev.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-mux.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-smbus.ko 2024-09-05T12:07:15.811 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/i2c/i2c-stub.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_core.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4 2024-09-05T12:07:15.812 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp 2024-09-05T12:07:15.813 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/gameport/gameport.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/joydev.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/appletouch.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/bcm5974.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/cyapatp.ko 2024-09-05T12:07:15.814 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/sermouse.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/altera_ps2.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/arc_ps2.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/serio/serio_raw.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/input/sparse-keymap.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-blinkm.ko 2024-09-05T12:07:15.815 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lm3530.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-lp3944.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/leds-ss4200.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache 2024-09-05T12:07:15.816 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/bcache/bcache.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-bio-prison.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache-smq.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-cache.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-crypt.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-delay.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-flakey.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-log-userspace.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-multipath.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-queue-length.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-raid.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-round-robin.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-service-time.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-switch.ko 2024-09-05T12:07:15.817 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-thin-pool.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/dm-verity.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid0.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid1.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid10.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/md/raid456.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptbase.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptctl.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptfc.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptlan.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptsas.ko 2024-09-05T12:07:15.818 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptscsih.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/message/fusion/mptspi.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_ich.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/lpc_sch.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/mfd-core.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/sm501.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/viperboard.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/vx855.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/mfd/wl1273-core.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-05T12:07:15.819 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds9802als.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/apds990x.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/bh1770glc.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/cb710/cb710.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/at24.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/eeprom/max6875.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/enclosure.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/hpilo.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29003.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/isl29020.ko 2024-09-05T12:07:15.820 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-me.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei-txe.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/mei/mei.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_7xx1.ko 2024-09-05T12:07:15.821 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tifm_core.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/tsl2550.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_balloon.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/bonding/bonding.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/dummy.ko 2024-09-05T12:07:15.822 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/eql.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-05T12:07:15.823 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-05T12:07:15.824 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet 2024-09-05T12:07:15.825 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-05T12:07:15.826 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink 2024-09-05T12:07:15.827 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/dnet.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ethoc.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/fealnx.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-05T12:07:15.828 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-05T12:07:15.829 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/jme.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-05T12:07:15.830 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-05T12:07:15.831 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-05T12:07:15.832 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-05T12:07:15.833 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-05T12:07:15.834 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/geneve.ko 2024-09-05T12:07:15.835 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ifb.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvlan.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/macvtap.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-05T12:07:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/mii.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/net_failover.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/netconsole.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/nlmon.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/amd.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/bcm87xx.ko 2024-09-05T12:07:15.837 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/broadcom.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/cicada.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/davicom.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/dp83640.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/et1011c.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/icplus.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/lxt.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/marvell.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/micrel.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/national.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/phylink.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/qsemi.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/realtek.ko 2024-09-05T12:07:15.838 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/smsc.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/ste10Xp.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/phy/vitesse.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_async.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppoe.ko 2024-09-05T12:07:15.839 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pppox.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/ppp/pptp.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slhc.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/slip/slip.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/sungem_phy.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tap.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_random.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-05T12:07:15.840 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/tun.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/veth.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/virtio_net.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/vxlan/vxlan.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/net/xen-netfront.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ntb/ntb.ko 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme 2024-09-05T12:07:15.841 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme-core.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/nvme/host/nvme.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_cs.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_pc.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/parport/parport_serial.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pci/xen-pcifront.ko 2024-09-05T12:07:15.842 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/i82092.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/pd6729.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/acerhdf.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-05T12:07:15.843 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/hdaps.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/intel_ips.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-05T12:07:15.844 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/platform/x86/wmi.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/clients/pps_parport.ko 2024-09-05T12:07:15.845 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/pps/pps_core.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ptp/ptp_kvm.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ras/amd/fmpm.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-05T12:07:15.846 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-em3027.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-max6900.ko 2024-09-05T12:07:15.847 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/rtc/rtc-x1205.ko 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi 2024-09-05T12:07:15.848 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-9xxx.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-sas.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/3w-xxxx.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/BusLogic.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/a100u2w.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/advansys.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/am53c974.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-05T12:07:15.849 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/atp870u.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bfa/bfa.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ch.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-05T12:07:15.850 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dc395x.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/dmx3191d.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-05T12:07:15.851 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/esp_scsi.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/fnic/fnic.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hpsa.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hptiop.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/hv_storvsc.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/initio.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ipr.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ips.ko 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci 2024-09-05T12:07:15.852 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/isci/isci.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libfc/libfc.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/libsas/libsas.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-05T12:07:15.853 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/mvumi.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/pmcraid.ko 2024-09-05T12:07:15.854 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla1280.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/raid_class.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_debug.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/ses.ko 2024-09-05T12:07:15.855 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/st.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/stex.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/virtio_scsi.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/scsi/wd719x.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ssb/ssb.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-05T12:07:15.856 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_file.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_iblock.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_mod.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_pscsi.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/target_core_user.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-05T12:07:15.857 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty 2024-09-05T12:07:15.858 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_gsm.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/n_hdlc.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/nozomi.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/arc_uart.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host 2024-09-05T12:07:15.859 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_aec.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_cif.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_pci_generic.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/uio/uio_sercos3.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-acm.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usblp.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/class/usbtmc.ko 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host 2024-09-05T12:07:15.860 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/mdc800.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/image/microtek.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/adutux.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/appledisplay.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi26.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/emi62.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ezusb.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/idmouse.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/iowarrior.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/ldusb.ko 2024-09-05T12:07:15.861 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/legousbtower.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usb3503.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usblcd.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/uss720.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/misc/yurex.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/phy/phy-generic.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/aircable.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ark3116.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-05T12:07:15.862 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ch341.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cp210x.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cyberjack.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/empeg.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/io_ti.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipaq.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ipw.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ir-usb.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-05T12:07:15.863 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mct_u232.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7720.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/mos7840.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/navman.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/omninet.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/opticon.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/option.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/oti6858.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/pl2303.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcaux.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/qcserial.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/quatech2.ko 2024-09-05T12:07:15.864 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/safe_serial.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/sierra.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ssu100.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/symbolserial.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_debug.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/visor.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/whiteheat.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/uas.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-05T12:07:15.865 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-karma.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-05T12:07:15.866 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/storage/usb-storage.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-05T12:07:15.867 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_net.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/vhost/vhost_scsi.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/apple_bl.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/lcd.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_balloon.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_mmio.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-05T12:07:15.868 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/virtio/virtio_ring.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2482.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/masters/ds2490.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-05T12:07:15.869 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/w1/wire.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/hpwdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/i6300esb.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ib700wdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ibmasr.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-05T12:07:15.870 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/it87_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/machzwd.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/nv_tco.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/softdog.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/via_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-05T12:07:15.871 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/wdt_pci.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/watchdog/xen_wdt.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-evtchn.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntalloc.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-gntdev.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xen-privcmd.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p 2024-09-05T12:07:15.872 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/9p/9p.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/affs/affs.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/befs/befs.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/binfmt_misc.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/btrfs/btrfs.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cachefiles/cachefiles.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ceph/ceph.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/coda/coda.ko 2024-09-05T12:07:15.873 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/cramfs/cramfs.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/dlm/dlm.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/f2fs/f2fs.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/fat.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/msdos.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fat/vfat.ko 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse 2024-09-05T12:07:15.874 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/cuse.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/fuse/fuse.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/gfs2/gfs2.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfs/hfs.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/hfsplus/hfsplus.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/isofs/isofs.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/jfs/jfs.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/lockd/lockd.ko 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix 2024-09-05T12:07:15.875 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/minix/minix.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/netfs/netfs.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfs.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv3.ko 2024-09-05T12:07:15.876 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs/nfsv4.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/grace.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfs_common/nfs_acl.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nfsd/nfsd.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nilfs2/nilfs2.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-celtic.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-centeuro.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-croatian.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-cyrillic.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-gaelic.ko 2024-09-05T12:07:15.877 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-greek.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-iceland.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-inuit.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-roman.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-romanian.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/mac-turkish.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1250.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1251.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp1255.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp737.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp775.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp850.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp852.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp855.ko 2024-09-05T12:07:15.878 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp857.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp860.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp861.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp862.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp863.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp864.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp865.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp866.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp869.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp874.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp932.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp936.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp949.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_cp950.ko 2024-09-05T12:07:15.879 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_euc-jp.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-1.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-13.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-14.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-15.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-2.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-3.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-4.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-5.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-6.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-7.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_iso8859-9.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-r.ko 2024-09-05T12:07:15.880 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-ru.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_koi8-u.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/nls/nls_utf8.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-05T12:07:15.881 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/overlayfs/overlay.ko 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/pstore/ramoops.ko 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/reiserfs/reiserfs.ko 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/romfs/romfs.ko 2024-09-05T12:07:15.882 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/client/cifs.ko 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_arc4.ko 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/smb/common/cifs_md4.ko 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/squashfs/squashfs.ko 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/sysv/sysv.ko 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf 2024-09-05T12:07:15.883 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/udf/udf.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/ufs/ufs.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/fs/xfs/xfs.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/rcu/rcutorture.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/torture.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/asn1_encoder.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc-itu-t.ko 2024-09-05T12:07:15.884 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crc8.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libaescfb.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libarc4.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/crypto/libdes.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/libcrc32c.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lru_cache.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4_compress.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/lz4/lz4hc_compress.ko 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math 2024-09-05T12:07:15.885 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/math/cordic.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/raid6/raid6_pq.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_bm.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_fsm.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/ts_kmp.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/lib/zstd/zstd_compress.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/mm/hwpoison-inject.ko 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net 2024-09-05T12:07:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/6lowpan.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_dest.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_fragment.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_hop.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_mobility.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_routing.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/6lowpan/nhc_udp.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/garp.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/mrp.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/p8022.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/psnap.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/802/stp.ko 2024-09-05T12:07:15.887 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/8021q/8021q.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_fd.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_rdma.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/9p/9pnet_virtio.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/batman-adv/batman-adv.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/br_netfilter.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/bridge.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-05T12:07:15.888 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-05T12:07:15.889 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/ebtables.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ceph/libceph.ko 2024-09-05T12:07:15.890 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/failover.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/core/pktgen.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_diag.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv4.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dccp/dccp_ipv6.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dns_resolver/dns_resolver.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/dsa_core.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_brcm.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_dsa.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_none.ko 2024-09-05T12:07:15.891 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/dsa/tag_trailer.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ah4.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/esp4.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/fou.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/gre.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/inet_diag.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_gre.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_tunnel.ko 2024-09-05T12:07:15.892 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ip_vti.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipcomp.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/ipip.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-05T12:07:15.893 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-05T12:07:15.894 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_bic.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_dctcp.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_diag.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_highspeed.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_htcp.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_hybla.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_illinois.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_lp.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_scalable.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_vegas.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_veno.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_westwood.ko 2024-09-05T12:07:15.895 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tcp_yeah.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/tunnel4.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_diag.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/udp_tunnel.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ah6.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/esp6.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/fou6.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_tunnel.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ip6_vti.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/ipcomp6.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-05T12:07:15.896 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-05T12:07:15.897 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/sit.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/tunnel6.ko 2024-09-05T12:07:15.898 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/key/af_key.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_core.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_eth.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ip6.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_netlink.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/l2tp/l2tp_ppp.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/llc/llc.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mac802154/mac802154.ko 2024-09-05T12:07:15.899 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/mpls/mpls_gso.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-05T12:07:15.900 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-05T12:07:15.901 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-05T12:07:15.902 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conncount.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-05T12:07:15.903 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_log_syslog.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_irc.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_sip.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nf_tables.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-05T12:07:15.904 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_log.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_chain_nat.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_compat.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_ct.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fib.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_hash.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_limit.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_log.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_masq.ko 2024-09-05T12:07:15.905 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_nat.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_numgen.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_queue.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_quota.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_redir.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_reject_inet.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_socket.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/nft_tunnel.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_AUDIT.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-05T12:07:15.906 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_CT.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_DSCP.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HL.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_HMARK.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LED.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_LOG.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NETMAP.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFLOG.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_RATEEST.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_SECMARK.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-05T12:07:15.907 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TEE.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TPROXY.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_TRACE.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_addrtype.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_bpf.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cgroup.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cluster.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_comment.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connbytes.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlabel.ko 2024-09-05T12:07:15.908 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connlimit.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_connmark.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_conntrack.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_cpu.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dccp.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_devgroup.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_dscp.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ecn.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_esp.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hashlimit.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_helper.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_hl.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipcomp.ko 2024-09-05T12:07:15.909 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_iprange.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_ipvs.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_l2tp.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_length.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_limit.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mac.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_mark.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_multiport.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nat.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_nfacct.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_osf.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_owner.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_physdev.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_pkttype.ko 2024-09-05T12:07:15.910 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_policy.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_quota.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_rateest.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_realm.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_recent.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_sctp.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_set.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_socket.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_state.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_statistic.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_string.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_tcpmss.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_time.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netfilter/xt_u32.ko 2024-09-05T12:07:15.911 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/netlink/netlink_diag.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/nsh/nsh.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/openvswitch.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-geneve.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-gre.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/openvswitch/vport-vxlan.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/packet/af_packet_diag.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_rdma.ko 2024-09-05T12:07:15.912 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/rds/rds_tcp.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_csum.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_gact.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_mirred.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_nat.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_pedit.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_police.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_simple.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_skbedit.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/act_vlan.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_basic.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_bpf.ko 2024-09-05T12:07:15.913 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_flow.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_fw.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_route.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/cls_u32.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_cmp.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_ipset.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_meta.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_nbyte.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_text.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/em_u32.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_choke.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_codel.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_drr.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_fq.ko 2024-09-05T12:07:15.914 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_gred.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hfsc.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_hhf.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_htb.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_ingress.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_mqprio_lib.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_multiq.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_netem.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_pie.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_plug.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_prio.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_qfq.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_red.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfb.ko 2024-09-05T12:07:15.915 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_sfq.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_tbf.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sched/sch_teql.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sctp/sctp_diag.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/sunrpc.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc 2024-09-05T12:07:15.916 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/tipc/tipc_diag.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/unix/unix_diag.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security 2024-09-05T12:07:15.917 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/kernel/security/keys/trusted-keys/trusted.ko 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.alias.bin 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.alias.bin 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.bin 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.builtin.modinfo 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep.bin 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.devname 2024-09-05T12:07:15.918 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.order 2024-09-05T12:07:15.919 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.softdep 2024-09-05T12:07:15.919 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols 2024-09-05T12:07:15.919 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.symbols.bin 2024-09-05T12:07:15.919 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.weakdep 2024-09-05T12:07:15.919 INFO:teuthology.orchestra.run.smithi071.stdout:/lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-09-05T12:07:15.922 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.923 DEBUG:teuthology.orchestra.run.smithi071:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout:postinstall scriptlet (using /bin/sh): 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g8ad8b8b16098 /lib/modules/6.11.0-rc5-g8ad8b8b16098/vmlinuz 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout:for file in vmlinuz System.map config; do 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098"; then 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: cp "/lib/modules/6.11.0-rc5-g8ad8b8b16098/${file}" "/boot/${file}-6.11.0-rc5-g8ad8b8b16098" 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout: fi 2024-09-05T12:07:15.947 INFO:teuthology.orchestra.run.smithi071.stdout:done 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -d "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" ] && \ 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" >/dev/null 2>&1; then 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: cp -r "/lib/modules/6.11.0-rc5-g8ad8b8b16098/dtb" "/boot/dtb-6.11.0-rc5-g8ad8b8b16098" 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g8ad8b8b16098/modules.dep" ]; then 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout: /usr/sbin/depmod 6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:kernel-install remove 6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:15.948 INFO:teuthology.orchestra.run.smithi071.stdout:fi 2024-09-05T12:07:15.949 DEBUG:teuthology.orchestra.run.smithi071:> sudo rpm -qi grub2-tools 2024-09-05T12:07:16.024 INFO:teuthology.orchestra.run.smithi071.stdout:Name : grub2-tools 2024-09-05T12:07:16.024 INFO:teuthology.orchestra.run.smithi071.stdout:Epoch : 1 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Version : 2.06 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Release : 82.el9 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Architecture: x86_64 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Group : Unspecified 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Size : 8274670 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:License : GPLv3+ 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Packager : builder@centos.org 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Vendor : CentOS 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-05T12:07:16.025 INFO:teuthology.orchestra.run.smithi071.stdout:Summary : Support tools for GRUB. 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout:Description : 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout:hardware devices. 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:07:16.026 INFO:teuthology.orchestra.run.smithi071.stdout:This subpackage provides tools for support of all platforms. 2024-09-05T12:07:16.027 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-05T12:07:16.027 INFO:teuthology.task.kernel:Updating grub on smithi071 to boot 6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:16.027 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-05T12:07:16.569 INFO:teuthology.orchestra.run.smithi071.stderr:Generating grub configuration file ... 2024-09-05T12:07:17.945 INFO:teuthology.orchestra.run.smithi071.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-05T12:07:17.974 INFO:teuthology.orchestra.run.smithi071.stderr:done 2024-09-05T12:07:17.976 DEBUG:teuthology.orchestra.run.smithi071:> mktemp 2024-09-05T12:07:17.992 INFO:teuthology.orchestra.run.smithi071.stdout:/tmp/tmp.HpgPWJLIbe 2024-09-05T12:07:17.992 DEBUG:teuthology.orchestra.run.smithi071:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HpgPWJLIbe 2024-09-05T12:07:18.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0666 /tmp/tmp.HpgPWJLIbe 2024-09-05T12:07:18.201 DEBUG:teuthology.orchestra.remote:smithi071:/tmp/tmp.HpgPWJLIbe is 6KB 2024-09-05T12:07:18.250 DEBUG:teuthology.orchestra.run.smithi071:> rm -fr /tmp/tmp.HpgPWJLIbe 2024-09-05T12:07:18.264 DEBUG:teuthology.orchestra.run.smithi071:> sudo /bin/ls /boot/loader/entries || true 2024-09-05T12:07:18.327 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-05T12:07:18.327 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-05T12:07:18.328 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-05T12:07:18.328 INFO:teuthology.orchestra.run.smithi071.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098.conf 2024-09-05T12:07:18.329 DEBUG:teuthology.orchestra.run.smithi071:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:07:18.477 DEBUG:teuthology.orchestra.run.smithi071:> sudo shutdown -r now 2024-09-05T12:07:48.480 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-05T12:07:48.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-05T12:07:48.482 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:08:06.973 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-05T12:08:15.974 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-05T12:08:15.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:08:19.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.71 2024-09-05T12:08:31.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-05T12:08:31.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:09:02.914 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-05T12:09:03.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-05T12:09:03.511 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g8ad8b8b16098"... 2024-09-05T12:09:03.511 DEBUG:teuthology.orchestra.run.smithi071:> uname -r 2024-09-05T12:09:03.546 INFO:teuthology.orchestra.run.smithi071.stdout:6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:09:03.546 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g8ad8b8b16098 vs 6.11.0-rc5-g8ad8b8b16098 2024-09-05T12:09:03.546 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-05T12:09:03.546 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-05T12:09:04.547 DEBUG:teuthology.parallel:result is None 2024-09-05T12:09:04.548 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-05T12:09:04.554 INFO:teuthology.task.internal:Creating test directory... 2024-09-05T12:09:04.555 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T12:09:04.559 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T12:09:04.561 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-05T12:09:04.590 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-05T12:09:04.597 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-05T12:09:04.604 INFO:teuthology.task.internal:Creating archive directory... 2024-09-05T12:09:04.604 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T12:09:04.634 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T12:09:04.636 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-05T12:09:04.766 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-05T12:09:04.772 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-05T12:09:04.772 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T12:09:04.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:09:04.790 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-05T12:09:04.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:09:04.807 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T12:09:04.811 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T12:09:04.832 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-05T12:09:04.881 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T12:09:04.897 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T12:09:04.897 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T12:09:04.914 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T12:09:04.940 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T12:09:05.012 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-05T12:09:05.015 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-05T12:09:05.025 INFO:teuthology.task.internal:Configuring sudo... 2024-09-05T12:09:05.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T12:09:05.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T12:09:05.032 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-05T12:09:05.082 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-05T12:09:05.091 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-05T12:09:05.091 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T12:09:05.095 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T12:09:05.099 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-05T12:09:05.125 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T12:09:05.222 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T12:09:05.311 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T12:09:05.367 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T12:09:05.443 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:09:05.443 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T12:09:05.540 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T12:09:05.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T12:09:05.650 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T12:09:05.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T12:09:05.736 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:09:05.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T12:09:05.802 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T12:09:05.849 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-05T12:09:05.905 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T12:09:05.983 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-05T12:09:06.031 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:09:06.032 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-05T12:09:06.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-09-05T12:09:06.102 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-05T12:09:06.104 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-09-05T12:09:06.141 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:09:06.181 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:09:06.214 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:09:06.614 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-05T12:09:06.621 INFO:teuthology.task.internal:Starting timer... 2024-09-05T12:09:06.622 INFO:teuthology.run_tasks:Running task pcp... 2024-09-05T12:09:06.631 INFO:teuthology.run_tasks:Running task selinux... 2024-09-05T12:09:06.640 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-05T12:09:06.640 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-09-05T12:09:06.722 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-09-05T12:09:06.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-05T12:09:06.807 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-05T12:09:06.809 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-09-05T12:09:06.863 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-09-05T12:09:06.865 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-05T12:09:06.865 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-09-05T12:09:06.908 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-09-05T12:09:06.908 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-05T12:09:06.928 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-05T12:09:06.928 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-09-05T12:09:06.956 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-09-05T12:09:06.956 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-09-05T12:09:06.956 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:09:07.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:09:07.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:09:07.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:09:07.046 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:09:07.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:09:07.072 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-05T12:09:07.072 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-09-05T12:09:07.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-05T12:09:07.215 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-09-05T12:09:07.245 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-05T12:09:07.254 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-05T12:09:07.266 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-05T12:09:07.267 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi071.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-05T12:13:41.356 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-09-05T12:13:41.357 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-05T12:13:41.358 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:13:41.446 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-09-05T12:13:41.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-09-05T12:13:41.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-05T12:13:41.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:13:41.638 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-05T12:13:41.711 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-05T12:13:41.712 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-05T12:13:41.712 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-05T12:13:41.791 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-05T12:13:41.865 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-05T12:13:41.866 INFO:teuthology.run_tasks:Running task clock... 2024-09-05T12:13:41.877 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-05T12:13:41.877 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T12:13:41.878 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:13:41.882 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T12:13:41.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:13:41.884 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-05T12:13:41.884 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:13:41.910 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T12:13:41.924 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T12:13:41.937 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T12:13:41.939 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-05T12:13:41.948 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-05T12:13:41.952 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T12:13:41.959 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-05T12:13:41.968 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-05T12:13:41.973 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T12:13:41.978 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-09-05T12:13:41.987 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T12:13:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-09-05T12:13:42.003 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T12:13:42.018 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T12:13:42.019 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-09-05T12:13:42.043 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-09-05T12:13:42.044 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:13:42.048 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:13:42.048 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-05T12:13:42.048 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.048 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.048 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.048 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.073 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-05T12:13:42.082 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:13:42.085 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:13:42.086 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-05T12:13:42.086 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.086 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.086 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.086 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.102 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-05T12:13:42.199 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:13:42.206 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:13:42.206 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-05T12:13:42.206 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.206 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.206 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.206 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:13:42.207 INFO:teuthology.run_tasks:Running task install... 2024-09-05T12:13:42.218 DEBUG:teuthology.task.install:project ceph 2024-09-05T12:13:42.218 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac'}} 2024-09-05T12:13:42.219 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac'} 2024-09-05T12:13:42.219 INFO:teuthology.task.install:Using flavor: default 2024-09-05T12:13:42.224 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-05T12:13:42.224 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-05T12:13:42.225 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'tag': None, 'wait_for_package': False} 2024-09-05T12:13:42.225 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:13:42.227 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'tag': None, 'wait_for_package': False} 2024-09-05T12:13:42.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:13:42.228 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'tag': None, 'wait_for_package': False} 2024-09-05T12:13:42.228 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:13:42.369 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/ 2024-09-05T12:13:42.369 INFO:teuthology.task.install.rpm:Package version is 19.1.1-261.ge856d6b2 2024-09-05T12:13:42.377 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/ 2024-09-05T12:13:42.377 INFO:teuthology.task.install.rpm:Package version is 19.1.1-261.ge856d6b2 2024-09-05T12:13:42.387 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/ 2024-09-05T12:13:42.388 INFO:teuthology.task.install.rpm:Package version is 19.1.1-261.ge856d6b2 2024-09-05T12:13:42.480 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T12:13:42.481 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:13:42.481 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T12:13:42.486 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T12:13:42.486 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:13:42.486 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T12:13:42.494 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-05T12:13:42.494 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:13:42.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-05T12:13:42.513 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T12:13:42.514 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e856d6b2462509a377ef39149a05475ed006ebac/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T12:13:42.521 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T12:13:42.521 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e856d6b2462509a377ef39149a05475ed006ebac/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T12:13:42.537 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-05T12:13:42.538 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e856d6b2462509a377ef39149a05475ed006ebac/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-05T12:13:42.581 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T12:13:42.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T12:13:42.635 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-05T12:13:42.658 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T12:13:42.662 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T12:13:42.714 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-05T12:13:42.722 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-09-05T12:13:42.723 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-09-05T12:13:42.726 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-05T12:13:42.727 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-05T12:13:42.769 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-09-05T12:13:42.772 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-09-05T12:13:42.950 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-09-05T12:13:42.953 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-05T12:13:42.977 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T12:13:42.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T12:13:43.039 INFO:teuthology.orchestra.run.smithi071.stdout:45 files removed 2024-09-05T12:13:43.072 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-05T12:13:43.523 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-05T12:13:43.526 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-05T12:13:43.799 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 249 kB/s | 84 kB 00:00 2024-09-05T12:13:43.810 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-05T12:13:43.815 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-05T12:13:44.082 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-05T12:13:44.085 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-05T12:13:44.158 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-09-05T12:13:44.509 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-09-05T12:13:44.599 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-05T12:13:44.694 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-05T12:13:45.651 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 7.4 MB/s | 8.2 MB 00:01 2024-09-05T12:13:46.553 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-05T12:13:47.437 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 31 MB/s | 20 MB 00:00 2024-09-05T12:13:50.533 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 4.2 MB/s | 20 MB 00:04 2024-09-05T12:13:51.979 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-05T12:13:52.204 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-05T12:13:53.547 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-05T12:13:53.688 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 19 kB 00:00 2024-09-05T12:13:54.809 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-05T12:13:54.817 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-05T12:13:54.842 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-05T12:13:56.276 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-05T12:13:57.181 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-05T12:14:00.273 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-05T12:14:01.239 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-05T12:14:02.517 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T12:14:02.517 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T12:14:02.529 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T12:14:02.530 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T12:14:02.531 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T12:14:02.531 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T12:14:02.533 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T12:14:02.535 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T12:14:02.536 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T12:14:02.597 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-05T12:14:02.614 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 6.6 k 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.4 M 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 893 k 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 145 k 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 1.5 M 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 146 k 2024-09-05T12:14:02.615 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 3.6 M 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 7.4 M 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 49 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 11 M 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 49 M 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 285 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 765 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 222 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 27 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 18 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T12:14:02.616 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 32 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 749 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 127 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 162 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 320 k 2024-09-05T12:14:02.617 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 300 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 99 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 85 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.1 M 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 170 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.4 M 2024-09-05T12:14:02.618 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.2 M 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 21 M 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 32 k 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 2.4 M 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 249 k 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 4.7 M 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 17 M 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 16 k 2024-09-05T12:14:02.619 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 25 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 163 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T12:14:02.620 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 503 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.3 M 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T12:14:02.621 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 45 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 136 k 2024-09-05T12:14:02.622 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T12:14:02.623 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T12:14:02.624 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T12:14:02.625 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T12:14:02.626 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T12:14:02.627 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout:====================================================================================== 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 6 Packages 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:02.628 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 222 M 2024-09-05T12:14:02.629 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-05T12:14:02.636 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-09-05T12:14:03.041 INFO:teuthology.orchestra.run.smithi174.stdout:(1/154): ceph-19.1.1-261.ge856d6b2.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-05T12:14:03.233 INFO:teuthology.orchestra.run.smithi174.stdout:(2/154): ceph-fuse-19.1.1-261.ge856d6b2.el9.x86 4.6 MB/s | 893 kB 00:00 2024-09-05T12:14:03.325 INFO:teuthology.orchestra.run.smithi174.stdout:(3/154): ceph-immutable-object-cache-19.1.1-261 1.5 MB/s | 145 kB 00:00 2024-09-05T12:14:03.400 INFO:teuthology.orchestra.run.smithi174.stdout:(4/154): ceph-base-19.1.1-261.ge856d6b2.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-05T12:14:03.492 INFO:teuthology.orchestra.run.smithi174.stdout:(5/154): ceph-mgr-19.1.1-261.ge856d6b2.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-05T12:14:03.567 INFO:teuthology.orchestra.run.smithi174.stdout:(6/154): ceph-mds-19.1.1-261.ge856d6b2.el9.x86_ 9.8 MB/s | 2.4 MB 00:00 2024-09-05T12:14:03.721 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T12:14:03.721 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T12:14:03.735 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T12:14:03.736 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T12:14:03.737 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T12:14:03.738 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T12:14:03.740 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T12:14:03.743 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T12:14:03.744 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T12:14:03.751 INFO:teuthology.orchestra.run.smithi174.stdout:(7/154): ceph-mon-19.1.1-261.ge856d6b2.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-05T12:14:03.809 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-09-05T12:14:03.833 INFO:teuthology.orchestra.run.smithi071.stdout:====================================================================================== 2024-09-05T12:14:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-09-05T12:14:03.833 INFO:teuthology.orchestra.run.smithi071.stdout:====================================================================================== 2024-09-05T12:14:03.833 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-09-05T12:14:03.833 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 6.6 k 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.4 M 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 893 k 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 145 k 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 1.5 M 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 146 k 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 3.6 M 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 7.4 M 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 49 k 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 11 M 2024-09-05T12:14:03.834 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 49 M 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 285 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 765 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 222 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 27 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 18 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 32 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 749 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 127 k 2024-09-05T12:14:03.835 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 162 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 320 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 300 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 99 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 85 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.1 M 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 170 k 2024-09-05T12:14:03.836 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.4 M 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.2 M 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 21 M 2024-09-05T12:14:03.837 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 32 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 2.4 M 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 249 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 4.7 M 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 17 M 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 16 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 25 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T12:14:03.838 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 163 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 503 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.3 M 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T12:14:03.839 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T12:14:03.840 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 45 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 136 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T12:14:03.841 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T12:14:03.842 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T12:14:03.843 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T12:14:03.844 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T12:14:03.845 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout:====================================================================================== 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout:Install 148 Packages 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 6 Packages 2024-09-05T12:14:03.846 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:14:03.847 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 222 M 2024-09-05T12:14:03.847 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-09-05T12:14:04.193 INFO:teuthology.orchestra.run.smithi174.stdout:(8/154): ceph-common-19.1.1-261.ge856d6b2.el9.x 17 MB/s | 21 MB 00:01 2024-09-05T12:14:04.243 INFO:teuthology.orchestra.run.smithi174.stdout:(9/154): ceph-selinux-19.1.1-261.ge856d6b2.el9. 508 kB/s | 25 kB 00:00 2024-09-05T12:14:04.360 INFO:teuthology.orchestra.run.smithi174.stdout:(10/154): ceph-radosgw-19.1.1-261.ge856d6b2.el9 17 MB/s | 11 MB 00:00 2024-09-05T12:14:04.435 INFO:teuthology.orchestra.run.smithi174.stdout:(11/154): cephfs-mirror-19.1.1-261.ge856d6b2.el 2.9 MB/s | 222 kB 00:00 2024-09-05T12:14:04.502 INFO:teuthology.orchestra.run.smithi174.stdout:(12/154): cephfs-shell-19.1.1-261.ge856d6b2.el9 407 kB/s | 27 kB 00:00 2024-09-05T12:14:04.569 INFO:teuthology.orchestra.run.smithi174.stdout:(13/154): libcephfs-devel-19.1.1-261.ge856d6b2. 477 kB/s | 32 kB 00:00 2024-09-05T12:14:04.653 INFO:teuthology.orchestra.run.smithi174.stdout:(14/154): libcephfs2-19.1.1-261.ge856d6b2.el9.x 8.8 MB/s | 749 kB 00:00 2024-09-05T12:14:04.728 INFO:teuthology.orchestra.run.smithi174.stdout:(15/154): libcephsqlite-19.1.1-261.ge856d6b2.el 2.1 MB/s | 163 kB 00:00 2024-09-05T12:14:04.765 INFO:teuthology.orchestra.run.smithi071.stdout:(1/154): ceph-19.1.1-261.ge856d6b2.el9.x86_64.r 36 kB/s | 6.6 kB 00:00 2024-09-05T12:14:04.854 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T12:14:04.854 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-05T12:14:04.865 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-05T12:14:04.866 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-05T12:14:04.867 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-05T12:14:04.868 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-05T12:14:04.869 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-05T12:14:04.872 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-05T12:14:04.873 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-05T12:14:04.895 INFO:teuthology.orchestra.run.smithi174.stdout:(16/154): ceph-osd-19.1.1-261.ge856d6b2.el9.x86 13 MB/s | 17 MB 00:01 2024-09-05T12:14:04.928 INFO:teuthology.orchestra.run.smithi174.stdout:(17/154): librados-devel-19.1.1-261.ge856d6b2.e 633 kB/s | 127 kB 00:00 2024-09-05T12:14:04.934 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-05T12:14:04.949 INFO:teuthology.orchestra.run.smithi071.stdout:(2/154): ceph-fuse-19.1.1-261.ge856d6b2.el9.x86 4.8 MB/s | 893 kB 00:00 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 6.6 k 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.4 M 2024-09-05T12:14:04.954 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 893 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 145 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 1.5 M 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 146 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 3.6 M 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 7.4 M 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 49 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 11 M 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 49 M 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 285 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 765 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 222 k 2024-09-05T12:14:04.955 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 27 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 18 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 32 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 749 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 127 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-05T12:14:04.956 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-4.el9 appstream 4.4 M 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 162 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 320 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 300 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 99 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 85 k 2024-09-05T12:14:04.957 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.1 M 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 170 k 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.48.10-1.el9 baseos 2.3 M 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.48.10-1.el9 baseos 1.8 M 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.48.10-1.el9 baseos 40 k 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.48.10-1.el9 baseos 248 k 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.4 M 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 3.2 M 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-05T12:14:04.958 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 21 M 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 32 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 2.4 M 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 249 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 4.7 M 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 17 M 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.1.1-261.ge856d6b2.el9 ceph-noarch 16 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 25 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-05T12:14:04.959 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 163 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-05T12:14:04.960 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 503 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 5.3 M 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-05T12:14:04.961 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 45 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.1.1-261.ge856d6b2.el9 ceph 136 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-05T12:14:04.962 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-05T12:14:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-05T12:14:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-05T12:14:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-05T12:14:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-05T12:14:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-05T12:14:04.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-05T12:14:04.964 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-05T12:14:04.965 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-05T12:14:04.966 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-05T12:14:04.967 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-05T12:14:04.968 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout:====================================================================================== 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 6 Packages 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:04.969 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 222 M 2024-09-05T12:14:04.970 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-05T12:14:04.970 INFO:teuthology.orchestra.run.smithi174.stdout:(18/154): libradosstriper1-19.1.1-261.ge856d6b2 6.6 MB/s | 503 kB 00:00 2024-09-05T12:14:05.241 INFO:teuthology.orchestra.run.smithi071.stdout:(3/154): ceph-base-19.1.1-261.ge856d6b2.el9.x86 8.2 MB/s | 5.4 MB 00:00 2024-09-05T12:14:05.304 INFO:teuthology.orchestra.run.smithi174.stdout:(19/154): librgw2-19.1.1-261.ge856d6b2.el9.x86_ 14 MB/s | 5.3 MB 00:00 2024-09-05T12:14:05.379 INFO:teuthology.orchestra.run.smithi174.stdout:(20/154): python3-ceph-common-19.1.1-261.ge856d 1.8 MB/s | 136 kB 00:00 2024-09-05T12:14:05.404 INFO:teuthology.orchestra.run.smithi174.stdout:(21/154): python3-ceph-argparse-19.1.1-261.ge85 104 kB/s | 45 kB 00:00 2024-09-05T12:14:05.425 INFO:teuthology.orchestra.run.smithi071.stdout:(4/154): ceph-mds-19.1.1-261.ge856d6b2.el9.x86_ 13 MB/s | 2.4 MB 00:00 2024-09-05T12:14:05.438 INFO:teuthology.orchestra.run.smithi174.stdout:(22/154): python3-cephfs-19.1.1-261.ge856d6b2.e 2.7 MB/s | 162 kB 00:00 2024-09-05T12:14:05.475 INFO:teuthology.orchestra.run.smithi071.stdout:(5/154): ceph-immutable-object-cache-19.1.1-261 276 kB/s | 145 kB 00:00 2024-09-05T12:14:05.481 INFO:teuthology.orchestra.run.smithi174.stdout:(23/154): python3-rados-19.1.1-261.ge856d6b2.el 4.1 MB/s | 320 kB 00:00 2024-09-05T12:14:05.522 INFO:teuthology.orchestra.run.smithi174.stdout:(24/154): python3-rbd-19.1.1-261.ge856d6b2.el9. 3.5 MB/s | 300 kB 00:00 2024-09-05T12:14:05.546 INFO:teuthology.orchestra.run.smithi123.stdout:(1/154): ceph-19.1.1-261.ge856d6b2.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-09-05T12:14:05.567 INFO:teuthology.orchestra.run.smithi071.stdout:(6/154): ceph-mgr-19.1.1-261.ge856d6b2.el9.x86_ 11 MB/s | 1.5 MB 00:00 2024-09-05T12:14:05.572 INFO:teuthology.orchestra.run.smithi174.stdout:(25/154): python3-rgw-19.1.1-261.ge856d6b2.el9. 1.1 MB/s | 99 kB 00:00 2024-09-05T12:14:05.606 INFO:teuthology.orchestra.run.smithi174.stdout:(26/154): rbd-fuse-19.1.1-261.ge856d6b2.el9.x86 1.0 MB/s | 85 kB 00:00 2024-09-05T12:14:05.706 INFO:teuthology.orchestra.run.smithi174.stdout:(27/154): rbd-nbd-19.1.1-261.ge856d6b2.el9.x86_ 1.7 MB/s | 170 kB 00:00 2024-09-05T12:14:05.807 INFO:teuthology.orchestra.run.smithi174.stdout:(28/154): ceph-grafana-dashboards-19.1.1-261.ge 317 kB/s | 32 kB 00:00 2024-09-05T12:14:05.907 INFO:teuthology.orchestra.run.smithi174.stdout:(29/154): ceph-mgr-cephadm-19.1.1-261.ge856d6b2 1.4 MB/s | 146 kB 00:00 2024-09-05T12:14:06.109 INFO:teuthology.orchestra.run.smithi071.stdout:(7/154): ceph-mon-19.1.1-261.ge856d6b2.el9.x86_ 7.4 MB/s | 4.7 MB 00:00 2024-09-05T12:14:06.165 INFO:teuthology.orchestra.run.smithi174.stdout:(30/154): rbd-mirror-19.1.1-261.ge856d6b2.el9.x 5.2 MB/s | 3.1 MB 00:00 2024-09-05T12:14:06.354 INFO:teuthology.orchestra.run.smithi123.stdout:(2/154): ceph-fuse-19.1.1-261.ge856d6b2.el9.x86 1.1 MB/s | 893 kB 00:00 2024-09-05T12:14:06.491 INFO:teuthology.orchestra.run.smithi174.stdout:(31/154): ceph-mgr-dashboard-19.1.1-261.ge856d6 6.1 MB/s | 3.6 MB 00:00 2024-09-05T12:14:06.522 INFO:teuthology.orchestra.run.smithi123.stdout:(3/154): ceph-immutable-object-cache-19.1.1-261 872 kB/s | 145 kB 00:00 2024-09-05T12:14:06.591 INFO:teuthology.orchestra.run.smithi174.stdout:(32/154): ceph-mgr-modules-core-19.1.1-261.ge85 2.4 MB/s | 249 kB 00:00 2024-09-05T12:14:06.650 INFO:teuthology.orchestra.run.smithi174.stdout:(33/154): ceph-mgr-rook-19.1.1-261.ge856d6b2.el 848 kB/s | 49 kB 00:00 2024-09-05T12:14:06.708 INFO:teuthology.orchestra.run.smithi174.stdout:(34/154): ceph-prometheus-alerts-19.1.1-261.ge8 282 kB/s | 16 kB 00:00 2024-09-05T12:14:06.751 INFO:teuthology.orchestra.run.smithi071.stdout:(8/154): ceph-common-19.1.1-261.ge856d6b2.el9.x 9.8 MB/s | 21 MB 00:02 2024-09-05T12:14:06.792 INFO:teuthology.orchestra.run.smithi174.stdout:(35/154): ceph-volume-19.1.1-261.ge856d6b2.el9. 3.3 MB/s | 285 kB 00:00 2024-09-05T12:14:06.826 INFO:teuthology.orchestra.run.smithi071.stdout:(9/154): ceph-selinux-19.1.1-261.ge856d6b2.el9. 338 kB/s | 25 kB 00:00 2024-09-05T12:14:07.113 INFO:teuthology.orchestra.run.smithi123.stdout:(4/154): ceph-mds-19.1.1-261.ge856d6b2.el9.x86_ 4.0 MB/s | 2.4 MB 00:00 2024-09-05T12:14:07.175 INFO:teuthology.orchestra.run.smithi071.stdout:(10/154): ceph-osd-19.1.1-261.ge856d6b2.el9.x86 10 MB/s | 17 MB 00:01 2024-09-05T12:14:07.217 INFO:teuthology.orchestra.run.smithi174.stdout:(36/154): ceph-test-19.1.1-261.ge856d6b2.el9.x8 16 MB/s | 49 MB 00:02 2024-09-05T12:14:07.259 INFO:teuthology.orchestra.run.smithi174.stdout:(37/154): cephadm-19.1.1-261.ge856d6b2.el9.noar 1.6 MB/s | 765 kB 00:00 2024-09-05T12:14:07.284 INFO:teuthology.orchestra.run.smithi174.stdout:(38/154): cephfs-top-19.1.1-261.ge856d6b2.el9.n 274 kB/s | 18 kB 00:00 2024-09-05T12:14:07.288 INFO:teuthology.orchestra.run.smithi123.stdout:(5/154): ceph-mgr-19.1.1-261.ge856d6b2.el9.x86_ 8.7 MB/s | 1.5 MB 00:00 2024-09-05T12:14:07.318 INFO:teuthology.orchestra.run.smithi071.stdout:(11/154): ceph-radosgw-19.1.1-261.ge856d6b2.el9 8.8 MB/s | 11 MB 00:01 2024-09-05T12:14:07.343 INFO:teuthology.orchestra.run.smithi071.stdout:(12/154): cephfs-mirror-19.1.1-261.ge856d6b2.el 1.3 MB/s | 222 kB 00:00 2024-09-05T12:14:07.369 INFO:teuthology.orchestra.run.smithi071.stdout:(13/154): cephfs-shell-19.1.1-261.ge856d6b2.el9 541 kB/s | 27 kB 00:00 2024-09-05T12:14:07.385 INFO:teuthology.orchestra.run.smithi174.stdout:(39/154): ceph-mgr-diskprediction-local-19.1.1- 6.1 MB/s | 7.4 MB 00:01 2024-09-05T12:14:07.394 INFO:teuthology.orchestra.run.smithi071.stdout:(14/154): libcephfs-devel-19.1.1-261.ge856d6b2. 634 kB/s | 32 kB 00:00 2024-09-05T12:14:07.410 INFO:teuthology.orchestra.run.smithi174.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-05T12:14:07.452 INFO:teuthology.orchestra.run.smithi071.stdout:(15/154): libcephfs2-19.1.1-261.ge856d6b2.el9.x 8.8 MB/s | 749 kB 00:00 2024-09-05T12:14:07.478 INFO:teuthology.orchestra.run.smithi071.stdout:(16/154): libcephsqlite-19.1.1-261.ge856d6b2.el 1.9 MB/s | 163 kB 00:00 2024-09-05T12:14:07.493 INFO:teuthology.orchestra.run.smithi174.stdout:(41/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-05T12:14:07.503 INFO:teuthology.orchestra.run.smithi071.stdout:(17/154): librados-devel-19.1.1-261.ge856d6b2.e 2.5 MB/s | 127 kB 00:00 2024-09-05T12:14:07.535 INFO:teuthology.orchestra.run.smithi174.stdout:(42/154): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-05T12:14:07.561 INFO:teuthology.orchestra.run.smithi174.stdout:(43/154): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-05T12:14:07.570 INFO:teuthology.orchestra.run.smithi071.stdout:(18/154): libradosstriper1-19.1.1-261.ge856d6b2 5.3 MB/s | 503 kB 00:00 2024-09-05T12:14:07.586 INFO:teuthology.orchestra.run.smithi174.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-05T12:14:07.611 INFO:teuthology.orchestra.run.smithi174.stdout:(45/154): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-05T12:14:07.620 INFO:teuthology.orchestra.run.smithi071.stdout:(19/154): python3-ceph-argparse-19.1.1-261.ge85 906 kB/s | 45 kB 00:00 2024-09-05T12:14:07.636 INFO:teuthology.orchestra.run.smithi174.stdout:(46/154): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-05T12:14:07.639 INFO:teuthology.orchestra.run.smithi123.stdout:(6/154): ceph-mon-19.1.1-261.ge856d6b2.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-09-05T12:14:07.678 INFO:teuthology.orchestra.run.smithi174.stdout:(47/154): python3-urllib3-1.26.5-6.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-09-05T12:14:07.679 INFO:teuthology.orchestra.run.smithi071.stdout:(20/154): python3-ceph-common-19.1.1-261.ge856d 2.3 MB/s | 136 kB 00:00 2024-09-05T12:14:07.712 INFO:teuthology.orchestra.run.smithi174.stdout:(48/154): python3-cryptography-36.0.1-4.el9.x86 5.6 MB/s | 1.2 MB 00:00 2024-09-05T12:14:07.746 INFO:teuthology.orchestra.run.smithi071.stdout:(21/154): python3-cephfs-19.1.1-261.ge856d6b2.e 2.4 MB/s | 162 kB 00:00 2024-09-05T12:14:07.812 INFO:teuthology.orchestra.run.smithi174.stdout:(49/154): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-05T12:14:07.821 INFO:teuthology.orchestra.run.smithi071.stdout:(22/154): python3-rados-19.1.1-261.ge856d6b2.el 4.2 MB/s | 320 kB 00:00 2024-09-05T12:14:07.854 INFO:teuthology.orchestra.run.smithi174.stdout:(50/154): boost-program-options-1.75.0-8.el9.x8 753 kB/s | 107 kB 00:00 2024-09-05T12:14:07.880 INFO:teuthology.orchestra.run.smithi123.stdout:(7/154): ceph-base-19.1.1-261.ge856d6b2.el9.x86 2.1 MB/s | 5.4 MB 00:02 2024-09-05T12:14:07.896 INFO:teuthology.orchestra.run.smithi174.stdout:(51/154): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-09-05T12:14:07.922 INFO:teuthology.orchestra.run.smithi174.stdout:(52/154): elfutils-libelf-devel-0.191-4.el9.x86 199 kB/s | 22 kB 00:00 2024-09-05T12:14:07.940 INFO:teuthology.orchestra.run.smithi071.stdout:(23/154): librgw2-19.1.1-261.ge856d6b2.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-09-05T12:14:07.955 INFO:teuthology.orchestra.run.smithi174.stdout:(53/154): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-09-05T12:14:07.972 INFO:teuthology.orchestra.run.smithi071.stdout:(24/154): python3-rbd-19.1.1-261.ge856d6b2.el9. 2.0 MB/s | 300 kB 00:00 2024-09-05T12:14:07.980 INFO:teuthology.orchestra.run.smithi174.stdout:(54/154): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-05T12:14:07.997 INFO:teuthology.orchestra.run.smithi071.stdout:(25/154): python3-rgw-19.1.1-261.ge856d6b2.el9. 1.7 MB/s | 99 kB 00:00 2024-09-05T12:14:08.031 INFO:teuthology.orchestra.run.smithi071.stdout:(26/154): rbd-fuse-19.1.1-261.ge856d6b2.el9.x86 1.4 MB/s | 85 kB 00:00 2024-09-05T12:14:08.064 INFO:teuthology.orchestra.run.smithi174.stdout:(55/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-09-05T12:14:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 104 kB/s | 11 kB 00:00 2024-09-05T12:14:08.113 INFO:teuthology.orchestra.run.smithi174.stdout:(57/154): flexiblas-openblas-openmp-3.0.4-8.el9 115 kB/s | 18 kB 00:00 2024-09-05T12:14:08.122 INFO:teuthology.orchestra.run.smithi071.stdout:(27/154): rbd-nbd-19.1.1-261.ge856d6b2.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-05T12:14:08.163 INFO:teuthology.orchestra.run.smithi174.stdout:(58/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-05T12:14:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:(59/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 451 kB/s | 45 kB 00:00 2024-09-05T12:14:08.198 INFO:teuthology.orchestra.run.smithi071.stdout:(28/154): ceph-grafana-dashboards-19.1.1-261.ge 425 kB/s | 32 kB 00:00 2024-09-05T12:14:08.214 INFO:teuthology.orchestra.run.smithi174.stdout:(60/154): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-05T12:14:08.247 INFO:teuthology.orchestra.run.smithi174.stdout:(61/154): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-05T12:14:08.273 INFO:teuthology.orchestra.run.smithi174.stdout:(62/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 291 kB/s | 24 kB 00:00 2024-09-05T12:14:08.298 INFO:teuthology.orchestra.run.smithi174.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-05T12:14:08.323 INFO:teuthology.orchestra.run.smithi174.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-05T12:14:08.336 INFO:teuthology.orchestra.run.smithi071.stdout:(29/154): rbd-mirror-19.1.1-261.ge856d6b2.el9.x 9.2 MB/s | 3.1 MB 00:00 2024-09-05T12:14:08.348 INFO:teuthology.orchestra.run.smithi174.stdout:(65/154): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-05T12:14:08.365 INFO:teuthology.orchestra.run.smithi071.stdout:(30/154): ceph-mgr-cephadm-19.1.1-261.ge856d6b2 876 kB/s | 146 kB 00:00 2024-09-05T12:14:08.373 INFO:teuthology.orchestra.run.smithi174.stdout:(66/154): openblas-0.3.26-2.el9.x86_64.rpm 510 kB/s | 38 kB 00:00 2024-09-05T12:14:08.514 INFO:teuthology.orchestra.run.smithi123.stdout:(8/154): ceph-osd-19.1.1-261.ge856d6b2.el9.x86_ 19 MB/s | 17 MB 00:00 2024-09-05T12:14:08.515 INFO:teuthology.orchestra.run.smithi174.stdout:(67/154): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-05T12:14:08.564 INFO:teuthology.orchestra.run.smithi123.stdout:(9/154): ceph-selinux-19.1.1-261.ge856d6b2.el9. 507 kB/s | 25 kB 00:00 2024-09-05T12:14:08.582 INFO:teuthology.orchestra.run.smithi174.stdout:(68/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-05T12:14:08.590 INFO:teuthology.orchestra.run.smithi071.stdout:(31/154): ceph-mgr-dashboard-19.1.1-261.ge856d6 14 MB/s | 3.6 MB 00:00 2024-09-05T12:14:08.624 INFO:teuthology.orchestra.run.smithi174.stdout:(69/154): postgresql-13.16-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-09-05T12:14:08.650 INFO:teuthology.orchestra.run.smithi174.stdout:(70/154): postgresql-contrib-13.16-1.el9.x86_64 6.6 MB/s | 910 kB 00:00 2024-09-05T12:14:08.675 INFO:teuthology.orchestra.run.smithi174.stdout:(71/154): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-05T12:14:08.706 INFO:teuthology.orchestra.run.smithi071.stdout:(32/154): ceph-mgr-modules-core-19.1.1-261.ge85 2.1 MB/s | 249 kB 00:00 2024-09-05T12:14:08.742 INFO:teuthology.orchestra.run.smithi174.stdout:(72/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-05T12:14:08.775 INFO:teuthology.orchestra.run.smithi174.stdout:(73/154): protobuf-3.14.0-13.el9.x86_64.rpm 8.0 MB/s | 1.0 MB 00:00 2024-09-05T12:14:08.791 INFO:teuthology.orchestra.run.smithi071.stdout:(33/154): ceph-mgr-rook-19.1.1-261.ge856d6b2.el 587 kB/s | 49 kB 00:00 2024-09-05T12:14:08.851 INFO:teuthology.orchestra.run.smithi174.stdout:(74/154): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-05T12:14:08.866 INFO:teuthology.orchestra.run.smithi071.stdout:(34/154): ceph-prometheus-alerts-19.1.1-261.ge8 218 kB/s | 16 kB 00:00 2024-09-05T12:14:08.867 INFO:teuthology.orchestra.run.smithi174.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 666 kB/s | 61 kB 00:00 2024-09-05T12:14:08.958 INFO:teuthology.orchestra.run.smithi071.stdout:(35/154): ceph-volume-19.1.1-261.ge856d6b2.el9. 3.0 MB/s | 285 kB 00:00 2024-09-05T12:14:08.959 INFO:teuthology.orchestra.run.smithi174.stdout:(76/154): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-09-05T12:14:08.985 INFO:teuthology.orchestra.run.smithi174.stdout:(77/154): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-05T12:14:09.011 INFO:teuthology.orchestra.run.smithi174.stdout:(78/154): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-05T12:14:09.027 INFO:teuthology.orchestra.run.smithi174.stdout:(79/154): python3-jsonpatch-1.21-16.el9.noarch. 383 kB/s | 26 kB 00:00 2024-09-05T12:14:09.069 INFO:teuthology.orchestra.run.smithi174.stdout:(80/154): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-09-05T12:14:09.085 INFO:teuthology.orchestra.run.smithi071.stdout:(36/154): ceph-mgr-diskprediction-local-19.1.1- 10 MB/s | 7.4 MB 00:00 2024-09-05T12:14:09.094 INFO:teuthology.orchestra.run.smithi174.stdout:(81/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-05T12:14:09.111 INFO:teuthology.orchestra.run.smithi174.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 414 kB/s | 35 kB 00:00 2024-09-05T12:14:09.125 INFO:teuthology.orchestra.run.smithi071.stdout:(37/154): cephadm-19.1.1-261.ge856d6b2.el9.noar 4.5 MB/s | 765 kB 00:00 2024-09-05T12:14:09.161 INFO:teuthology.orchestra.run.smithi174.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-05T12:14:09.167 INFO:teuthology.orchestra.run.smithi071.stdout:(38/154): cephfs-top-19.1.1-261.ge856d6b2.el9.n 222 kB/s | 18 kB 00:00 2024-09-05T12:14:09.186 INFO:teuthology.orchestra.run.smithi174.stdout:(84/154): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-09-05T12:14:09.229 INFO:teuthology.orchestra.run.smithi174.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-05T12:14:09.245 INFO:teuthology.orchestra.run.smithi174.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-09-05T12:14:09.321 INFO:teuthology.orchestra.run.smithi174.stdout:(87/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-05T12:14:09.348 INFO:teuthology.orchestra.run.smithi174.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-09-05T12:14:09.371 INFO:teuthology.orchestra.run.smithi174.stdout:(89/154): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-09-05T12:14:09.413 INFO:teuthology.orchestra.run.smithi174.stdout:(90/154): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-09-05T12:14:09.430 INFO:teuthology.orchestra.run.smithi174.stdout:(91/154): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-05T12:14:09.472 INFO:teuthology.orchestra.run.smithi174.stdout:(92/154): python3-requests-oauthlib-1.3.0-12.el 534 kB/s | 54 kB 00:00 2024-09-05T12:14:09.497 INFO:teuthology.orchestra.run.smithi174.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-09-05T12:14:09.531 INFO:teuthology.orchestra.run.smithi174.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 754 kB/s | 44 kB 00:00 2024-09-05T12:14:09.565 INFO:teuthology.orchestra.run.smithi174.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-05T12:14:09.581 INFO:teuthology.orchestra.run.smithi174.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-09-05T12:14:09.640 INFO:teuthology.orchestra.run.smithi174.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-09-05T12:14:09.673 INFO:teuthology.orchestra.run.smithi174.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-05T12:14:09.751 INFO:teuthology.orchestra.run.smithi071.stdout:(39/154): fuse-2.9.9-16.el9.x86_64.rpm 128 kB/s | 80 kB 00:00 2024-09-05T12:14:09.832 INFO:teuthology.orchestra.run.smithi174.stdout:(99/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-09-05T12:14:09.858 INFO:teuthology.orchestra.run.smithi174.stdout:(100/154): uuid-1.6.2-55.el9.x86_64.rpm 266 kB/s | 58 kB 00:00 2024-09-05T12:14:09.883 INFO:teuthology.orchestra.run.smithi174.stdout:(101/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 458 kB/s | 96 kB 00:00 2024-09-05T12:14:09.908 INFO:teuthology.orchestra.run.smithi174.stdout:(102/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-09-05T12:14:09.933 INFO:teuthology.orchestra.run.smithi174.stdout:(103/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-05T12:14:09.983 INFO:teuthology.orchestra.run.smithi174.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-05T12:14:10.009 INFO:teuthology.orchestra.run.smithi174.stdout:(105/154): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-09-05T12:14:10.043 INFO:teuthology.orchestra.run.smithi071.stdout:(40/154): libgfortran-11.5.0-2.el9.x86_64.rpm 915 kB/s | 801 kB 00:00 2024-09-05T12:14:10.051 INFO:teuthology.orchestra.run.smithi174.stdout:(106/154): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-09-05T12:14:10.076 INFO:teuthology.orchestra.run.smithi174.stdout:(107/154): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-09-05T12:14:10.101 INFO:teuthology.orchestra.run.smithi174.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T12:14:10.127 INFO:teuthology.orchestra.run.smithi174.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-05T12:14:10.169 INFO:teuthology.orchestra.run.smithi174.stdout:(110/154): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-09-05T12:14:10.194 INFO:teuthology.orchestra.run.smithi174.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-05T12:14:10.236 INFO:teuthology.orchestra.run.smithi174.stdout:(112/154): luarocks-3.9.2-5.el9.noarch.rpm 3.6 MB/s | 151 kB 00:00 2024-09-05T12:14:10.286 INFO:teuthology.orchestra.run.smithi174.stdout:(113/154): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-05T12:14:10.311 INFO:teuthology.orchestra.run.smithi174.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T12:14:10.337 INFO:teuthology.orchestra.run.smithi174.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T12:14:10.348 INFO:teuthology.orchestra.run.smithi123.stdout:(10/154): ceph-test-19.1.1-261.ge856d6b2.el9.x8 27 MB/s | 49 MB 00:01 2024-09-05T12:14:10.362 INFO:teuthology.orchestra.run.smithi174.stdout:(116/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-09-05T12:14:10.387 INFO:teuthology.orchestra.run.smithi174.stdout:(117/154): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-05T12:14:10.398 INFO:teuthology.orchestra.run.smithi123.stdout:(11/154): cephfs-mirror-19.1.1-261.ge856d6b2.el 4.4 MB/s | 222 kB 00:00 2024-09-05T12:14:10.412 INFO:teuthology.orchestra.run.smithi174.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-09-05T12:14:10.440 INFO:teuthology.orchestra.run.smithi123.stdout:(12/154): cephfs-shell-19.1.1-261.ge856d6b2.el9 649 kB/s | 27 kB 00:00 2024-09-05T12:14:10.446 INFO:teuthology.orchestra.run.smithi174.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-05T12:14:10.471 INFO:teuthology.orchestra.run.smithi174.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-05T12:14:10.473 INFO:teuthology.orchestra.run.smithi123.stdout:(13/154): libcephfs-devel-19.1.1-261.ge856d6b2. 959 kB/s | 32 kB 00:00 2024-09-05T12:14:10.496 INFO:teuthology.orchestra.run.smithi174.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-05T12:14:10.540 INFO:teuthology.orchestra.run.smithi123.stdout:(14/154): libcephfs2-19.1.1-261.ge856d6b2.el9.x 11 MB/s | 749 kB 00:00 2024-09-05T12:14:10.555 INFO:teuthology.orchestra.run.smithi174.stdout:(122/154): libarrow-9.0.0-11.el9.x86_64.rpm 8.8 MB/s | 4.4 MB 00:00 2024-09-05T12:14:10.580 INFO:teuthology.orchestra.run.smithi174.stdout:(123/154): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-09-05T12:14:10.583 INFO:teuthology.orchestra.run.smithi071.stdout:(41/154): ceph-test-19.1.1-261.ge856d6b2.el9.x8 13 MB/s | 49 MB 00:03 2024-09-05T12:14:10.590 INFO:teuthology.orchestra.run.smithi123.stdout:(15/154): libcephsqlite-19.1.1-261.ge856d6b2.el 3.2 MB/s | 163 kB 00:00 2024-09-05T12:14:10.597 INFO:teuthology.orchestra.run.smithi174.stdout:(124/154): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-09-05T12:14:10.614 INFO:teuthology.orchestra.run.smithi174.stdout:(125/154): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-05T12:14:10.631 INFO:teuthology.orchestra.run.smithi174.stdout:(126/154): python3-jaraco-collections-3.0.0-8.e 690 kB/s | 23 kB 00:00 2024-09-05T12:14:10.641 INFO:teuthology.orchestra.run.smithi123.stdout:(16/154): librados-devel-19.1.1-261.ge856d6b2.e 2.5 MB/s | 127 kB 00:00 2024-09-05T12:14:10.648 INFO:teuthology.orchestra.run.smithi174.stdout:(127/154): python3-jaraco-functools-3.5.0-2.el9 581 kB/s | 19 kB 00:00 2024-09-05T12:14:10.664 INFO:teuthology.orchestra.run.smithi174.stdout:(128/154): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-09-05T12:14:10.681 INFO:teuthology.orchestra.run.smithi174.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-05T12:14:10.691 INFO:teuthology.orchestra.run.smithi123.stdout:(17/154): libradosstriper1-19.1.1-261.ge856d6b2 9.9 MB/s | 503 kB 00:00 2024-09-05T12:14:10.707 INFO:teuthology.orchestra.run.smithi174.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T12:14:10.724 INFO:teuthology.orchestra.run.smithi174.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-05T12:14:10.758 INFO:teuthology.orchestra.run.smithi174.stdout:(132/154): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-09-05T12:14:10.783 INFO:teuthology.orchestra.run.smithi174.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 663 kB/s | 16 kB 00:00 2024-09-05T12:14:10.808 INFO:teuthology.orchestra.run.smithi174.stdout:(134/154): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-05T12:14:10.833 INFO:teuthology.orchestra.run.smithi174.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-05T12:14:10.859 INFO:teuthology.orchestra.run.smithi174.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-05T12:14:10.883 INFO:teuthology.orchestra.run.smithi123.stdout:(18/154): librgw2-19.1.1-261.ge856d6b2.el9.x86_ 28 MB/s | 5.3 MB 00:00 2024-09-05T12:14:10.884 INFO:teuthology.orchestra.run.smithi174.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-05T12:14:10.926 INFO:teuthology.orchestra.run.smithi174.stdout:(138/154): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-05T12:14:10.933 INFO:teuthology.orchestra.run.smithi123.stdout:(19/154): python3-ceph-argparse-19.1.1-261.ge85 908 kB/s | 45 kB 00:00 2024-09-05T12:14:10.951 INFO:teuthology.orchestra.run.smithi174.stdout:(139/154): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-09-05T12:14:10.961 INFO:teuthology.orchestra.run.smithi071.stdout:(42/154): libquadmath-11.5.0-2.el9.x86_64.rpm 158 kB/s | 191 kB 00:01 2024-09-05T12:14:10.976 INFO:teuthology.orchestra.run.smithi174.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-09-05T12:14:10.983 INFO:teuthology.orchestra.run.smithi123.stdout:(20/154): python3-ceph-common-19.1.1-261.ge856d 2.7 MB/s | 136 kB 00:00 2024-09-05T12:14:11.002 INFO:teuthology.orchestra.run.smithi174.stdout:(141/154): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-05T12:14:11.027 INFO:teuthology.orchestra.run.smithi174.stdout:(142/154): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-05T12:14:11.034 INFO:teuthology.orchestra.run.smithi123.stdout:(21/154): python3-cephfs-19.1.1-261.ge856d6b2.e 3.2 MB/s | 162 kB 00:00 2024-09-05T12:14:11.052 INFO:teuthology.orchestra.run.smithi174.stdout:(143/154): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-05T12:14:11.077 INFO:teuthology.orchestra.run.smithi174.stdout:(144/154): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-09-05T12:14:11.084 INFO:teuthology.orchestra.run.smithi123.stdout:(22/154): python3-rados-19.1.1-261.ge856d6b2.el 6.3 MB/s | 320 kB 00:00 2024-09-05T12:14:11.103 INFO:teuthology.orchestra.run.smithi174.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-05T12:14:11.128 INFO:teuthology.orchestra.run.smithi174.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-05T12:14:11.134 INFO:teuthology.orchestra.run.smithi123.stdout:(23/154): python3-rbd-19.1.1-261.ge856d6b2.el9. 5.9 MB/s | 300 kB 00:00 2024-09-05T12:14:11.153 INFO:teuthology.orchestra.run.smithi174.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-05T12:14:11.212 INFO:teuthology.orchestra.run.smithi174.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-05T12:14:11.243 INFO:teuthology.orchestra.run.smithi123.stdout:(24/154): ceph-radosgw-19.1.1-261.ge856d6b2.el9 3.2 MB/s | 11 MB 00:03 2024-09-05T12:14:11.268 INFO:teuthology.orchestra.run.smithi123.stdout:(25/154): python3-rgw-19.1.1-261.ge856d6b2.el9. 746 kB/s | 99 kB 00:00 2024-09-05T12:14:11.269 INFO:teuthology.orchestra.run.smithi071.stdout:(43/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 207 kB/s | 253 kB 00:01 2024-09-05T12:14:11.293 INFO:teuthology.orchestra.run.smithi123.stdout:(26/154): rbd-fuse-19.1.1-261.ge856d6b2.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-05T12:14:11.396 INFO:teuthology.orchestra.run.smithi123.stdout:(27/154): rbd-mirror-19.1.1-261.ge856d6b2.el9.x 25 MB/s | 3.1 MB 00:00 2024-09-05T12:14:11.404 INFO:teuthology.orchestra.run.smithi174.stdout:(149/154): NetworkManager-1.48.10-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-05T12:14:11.419 INFO:teuthology.orchestra.run.smithi123.stdout:(28/154): rbd-nbd-19.1.1-261.ge856d6b2.el9.x86_ 1.3 MB/s | 170 kB 00:00 2024-09-05T12:14:11.444 INFO:teuthology.orchestra.run.smithi123.stdout:(29/154): ceph-grafana-dashboards-19.1.1-261.ge 636 kB/s | 32 kB 00:00 2024-09-05T12:14:11.462 INFO:teuthology.orchestra.run.smithi174.stdout:(150/154): NetworkManager-libnm-1.48.10-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-09-05T12:14:11.469 INFO:teuthology.orchestra.run.smithi123.stdout:(30/154): ceph-mgr-cephadm-19.1.1-261.ge856d6b2 2.9 MB/s | 146 kB 00:00 2024-09-05T12:14:11.496 INFO:teuthology.orchestra.run.smithi174.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 1.2 MB/s | 40 kB 00:00 2024-09-05T12:14:11.563 INFO:teuthology.orchestra.run.smithi174.stdout:(152/154): librados2-19.1.1-261.ge856d6b2.el9.x 7.9 MB/s | 3.4 MB 00:00 2024-09-05T12:14:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:(153/154): NetworkManager-tui-1.48.10-1.el9.x86 2.6 MB/s | 248 kB 00:00 2024-09-05T12:14:11.594 INFO:teuthology.orchestra.run.smithi071.stdout:(44/154): python3-cryptography-36.0.1-4.el9.x86 1.2 MB/s | 1.2 MB 00:01 2024-09-05T12:14:11.595 INFO:teuthology.orchestra.run.smithi123.stdout:(31/154): ceph-mgr-dashboard-19.1.1-261.ge856d6 24 MB/s | 3.6 MB 00:00 2024-09-05T12:14:11.636 INFO:teuthology.orchestra.run.smithi071.stdout:(45/154): python3-ply-3.11-14.el9.noarch.rpm 158 kB/s | 106 kB 00:00 2024-09-05T12:14:11.645 INFO:teuthology.orchestra.run.smithi123.stdout:(32/154): ceph-mgr-modules-core-19.1.1-261.ge85 4.9 MB/s | 249 kB 00:00 2024-09-05T12:14:11.696 INFO:teuthology.orchestra.run.smithi123.stdout:(33/154): ceph-mgr-rook-19.1.1-261.ge856d6b2.el 987 kB/s | 49 kB 00:00 2024-09-05T12:14:11.746 INFO:teuthology.orchestra.run.smithi123.stdout:(34/154): ceph-prometheus-alerts-19.1.1-261.ge8 328 kB/s | 16 kB 00:00 2024-09-05T12:14:11.796 INFO:teuthology.orchestra.run.smithi123.stdout:(35/154): ceph-volume-19.1.1-261.ge856d6b2.el9. 5.6 MB/s | 285 kB 00:00 2024-09-05T12:14:11.855 INFO:teuthology.orchestra.run.smithi123.stdout:(36/154): cephadm-19.1.1-261.ge856d6b2.el9.noar 13 MB/s | 765 kB 00:00 2024-09-05T12:14:11.897 INFO:teuthology.orchestra.run.smithi123.stdout:(37/154): cephfs-top-19.1.1-261.ge856d6b2.el9.n 442 kB/s | 18 kB 00:00 2024-09-05T12:14:11.963 INFO:teuthology.orchestra.run.smithi174.stdout:(154/154): librbd1-19.1.1-261.ge856d6b2.el9.x86 3.9 MB/s | 3.2 MB 00:00 2024-09-05T12:14:11.970 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-05T12:14:11.970 INFO:teuthology.orchestra.run.smithi174.stdout:Total 24 MB/s | 222 MB 00:09 2024-09-05T12:14:11.986 INFO:teuthology.orchestra.run.smithi071.stdout:(46/154): python3-pycparser-2.20-6.el9.noarch.r 189 kB/s | 135 kB 00:00 2024-09-05T12:14:12.189 INFO:teuthology.orchestra.run.smithi123.stdout:(38/154): ceph-common-19.1.1-261.ge856d6b2.el9. 3.1 MB/s | 21 MB 00:06 2024-09-05T12:14:12.261 INFO:teuthology.orchestra.run.smithi071.stdout:(47/154): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 689 kB 00:00 2024-09-05T12:14:12.303 INFO:teuthology.orchestra.run.smithi071.stdout:(48/154): python3-requests-2.25.1-8.el9.noarch. 176 kB/s | 125 kB 00:00 2024-09-05T12:14:12.362 INFO:teuthology.orchestra.run.smithi071.stdout:(49/154): python3-urllib3-1.26.5-6.el9.noarch.r 296 kB/s | 215 kB 00:00 2024-09-05T12:14:12.445 INFO:teuthology.orchestra.run.smithi071.stdout:(50/154): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-05T12:14:12.471 INFO:teuthology.orchestra.run.smithi071.stdout:(51/154): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-05T12:14:12.496 INFO:teuthology.orchestra.run.smithi071.stdout:(52/154): elfutils-libelf-devel-0.191-4.el9.x86 163 kB/s | 22 kB 00:00 2024-09-05T12:14:12.529 INFO:teuthology.orchestra.run.smithi071.stdout:(53/154): flexiblas-3.0.4-8.el9.x86_64.rpm 561 kB/s | 33 kB 00:00 2024-09-05T12:14:12.580 INFO:teuthology.orchestra.run.smithi071.stdout:(54/154): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-05T12:14:12.613 INFO:teuthology.orchestra.run.smithi071.stdout:(55/154): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-09-05T12:14:12.647 INFO:teuthology.orchestra.run.smithi123.stdout:(39/154): ceph-mgr-diskprediction-local-19.1.1- 6.3 MB/s | 7.4 MB 00:01 2024-09-05T12:14:12.655 INFO:teuthology.orchestra.run.smithi071.stdout:(56/154): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-09-05T12:14:12.681 INFO:teuthology.orchestra.run.smithi123.stdout:(40/154): fuse-2.9.9-16.el9.x86_64.rpm 102 kB/s | 80 kB 00:00 2024-09-05T12:14:12.705 INFO:teuthology.orchestra.run.smithi071.stdout:(57/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 914 kB/s | 45 kB 00:00 2024-09-05T12:14:12.739 INFO:teuthology.orchestra.run.smithi071.stdout:(58/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-05T12:14:12.865 INFO:teuthology.orchestra.run.smithi071.stdout:(59/154): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-05T12:14:12.915 INFO:teuthology.orchestra.run.smithi071.stdout:(60/154): libtool-2.4.6-46.el9.x86_64.rpm 3.2 MB/s | 581 kB 00:00 2024-09-05T12:14:12.940 INFO:teuthology.orchestra.run.smithi071.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 327 kB/s | 24 kB 00:00 2024-09-05T12:14:12.974 INFO:teuthology.orchestra.run.smithi071.stdout:(62/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-05T12:14:12.999 INFO:teuthology.orchestra.run.smithi071.stdout:(63/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 292 kB 00:00 2024-09-05T12:14:13.058 INFO:teuthology.orchestra.run.smithi071.stdout:(64/154): openblas-0.3.26-2.el9.x86_64.rpm 662 kB/s | 38 kB 00:00 2024-09-05T12:14:13.083 INFO:teuthology.orchestra.run.smithi071.stdout:(65/154): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-05T12:14:13.162 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-05T12:14:13.173 INFO:teuthology.orchestra.run.smithi123.stdout:(41/154): libgfortran-11.5.0-2.el9.x86_64.rpm 815 kB/s | 801 kB 00:00 2024-09-05T12:14:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:(66/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.9 MB/s | 3.0 MB 00:00 2024-09-05T12:14:13.294 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-05T12:14:13.294 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-05T12:14:13.389 INFO:teuthology.orchestra.run.smithi123.stdout:(42/154): libquadmath-11.5.0-2.el9.x86_64.rpm 258 kB/s | 191 kB 00:00 2024-09-05T12:14:13.456 INFO:teuthology.orchestra.run.smithi123.stdout:(43/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-09-05T12:14:14.060 INFO:teuthology.orchestra.run.smithi123.stdout:(44/154): python3-ply-3.11-14.el9.noarch.rpm 162 kB/s | 106 kB 00:00 2024-09-05T12:14:14.067 INFO:teuthology.orchestra.run.smithi071.stdout:(67/154): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-09-05T12:14:14.107 INFO:teuthology.orchestra.run.smithi123.stdout:(45/154): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-05T12:14:14.215 INFO:teuthology.orchestra.run.smithi123.stdout:(46/154): python3-pycparser-2.20-6.el9.noarch.r 178 kB/s | 135 kB 00:00 2024-09-05T12:14:14.391 INFO:teuthology.orchestra.run.smithi123.stdout:(47/154): automake-1.16.2-8.el9.noarch.rpm 3.9 MB/s | 689 kB 00:00 2024-09-05T12:14:14.474 INFO:teuthology.orchestra.run.smithi123.stdout:(48/154): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-09-05T12:14:14.533 INFO:teuthology.orchestra.run.smithi123.stdout:(49/154): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-05T12:14:14.542 INFO:teuthology.orchestra.run.smithi071.stdout:(68/154): postgresql-contrib-13.16-1.el9.x86_64 1.9 MB/s | 910 kB 00:00 2024-09-05T12:14:14.600 INFO:teuthology.orchestra.run.smithi123.stdout:(50/154): elfutils-libelf-devel-0.191-4.el9.x86 325 kB/s | 22 kB 00:00 2024-09-05T12:14:14.659 INFO:teuthology.orchestra.run.smithi071.stdout:(69/154): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-05T12:14:14.667 INFO:teuthology.orchestra.run.smithi123.stdout:(51/154): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-05T12:14:14.733 INFO:teuthology.orchestra.run.smithi123.stdout:(52/154): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-05T12:14:14.859 INFO:teuthology.orchestra.run.smithi123.stdout:(53/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-05T12:14:14.867 INFO:teuthology.orchestra.run.smithi071.stdout:(70/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 2.5 MB/s | 4.4 MB 00:01 2024-09-05T12:14:14.884 INFO:teuthology.orchestra.run.smithi123.stdout:(54/154): python3-requests-2.25.1-8.el9.noarch. 151 kB/s | 125 kB 00:00 2024-09-05T12:14:14.909 INFO:teuthology.orchestra.run.smithi123.stdout:(55/154): python3-urllib3-1.26.5-6.el9.noarch.r 268 kB/s | 215 kB 00:00 2024-09-05T12:14:14.934 INFO:teuthology.orchestra.run.smithi123.stdout:(56/154): flexiblas-openblas-openmp-3.0.4-8.el9 246 kB/s | 18 kB 00:00 2024-09-05T12:14:14.993 INFO:teuthology.orchestra.run.smithi123.stdout:(57/154): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-05T12:14:15.018 INFO:teuthology.orchestra.run.smithi123.stdout:(58/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 541 kB/s | 45 kB 00:00 2024-09-05T12:14:15.058 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-05T12:14:15.058 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-05T12:14:15.060 INFO:teuthology.orchestra.run.smithi123.stdout:(59/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-09-05T12:14:15.086 INFO:teuthology.orchestra.run.smithi123.stdout:(60/154): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-05T12:14:15.128 INFO:teuthology.orchestra.run.smithi123.stdout:(61/154): libuuid-devel-2.37.4-20.el9.x86_64.rp 367 kB/s | 24 kB 00:00 2024-09-05T12:14:15.153 INFO:teuthology.orchestra.run.smithi123.stdout:(62/154): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-05T12:14:15.178 INFO:teuthology.orchestra.run.smithi123.stdout:(63/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 514 kB/s | 47 kB 00:00 2024-09-05T12:14:15.237 INFO:teuthology.orchestra.run.smithi123.stdout:(64/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-05T12:14:15.262 INFO:teuthology.orchestra.run.smithi123.stdout:(65/154): openblas-0.3.26-2.el9.x86_64.rpm 460 kB/s | 38 kB 00:00 2024-09-05T12:14:15.287 INFO:teuthology.orchestra.run.smithi123.stdout:(66/154): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-05T12:14:15.326 INFO:teuthology.orchestra.run.smithi071.stdout:(71/154): openblas-openmp-0.3.26-2.el9.x86_64.r 2.2 MB/s | 4.9 MB 00:02 2024-09-05T12:14:15.379 INFO:teuthology.orchestra.run.smithi123.stdout:(67/154): openssl-devel-3.2.2-4.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-05T12:14:15.410 INFO:teuthology.orchestra.run.smithi071.stdout:(72/154): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-05T12:14:15.429 INFO:teuthology.orchestra.run.smithi123.stdout:(68/154): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-05T12:14:15.443 INFO:teuthology.orchestra.run.smithi071.stdout:(73/154): protobuf-3.14.0-13.el9.x86_64.rpm 1.8 MB/s | 1.0 MB 00:00 2024-09-05T12:14:15.471 INFO:teuthology.orchestra.run.smithi123.stdout:(69/154): postgresql-contrib-13.16-1.el9.x86_64 9.7 MB/s | 910 kB 00:00 2024-09-05T12:14:15.497 INFO:teuthology.orchestra.run.smithi123.stdout:(70/154): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-05T12:14:15.501 INFO:teuthology.orchestra.run.smithi071.stdout:(74/154): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-05T12:14:15.639 INFO:teuthology.orchestra.run.smithi123.stdout:(71/154): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-05T12:14:15.643 INFO:teuthology.orchestra.run.smithi071.stdout:(75/154): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-05T12:14:15.672 INFO:teuthology.orchestra.run.smithi123.stdout:(72/154): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-09-05T12:14:15.756 INFO:teuthology.orchestra.run.smithi123.stdout:(73/154): openblas-openmp-0.3.26-2.el9.x86_64.r 9.5 MB/s | 4.9 MB 00:00 2024-09-05T12:14:15.781 INFO:teuthology.orchestra.run.smithi123.stdout:(74/154): python3-attrs-20.3.0-7.el9.noarch.rpm 616 kB/s | 87 kB 00:00 2024-09-05T12:14:15.785 INFO:teuthology.orchestra.run.smithi071.stdout:(76/154): python3-jinja2-2.11.3-6.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-09-05T12:14:15.823 INFO:teuthology.orchestra.run.smithi123.stdout:(75/154): python3-colorama-0.4.6-3.el9.noarch.r 918 kB/s | 61 kB 00:00 2024-09-05T12:14:15.836 INFO:teuthology.orchestra.run.smithi071.stdout:(77/154): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-09-05T12:14:15.847 INFO:teuthology.orchestra.run.smithi123.stdout:(76/154): python3-devel-3.9.19-8.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-09-05T12:14:15.886 INFO:teuthology.orchestra.run.smithi071.stdout:(78/154): python3-jsonpointer-2.0-4.el9.noarch. 386 kB/s | 19 kB 00:00 2024-09-05T12:14:15.923 INFO:teuthology.orchestra.run.smithi123.stdout:(77/154): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-09-05T12:14:15.948 INFO:teuthology.orchestra.run.smithi123.stdout:(78/154): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-09-05T12:14:15.974 INFO:teuthology.orchestra.run.smithi123.stdout:(79/154): python3-jinja2-2.11.3-6.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-09-05T12:14:16.003 INFO:teuthology.orchestra.run.smithi071.stdout:(79/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-05T12:14:16.007 INFO:teuthology.orchestra.run.smithi123.stdout:(80/154): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-05T12:14:16.032 INFO:teuthology.orchestra.run.smithi123.stdout:(81/154): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-05T12:14:16.049 INFO:teuthology.orchestra.run.smithi123.stdout:(82/154): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-09-05T12:14:16.061 INFO:teuthology.orchestra.run.smithi071.stdout:(80/154): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-09-05T12:14:16.108 INFO:teuthology.orchestra.run.smithi123.stdout:(83/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-05T12:14:16.175 INFO:teuthology.orchestra.run.smithi123.stdout:(84/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-09-05T12:14:16.200 INFO:teuthology.orchestra.run.smithi123.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 842 kB/s | 77 kB 00:00 2024-09-05T12:14:16.225 INFO:teuthology.orchestra.run.smithi123.stdout:(86/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-09-05T12:14:16.242 INFO:teuthology.orchestra.run.smithi123.stdout:(87/154): python3-prettytable-0.7.2-27.el9.noar 631 kB/s | 42 kB 00:00 2024-09-05T12:14:16.292 INFO:teuthology.orchestra.run.smithi123.stdout:(88/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-05T12:14:16.317 INFO:teuthology.orchestra.run.smithi123.stdout:(89/154): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-05T12:14:16.343 INFO:teuthology.orchestra.run.smithi123.stdout:(90/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-05T12:14:16.368 INFO:teuthology.orchestra.run.smithi123.stdout:(91/154): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-05T12:14:16.385 INFO:teuthology.orchestra.run.smithi123.stdout:(92/154): python3-requests-oauthlib-1.3.0-12.el 799 kB/s | 54 kB 00:00 2024-09-05T12:14:16.435 INFO:teuthology.orchestra.run.smithi123.stdout:(93/154): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-09-05T12:14:16.461 INFO:teuthology.orchestra.run.smithi123.stdout:(94/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-09-05T12:14:16.502 INFO:teuthology.orchestra.run.smithi123.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-05T12:14:16.528 INFO:teuthology.orchestra.run.smithi123.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-09-05T12:14:16.570 INFO:teuthology.orchestra.run.smithi123.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 710 kB/s | 47 kB 00:00 2024-09-05T12:14:16.603 INFO:teuthology.orchestra.run.smithi123.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-05T12:14:16.637 INFO:teuthology.orchestra.run.smithi123.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 865 kB/s | 58 kB 00:00 2024-09-05T12:14:16.787 INFO:teuthology.orchestra.run.smithi123.stdout:(100/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-05T12:14:16.812 INFO:teuthology.orchestra.run.smithi123.stdout:(101/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 363 kB/s | 64 kB 00:00 2024-09-05T12:14:16.837 INFO:teuthology.orchestra.run.smithi123.stdout:(102/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 409 kB/s | 96 kB 00:00 2024-09-05T12:14:16.863 INFO:teuthology.orchestra.run.smithi123.stdout:(103/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-09-05T12:14:16.913 INFO:teuthology.orchestra.run.smithi123.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-05T12:14:16.939 INFO:teuthology.orchestra.run.smithi123.stdout:(105/154): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-05T12:14:16.964 INFO:teuthology.orchestra.run.smithi123.stdout:(106/154): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-05T12:14:17.064 INFO:teuthology.orchestra.run.smithi123.stdout:(107/154): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-05T12:14:17.089 INFO:teuthology.orchestra.run.smithi123.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T12:14:17.114 INFO:teuthology.orchestra.run.smithi123.stdout:(109/154): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-05T12:14:17.140 INFO:teuthology.orchestra.run.smithi123.stdout:(110/154): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-09-05T12:14:17.156 INFO:teuthology.orchestra.run.smithi123.stdout:(111/154): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-05T12:14:17.199 INFO:teuthology.orchestra.run.smithi123.stdout:(112/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 24 MB/s | 839 kB 00:00 2024-09-05T12:14:17.224 INFO:teuthology.orchestra.run.smithi123.stdout:(113/154): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-05T12:14:17.249 INFO:teuthology.orchestra.run.smithi123.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T12:14:17.274 INFO:teuthology.orchestra.run.smithi123.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-05T12:14:17.300 INFO:teuthology.orchestra.run.smithi123.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-05T12:14:17.325 INFO:teuthology.orchestra.run.smithi123.stdout:(117/154): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-09-05T12:14:17.359 INFO:teuthology.orchestra.run.smithi123.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-05T12:14:17.384 INFO:teuthology.orchestra.run.smithi123.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-05T12:14:17.417 INFO:teuthology.orchestra.run.smithi123.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-05T12:14:17.443 INFO:teuthology.orchestra.run.smithi123.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-05T12:14:17.468 INFO:teuthology.orchestra.run.smithi123.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-05T12:14:17.493 INFO:teuthology.orchestra.run.smithi123.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-05T12:14:17.519 INFO:teuthology.orchestra.run.smithi123.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 709 kB/s | 18 kB 00:00 2024-09-05T12:14:17.544 INFO:teuthology.orchestra.run.smithi123.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-05T12:14:17.569 INFO:teuthology.orchestra.run.smithi123.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-05T12:14:17.570 INFO:teuthology.orchestra.run.smithi071.stdout:(81/154): postgresql-server-13.16-1.el9.x86_64. 2.0 MB/s | 5.8 MB 00:02 2024-09-05T12:14:17.594 INFO:teuthology.orchestra.run.smithi123.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-09-05T12:14:17.678 INFO:teuthology.orchestra.run.smithi123.stdout:(128/154): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-05T12:14:17.703 INFO:teuthology.orchestra.run.smithi123.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T12:14:17.728 INFO:teuthology.orchestra.run.smithi071.stdout:(82/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-05T12:14:17.729 INFO:teuthology.orchestra.run.smithi123.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T12:14:17.754 INFO:teuthology.orchestra.run.smithi123.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T12:14:17.787 INFO:teuthology.orchestra.run.smithi123.stdout:(132/154): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-09-05T12:14:17.813 INFO:teuthology.orchestra.run.smithi123.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 657 kB/s | 16 kB 00:00 2024-09-05T12:14:17.838 INFO:teuthology.orchestra.run.smithi123.stdout:(134/154): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-05T12:14:17.855 INFO:teuthology.orchestra.run.smithi123.stdout:(135/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-09-05T12:14:17.871 INFO:teuthology.orchestra.run.smithi071.stdout:(83/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-09-05T12:14:17.872 INFO:teuthology.orchestra.run.smithi123.stdout:(136/154): python3-pyperclip-1.8.2-2.el9.noarch 835 kB/s | 28 kB 00:00 2024-09-05T12:14:17.889 INFO:teuthology.orchestra.run.smithi123.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 911 kB/s | 31 kB 00:00 2024-09-05T12:14:17.906 INFO:teuthology.orchestra.run.smithi123.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-05T12:14:17.931 INFO:teuthology.orchestra.run.smithi123.stdout:(139/154): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-09-05T12:14:17.948 INFO:teuthology.orchestra.run.smithi123.stdout:(140/154): python3-tempora-5.0.0-2.el9.noarch.r 852 kB/s | 36 kB 00:00 2024-09-05T12:14:17.954 INFO:teuthology.orchestra.run.smithi071.stdout:(84/154): python3-babel-2.9.1-2.el9.noarch.rpm 2.3 MB/s | 6.0 MB 00:02 2024-09-05T12:14:17.973 INFO:teuthology.orchestra.run.smithi123.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-05T12:14:17.996 INFO:teuthology.orchestra.run.smithi071.stdout:(85/154): python3-packaging-20.9-5.el9.noarch.r 617 kB/s | 77 kB 00:00 2024-09-05T12:14:17.998 INFO:teuthology.orchestra.run.smithi123.stdout:(142/154): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-05T12:14:18.021 INFO:teuthology.orchestra.run.smithi071.stdout:(86/154): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-09-05T12:14:18.024 INFO:teuthology.orchestra.run.smithi123.stdout:(143/154): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-05T12:14:18.049 INFO:teuthology.orchestra.run.smithi123.stdout:(144/154): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-05T12:14:18.074 INFO:teuthology.orchestra.run.smithi123.stdout:(145/154): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-05T12:14:18.080 INFO:teuthology.orchestra.run.smithi071.stdout:(87/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-05T12:14:18.100 INFO:teuthology.orchestra.run.smithi123.stdout:(146/154): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-05T12:14:18.138 INFO:teuthology.orchestra.run.smithi071.stdout:(88/154): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-05T12:14:18.150 INFO:teuthology.orchestra.run.smithi123.stdout:(147/154): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-05T12:14:18.175 INFO:teuthology.orchestra.run.smithi123.stdout:(148/154): protobuf-compiler-3.14.0-13.el9.x86_ 646 kB/s | 863 kB 00:01 2024-09-05T12:14:18.222 INFO:teuthology.orchestra.run.smithi071.stdout:(89/154): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-09-05T12:14:18.247 INFO:teuthology.orchestra.run.smithi071.stdout:(90/154): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-05T12:14:18.273 INFO:teuthology.orchestra.run.smithi071.stdout:(91/154): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-05T12:14:18.331 INFO:teuthology.orchestra.run.smithi071.stdout:(92/154): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-09-05T12:14:18.383 INFO:teuthology.orchestra.run.smithi071.stdout:(93/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-05T12:14:18.384 INFO:teuthology.orchestra.run.smithi123.stdout:(149/154): librados2-19.1.1-261.ge856d6b2.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-05T12:14:18.642 INFO:teuthology.orchestra.run.smithi123.stdout:(150/154): librbd1-19.1.1-261.ge856d6b2.el9.x86 6.4 MB/s | 3.2 MB 00:00 2024-09-05T12:14:18.682 INFO:teuthology.orchestra.run.smithi071.stdout:(94/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-05T12:14:18.707 INFO:teuthology.orchestra.run.smithi071.stdout:(95/154): qatlib-24.02.0-1.el9.x86_64.rpm 679 kB/s | 221 kB 00:00 2024-09-05T12:14:18.724 INFO:teuthology.orchestra.run.smithi071.stdout:(96/154): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-09-05T12:14:18.749 INFO:teuthology.orchestra.run.smithi071.stdout:(97/154): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-05T12:14:18.783 INFO:teuthology.orchestra.run.smithi071.stdout:(98/154): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-05T12:14:18.800 INFO:teuthology.orchestra.run.smithi071.stdout:(99/154): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-05T12:14:18.875 INFO:teuthology.orchestra.run.smithi071.stdout:(100/154): python3-numpy-1.20.1-5.el9.x86_64.rp 1.8 MB/s | 5.1 MB 00:02 2024-09-05T12:14:18.901 INFO:teuthology.orchestra.run.smithi071.stdout:(101/154): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 819 kB/s | 96 kB 00:00 2024-09-05T12:14:18.926 INFO:teuthology.orchestra.run.smithi071.stdout:(102/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 507 kB/s | 64 kB 00:00 2024-09-05T12:14:18.951 INFO:teuthology.orchestra.run.smithi071.stdout:(103/154): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-05T12:14:19.030 INFO:teuthology.orchestra.run.smithi071.stdout:(104/154): abseil-cpp-20211102.0-3.el9.x86_64.r 6.8 MB/s | 548 kB 00:00 2024-09-05T12:14:19.060 INFO:teuthology.orchestra.run.smithi071.stdout:(105/154): lua-devel-5.4.4-4.el9.x86_64.rpm 140 kB/s | 22 kB 00:00 2024-09-05T12:14:19.102 INFO:teuthology.orchestra.run.smithi071.stdout:(106/154): gperftools-libs-2.9.1-2.el9.x86_64.r 4.2 MB/s | 309 kB 00:00 2024-09-05T12:14:19.127 INFO:teuthology.orchestra.run.smithi071.stdout:(107/154): grpc-data-1.46.7-10.el9.noarch.rpm 292 kB/s | 19 kB 00:00 2024-09-05T12:14:19.153 INFO:teuthology.orchestra.run.smithi071.stdout:(108/154): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-05T12:14:19.178 INFO:teuthology.orchestra.run.smithi071.stdout:(109/154): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-05T12:14:19.203 INFO:teuthology.orchestra.run.smithi071.stdout:(110/154): liboath-2.6.7-2.el9.x86_64.rpm 972 kB/s | 49 kB 00:00 2024-09-05T12:14:19.234 INFO:teuthology.orchestra.run.smithi123.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 69 kB/s | 40 kB 00:00 2024-09-05T12:14:19.253 INFO:teuthology.orchestra.run.smithi071.stdout:(111/154): luarocks-3.9.2-5.el9.noarch.rpm 3.7 MB/s | 151 kB 00:00 2024-09-05T12:14:19.278 INFO:teuthology.orchestra.run.smithi071.stdout:(112/154): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-09-05T12:14:19.284 INFO:teuthology.orchestra.run.smithi123.stdout:(152/154): NetworkManager-1.48.10-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-05T12:14:19.320 INFO:teuthology.orchestra.run.smithi071.stdout:(113/154): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-05T12:14:19.345 INFO:teuthology.orchestra.run.smithi071.stdout:(114/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-05T12:14:19.399 INFO:teuthology.orchestra.run.smithi071.stdout:(115/154): python3-cachetools-4.2.4-1.el9.noarc 608 kB/s | 32 kB 00:00 2024-09-05T12:14:19.426 INFO:teuthology.orchestra.run.smithi123.stdout:(153/154): NetworkManager-libnm-1.48.10-1.el9.x 1.7 MB/s | 1.8 MB 00:01 2024-09-05T12:14:19.429 INFO:teuthology.orchestra.run.smithi071.stdout:(116/154): python3-certifi-2023.05.07-4.el9.noa 464 kB/s | 14 kB 00:00 2024-09-05T12:14:19.472 INFO:teuthology.orchestra.run.smithi071.stdout:(117/154): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-09-05T12:14:19.497 INFO:teuthology.orchestra.run.smithi071.stdout:(118/154): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-05T12:14:19.531 INFO:teuthology.orchestra.run.smithi071.stdout:(119/154): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-05T12:14:19.556 INFO:teuthology.orchestra.run.smithi071.stdout:(120/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-05T12:14:19.656 INFO:teuthology.orchestra.run.smithi071.stdout:(121/154): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-05T12:14:19.690 INFO:teuthology.orchestra.run.smithi071.stdout:(122/154): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-05T12:14:19.715 INFO:teuthology.orchestra.run.smithi071.stdout:(123/154): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-09-05T12:14:19.769 INFO:teuthology.orchestra.run.smithi071.stdout:(124/154): python3-jaraco-classes-3.2.1-5.el9.n 333 kB/s | 18 kB 00:00 2024-09-05T12:14:19.801 INFO:teuthology.orchestra.run.smithi071.stdout:(125/154): python3-jaraco-collections-3.0.0-8.e 767 kB/s | 23 kB 00:00 2024-09-05T12:14:19.824 INFO:teuthology.orchestra.run.smithi071.stdout:(126/154): python3-jaraco-functools-3.5.0-2.el9 784 kB/s | 19 kB 00:00 2024-09-05T12:14:19.849 INFO:teuthology.orchestra.run.smithi071.stdout:(127/154): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-05T12:14:19.925 INFO:teuthology.orchestra.run.smithi071.stdout:(128/154): libarrow-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-09-05T12:14:19.950 INFO:teuthology.orchestra.run.smithi071.stdout:(129/154): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-05T12:14:19.975 INFO:teuthology.orchestra.run.smithi071.stdout:(130/154): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-05T12:14:20.000 INFO:teuthology.orchestra.run.smithi071.stdout:(131/154): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-05T12:14:20.034 INFO:teuthology.orchestra.run.smithi071.stdout:(132/154): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-05T12:14:20.043 INFO:teuthology.orchestra.run.smithi123.stdout:(154/154): NetworkManager-tui-1.48.10-1.el9.x86 307 kB/s | 248 kB 00:00 2024-09-05T12:14:20.049 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-05T12:14:20.050 INFO:teuthology.orchestra.run.smithi123.stdout:Total 15 MB/s | 222 MB 00:15 2024-09-05T12:14:20.059 INFO:teuthology.orchestra.run.smithi071.stdout:(133/154): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-05T12:14:20.085 INFO:teuthology.orchestra.run.smithi071.stdout:(134/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-05T12:14:20.127 INFO:teuthology.orchestra.run.smithi071.stdout:(135/154): python3-pyperclip-1.8.2-2.el9.noarch 684 kB/s | 28 kB 00:00 2024-09-05T12:14:20.160 INFO:teuthology.orchestra.run.smithi071.stdout:(136/154): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-09-05T12:14:20.186 INFO:teuthology.orchestra.run.smithi071.stdout:(137/154): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-09-05T12:14:20.219 INFO:teuthology.orchestra.run.smithi071.stdout:(138/154): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-05T12:14:20.244 INFO:teuthology.orchestra.run.smithi071.stdout:(139/154): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-05T12:14:20.278 INFO:teuthology.orchestra.run.smithi071.stdout:(140/154): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-05T12:14:20.311 INFO:teuthology.orchestra.run.smithi071.stdout:(141/154): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-05T12:14:20.353 INFO:teuthology.orchestra.run.smithi071.stdout:(142/154): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-09-05T12:14:20.378 INFO:teuthology.orchestra.run.smithi071.stdout:(143/154): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-09-05T12:14:20.404 INFO:teuthology.orchestra.run.smithi071.stdout:(144/154): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-05T12:14:20.420 INFO:teuthology.orchestra.run.smithi071.stdout:(145/154): python3-zc-lockfile-2.0-10.el9.noarc 480 kB/s | 20 kB 00:00 2024-09-05T12:14:20.446 INFO:teuthology.orchestra.run.smithi071.stdout:(146/154): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-05T12:14:20.496 INFO:teuthology.orchestra.run.smithi071.stdout:(147/154): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-05T12:14:20.547 INFO:teuthology.orchestra.run.smithi071.stdout:(148/154): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-05T12:14:20.730 INFO:teuthology.orchestra.run.smithi071.stdout:(149/154): librados2-19.1.1-261.ge856d6b2.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-05T12:14:20.797 INFO:teuthology.orchestra.run.smithi071.stdout:(150/154): librbd1-19.1.1-261.ge856d6b2.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-05T12:14:21.099 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-05T12:14:21.174 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T12:14:21.223 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-05T12:14:21.263 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T12:14:21.357 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-05T12:14:21.357 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-05T12:14:21.372 INFO:teuthology.orchestra.run.smithi071.stdout:(151/154): NetworkManager-team-1.48.10-1.el9.x8 70 kB/s | 40 kB 00:00 2024-09-05T12:14:21.476 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T12:14:21.606 INFO:teuthology.orchestra.run.smithi071.stdout:(152/154): NetworkManager-1.48.10-1.el9.x86_64. 2.2 MB/s | 2.3 MB 00:01 2024-09-05T12:14:21.697 INFO:teuthology.orchestra.run.smithi071.stdout:(153/154): NetworkManager-libnm-1.48.10-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-05T12:14:21.820 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T12:14:21.925 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T12:14:22.048 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T12:14:22.084 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:14:22.166 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:14:22.209 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T12:14:22.239 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T12:14:22.264 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T12:14:22.329 INFO:teuthology.orchestra.run.smithi071.stdout:(154/154): NetworkManager-tui-1.48.10-1.el9.x86 259 kB/s | 248 kB 00:00 2024-09-05T12:14:22.336 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-05T12:14:22.337 INFO:teuthology.orchestra.run.smithi071.stdout:Total 12 MB/s | 222 MB 00:18 2024-09-05T12:14:22.415 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T12:14:22.461 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T12:14:22.489 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:14:22.592 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:14:22.626 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T12:14:22.686 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T12:14:22.716 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T12:14:22.738 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T12:14:22.926 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T12:14:23.068 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T12:14:23.107 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-05T12:14:23.108 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-05T12:14:23.196 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T12:14:23.257 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T12:14:23.371 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:23.458 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:23.683 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:23.862 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T12:14:24.007 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T12:14:24.046 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T12:14:24.081 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T12:14:24.155 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T12:14:24.216 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T12:14:24.256 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T12:14:24.301 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T12:14:24.311 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-09-05T12:14:24.331 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T12:14:24.360 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T12:14:24.430 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T12:14:24.468 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-09-05T12:14:24.468 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-09-05T12:14:24.491 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T12:14:24.520 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T12:14:24.556 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T12:14:24.632 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T12:14:24.666 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 33/160 2024-09-05T12:14:24.709 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T12:14:24.750 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 35/160 2024-09-05T12:14:24.820 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T12:14:24.865 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T12:14:24.939 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T12:14:25.081 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T12:14:25.134 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T12:14:25.665 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T12:14:25.746 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T12:14:25.774 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T12:14:25.809 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T12:14:25.847 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 45/160 2024-09-05T12:14:25.889 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T12:14:25.944 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T12:14:26.016 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T12:14:26.048 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T12:14:26.085 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T12:14:26.131 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T12:14:26.167 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T12:14:26.210 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T12:14:26.252 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T12:14:26.390 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T12:14:26.445 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T12:14:26.487 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T12:14:26.590 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T12:14:26.635 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T12:14:26.675 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T12:14:26.700 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T12:14:26.725 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T12:14:26.778 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-09-05T12:14:26.779 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-09-05T12:14:26.950 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T12:14:27.080 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T12:14:27.109 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T12:14:27.137 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T12:14:27.180 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T12:14:27.223 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T12:14:27.261 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T12:14:27.292 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T12:14:27.323 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T12:14:27.357 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T12:14:27.390 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T12:14:27.432 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T12:14:27.467 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T12:14:27.553 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T12:14:27.605 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T12:14:27.644 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T12:14:27.749 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T12:14:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T12:14:28.547 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T12:14:28.601 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T12:14:28.635 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T12:14:28.743 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T12:14:28.801 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T12:14:28.842 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T12:14:28.922 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T12:14:28.953 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T12:14:29.136 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T12:14:29.159 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T12:14:29.244 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T12:14:29.382 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-05T12:14:29.462 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T12:14:29.487 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T12:14:29.530 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T12:14:29.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T12:14:29.902 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T12:14:29.980 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T12:14:30.025 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T12:14:30.085 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T12:14:30.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:14:30.207 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:14:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T12:14:30.285 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T12:14:30.311 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T12:14:30.469 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T12:14:30.516 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T12:14:30.552 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:14:30.649 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:14:30.678 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T12:14:30.724 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T12:14:30.755 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T12:14:30.780 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T12:14:30.931 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T12:14:31.089 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T12:14:31.218 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T12:14:31.262 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T12:14:31.263 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T12:14:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T12:14:31.369 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T12:14:31.407 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T12:14:31.438 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:31.666 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:31.715 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T12:14:31.753 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T12:14:31.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T12:14:31.852 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T12:14:31.874 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T12:14:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T12:14:31.952 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T12:14:32.027 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T12:14:32.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 100/160 2024-09-05T12:14:32.087 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T12:14:32.111 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T12:14:32.136 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T12:14:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T12:14:32.212 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T12:14:32.257 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T12:14:32.328 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T12:14:32.380 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T12:14:32.409 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T12:14:32.462 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T12:14:32.549 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T12:14:32.581 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 33/160 2024-09-05T12:14:32.596 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T12:14:32.625 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T12:14:32.653 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 35/160 2024-09-05T12:14:32.734 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T12:14:32.776 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T12:14:32.805 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T12:14:32.853 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T12:14:32.876 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T12:14:32.898 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 104/160 2024-09-05T12:14:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T12:14:33.063 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T12:14:33.615 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-09-05T12:14:33.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T12:14:33.703 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T12:14:33.739 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T12:14:33.752 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/160 2024-09-05T12:14:33.781 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T12:14:33.820 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 45/160 2024-09-05T12:14:33.853 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T12:14:33.909 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T12:14:33.918 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/160 2024-09-05T12:14:33.983 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T12:14:33.988 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/160 2024-09-05T12:14:34.029 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T12:14:34.065 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T12:14:34.116 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T12:14:34.148 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T12:14:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T12:14:34.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T12:14:34.371 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2024-09-05T12:14:34.374 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T12:14:34.418 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T12:14:34.460 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T12:14:34.495 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T12:14:34.574 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T12:14:34.680 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 5/160 2024-09-05T12:14:34.718 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:14:34.741 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T12:14:34.831 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:14:34.907 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T12:14:34.935 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T12:14:34.941 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/160 2024-09-05T12:14:34.967 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T12:14:34.974 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/160 2024-09-05T12:14:34.992 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T12:14:35.077 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T12:14:35.192 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/160 2024-09-05T12:14:35.209 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:35.215 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T12:14:35.245 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:35.256 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T12:14:35.263 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/160 2024-09-05T12:14:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T12:14:35.307 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:14:35.318 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T12:14:35.362 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T12:14:35.400 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T12:14:35.432 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T12:14:35.447 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:14:35.461 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T12:14:35.474 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T12:14:35.496 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T12:14:35.530 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T12:14:35.540 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 13/160 2024-09-05T12:14:35.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T12:14:35.599 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T12:14:35.605 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T12:14:35.649 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/160 2024-09-05T12:14:35.695 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T12:14:35.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T12:14:35.784 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T12:14:35.790 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:35.831 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T12:14:35.890 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T12:14:35.895 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T12:14:35.939 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T12:14:35.939 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T12:14:35.939 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T12:14:35.939 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:35.943 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T12:14:35.974 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T12:14:36.105 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/160 2024-09-05T12:14:36.298 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T12:14:36.384 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 16/160 2024-09-05T12:14:36.513 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:36.626 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:36.703 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T12:14:36.757 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T12:14:36.790 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T12:14:36.902 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T12:14:36.913 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.48.10-1.el9.x86_64 17/160 2024-09-05T12:14:36.954 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T12:14:36.999 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T12:14:37.062 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/160 2024-09-05T12:14:37.089 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T12:14:37.134 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T12:14:37.177 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 19/160 2024-09-05T12:14:37.215 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/160 2024-09-05T12:14:37.261 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/160 2024-09-05T12:14:37.317 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T12:14:37.342 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/160 2024-09-05T12:14:37.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T12:14:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/160 2024-09-05T12:14:37.426 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T12:14:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 24/160 2024-09-05T12:14:37.505 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/160 2024-09-05T12:14:37.553 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/160 2024-09-05T12:14:37.588 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2024-09-05T12:14:37.677 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/160 2024-09-05T12:14:37.740 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 29/160 2024-09-05T12:14:37.799 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/160 2024-09-05T12:14:37.845 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 31/160 2024-09-05T12:14:37.928 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 32/160 2024-09-05T12:14:37.961 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 33/160 2024-09-05T12:14:38.004 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T12:14:38.033 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 35/160 2024-09-05T12:14:38.099 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T12:14:38.125 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/160 2024-09-05T12:14:38.193 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-5.el9.noarch 37/160 2024-09-05T12:14:38.217 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T12:14:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/160 2024-09-05T12:14:38.457 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/160 2024-09-05T12:14:38.548 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/160 2024-09-05T12:14:39.200 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/160 2024-09-05T12:14:39.306 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/160 2024-09-05T12:14:39.341 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 43/160 2024-09-05T12:14:39.385 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/160 2024-09-05T12:14:39.423 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 45/160 2024-09-05T12:14:39.465 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/160 2024-09-05T12:14:39.488 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T12:14:39.525 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/160 2024-09-05T12:14:39.587 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T12:14:39.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T12:14:39.665 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T12:14:39.668 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/160 2024-09-05T12:14:39.716 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 49/160 2024-09-05T12:14:39.752 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 50/160 2024-09-05T12:14:39.811 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/160 2024-09-05T12:14:39.851 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/160 2024-09-05T12:14:39.896 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/160 2024-09-05T12:14:39.955 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/160 2024-09-05T12:14:39.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T12:14:40.019 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T12:14:40.110 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T12:14:40.137 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/160 2024-09-05T12:14:40.140 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 100/160 2024-09-05T12:14:40.176 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T12:14:40.180 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/160 2024-09-05T12:14:40.290 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 57/160 2024-09-05T12:14:40.437 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 58/160 2024-09-05T12:14:40.480 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 59/160 2024-09-05T12:14:40.523 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 60/160 2024-09-05T12:14:40.552 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 61/160 2024-09-05T12:14:40.585 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 62/160 2024-09-05T12:14:40.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T12:14:40.703 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 63/160 2024-09-05T12:14:40.705 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T12:14:40.777 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T12:14:40.807 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 104/160 2024-09-05T12:14:40.868 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 64/160 2024-09-05T12:14:40.905 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 65/160 2024-09-05T12:14:40.942 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 66/160 2024-09-05T12:14:40.997 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/160 2024-09-05T12:14:41.066 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/160 2024-09-05T12:14:41.126 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 69/160 2024-09-05T12:14:41.165 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/160 2024-09-05T12:14:41.202 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/160 2024-09-05T12:14:41.236 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/160 2024-09-05T12:14:41.269 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/160 2024-09-05T12:14:41.314 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/160 2024-09-05T12:14:41.346 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/160 2024-09-05T12:14:41.447 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/160 2024-09-05T12:14:41.521 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/160 2024-09-05T12:14:41.558 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/160 2024-09-05T12:14:41.682 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/160 2024-09-05T12:14:41.746 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 80/160 2024-09-05T12:14:42.676 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 81/160 2024-09-05T12:14:42.738 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 82/160 2024-09-05T12:14:42.772 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 83/160 2024-09-05T12:14:42.892 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 84/160 2024-09-05T12:14:42.972 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 85/160 2024-09-05T12:14:43.033 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 86/160 2024-09-05T12:14:43.130 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:43.132 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/160 2024-09-05T12:14:43.175 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/160 2024-09-05T12:14:43.182 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:43.408 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.26-2.el9.x86_64 89/160 2024-09-05T12:14:43.446 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 90/160 2024-09-05T12:14:43.539 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/160 2024-09-05T12:14:43.739 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:43.781 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T12:14:43.870 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T12:14:43.870 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T12:14:43.870 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T12:14:43.870 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:43.899 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T12:14:44.362 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/160 2024-09-05T12:14:44.736 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/160 2024-09-05T12:14:46.350 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/160 2024-09-05T12:14:46.450 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/160 2024-09-05T12:14:46.486 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/160 2024-09-05T12:14:46.519 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 97/160 2024-09-05T12:14:46.853 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/160 2024-09-05T12:14:46.897 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T12:14:47.016 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 99/160 2024-09-05T12:14:47.044 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 100/160 2024-09-05T12:14:47.101 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/160 2024-09-05T12:14:47.625 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/160 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-09-05T12:14:47.636 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-09-05T12:14:47.637 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-09-05T12:14:47.637 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:47.698 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T12:14:47.791 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 103/160 2024-09-05T12:14:47.818 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 104/160 2024-09-05T12:14:47.833 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T12:14:47.850 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T12:14:47.850 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:47.850 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T12:14:47.850 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:49.328 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T12:14:49.355 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T12:14:49.355 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:49.355 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T12:14:49.355 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T12:14:49.356 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T12:14:49.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:49.579 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T12:14:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T12:14:49.602 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:49.602 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T12:14:49.602 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T12:14:49.603 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T12:14:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:50.038 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T12:14:50.061 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T12:14:50.061 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:50.062 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T12:14:50.062 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T12:14:50.062 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T12:14:50.062 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:50.168 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T12:14:50.224 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T12:14:50.279 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T12:14:50.361 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:50.406 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:50.465 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T12:14:50.521 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T12:14:50.615 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T12:14:50.704 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T12:14:50.835 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T12:14:50.871 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T12:14:50.903 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T12:14:50.931 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T12:14:51.034 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 105/160 2024-09-05T12:14:51.084 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T12:14:51.233 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 106/160 2024-09-05T12:14:51.233 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-05T12:14:51.234 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-05T12:14:51.234 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:14:51.277 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T12:14:51.416 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T12:14:51.462 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T12:14:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 124/160 2024-09-05T12:14:51.572 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 125/160 2024-09-05T12:14:51.610 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T12:14:52.646 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T12:14:52.687 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T12:14:53.648 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T12:14:53.679 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T12:14:53.781 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T12:14:53.957 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 129/160 2024-09-05T12:14:53.981 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T12:14:54.005 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T12:14:54.005 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:54.005 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T12:14:54.005 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T12:14:54.006 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T12:14:54.006 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:54.057 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T12:14:54.069 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T12:14:54.165 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 132/160 2024-09-05T12:14:54.222 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 133/160 2024-09-05T12:14:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T12:14:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T12:14:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T12:14:55.208 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:55.209 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T12:14:55.209 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T12:14:55.209 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T12:14:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:55.251 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T12:14:55.274 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T12:14:55.274 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:55.274 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T12:14:55.274 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:55.319 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T12:14:55.342 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T12:14:55.342 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:55.342 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T12:14:55.342 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T12:14:55.343 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T12:14:55.343 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-05T12:14:55.478 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-05T12:14:55.479 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-05T12:14:55.479 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:55.650 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T12:14:55.653 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T12:14:55.666 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T12:14:55.666 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:55.666 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T12:14:55.667 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:55.676 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T12:14:55.676 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:55.676 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T12:14:55.677 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T12:14:55.677 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T12:14:55.677 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:14:57.152 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T12:14:57.179 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T12:14:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T12:14:57.180 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T12:14:57.180 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T12:14:57.180 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:57.412 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T12:14:57.435 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T12:14:57.436 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:57.436 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T12:14:57.436 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T12:14:57.436 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T12:14:57.436 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:57.881 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T12:14:57.904 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T12:14:57.904 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:14:57.904 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T12:14:57.904 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T12:14:57.904 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T12:14:57.904 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:14:58.019 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T12:14:58.085 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T12:14:58.140 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T12:14:58.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T12:14:58.381 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T12:14:58.485 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T12:14:58.606 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T12:14:58.754 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T12:14:58.789 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T12:14:58.821 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T12:14:58.849 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T12:14:59.236 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T12:14:59.264 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T12:14:59.295 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 124/160 2024-09-05T12:14:59.358 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 125/160 2024-09-05T12:14:59.395 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T12:15:00.197 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 139/160 2024-09-05T12:15:00.239 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 140/160 2024-09-05T12:15:00.279 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 141/160 2024-09-05T12:15:00.320 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T12:15:00.355 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T12:15:00.441 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T12:15:00.480 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T12:15:00.536 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 144/160 2024-09-05T12:15:00.577 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T12:15:00.608 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 146/160 2024-09-05T12:15:00.636 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T12:15:00.683 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T12:15:01.288 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:01.322 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:01.342 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:01.444 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T12:15:01.472 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T12:15:01.578 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T12:15:01.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 129/160 2024-09-05T12:15:01.775 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T12:15:01.799 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T12:15:01.799 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:01.799 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T12:15:01.799 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T12:15:01.799 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T12:15:01.799 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:01.850 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T12:15:01.862 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T12:15:01.960 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 132/160 2024-09-05T12:15:02.007 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 133/160 2024-09-05T12:15:02.158 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 150/160 2024-09-05T12:15:02.243 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T12:15:02.283 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T12:15:02.363 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T12:15:02.396 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T12:15:02.422 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T12:15:02.422 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T12:15:02.506 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T12:15:02.530 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T12:15:02.536 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:02.536 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:02.715 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:02.716 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T12:15:02.800 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T12:15:02.800 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T12:15:02.944 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T12:15:02.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T12:15:03.001 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T12:15:03.001 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:03.002 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T12:15:03.002 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T12:15:03.002 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T12:15:03.002 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:03.045 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T12:15:03.068 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T12:15:03.068 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:03.069 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T12:15:03.069 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:03.112 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T12:15:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T12:15:03.136 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:03.136 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T12:15:03.136 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T12:15:03.136 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T12:15:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:03.446 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T12:15:03.469 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T12:15:03.470 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:03.470 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T12:15:03.470 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T12:15:03.470 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T12:15:03.470 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:04.117 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 107/160 2024-09-05T12:15:04.117 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-09-05T12:15:04.117 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-09-05T12:15:04.117 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-09-05T12:15:04.117 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-09-05T12:15:04.117 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-09-05T12:15:04.118 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-09-05T12:15:04.118 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-09-05T12:15:04.118 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:04.330 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T12:15:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 108/160 2024-09-05T12:15:04.366 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:04.366 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-05T12:15:04.366 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T12:15:06.015 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 109/160 2024-09-05T12:15:06.015 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:06.016 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-05T12:15:06.016 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T12:15:06.016 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-05T12:15:06.016 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:06.266 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T12:15:06.317 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 110/160 2024-09-05T12:15:06.318 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:06.318 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-05T12:15:06.318 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T12:15:06.318 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-05T12:15:06.318 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:06.792 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T12:15:06.843 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 111/160 2024-09-05T12:15:06.843 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:06.843 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-05T12:15:06.843 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T12:15:06.843 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-05T12:15:06.844 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:06.975 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 112/160 2024-09-05T12:15:07.044 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/160 2024-09-05T12:15:07.116 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/160 2024-09-05T12:15:07.340 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/160 2024-09-05T12:15:07.404 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/160 2024-09-05T12:15:07.499 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/160 2024-09-05T12:15:07.608 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/160 2024-09-05T12:15:07.795 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/160 2024-09-05T12:15:07.837 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/160 2024-09-05T12:15:07.885 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/160 2024-09-05T12:15:07.919 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-16.el9.x86_64 122/160 2024-09-05T12:15:08.066 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 139/160 2024-09-05T12:15:08.107 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 140/160 2024-09-05T12:15:08.149 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 141/160 2024-09-05T12:15:08.180 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T12:15:08.208 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T12:15:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T12:15:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 1/160 2024-09-05T12:15:08.253 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 3/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 4/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 5/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 8/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 9/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 10/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 11/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:15:08.254 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 13/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 14/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 15/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 16/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 17/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 18/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 19/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 20/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 21/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 22/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 23/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 24/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 25/160 2024-09-05T12:15:08.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 26/160 2024-09-05T12:15:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 27/160 2024-09-05T12:15:08.256 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 28/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 29/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 30/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 31/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 32/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 33/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T12:15:08.268 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 35/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 36/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 37/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 38/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 39/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T12:15:08.269 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T12:15:08.270 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T12:15:08.271 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T12:15:08.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T12:15:08.273 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T12:15:08.274 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T12:15:08.275 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T12:15:08.276 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T12:15:08.277 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 149/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 151/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:08.278 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T12:15:08.399 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 144/160 2024-09-05T12:15:08.446 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T12:15:08.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 146/160 2024-09-05T12:15:08.505 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T12:15:08.544 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T12:15:08.776 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T12:15:08.802 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 123/160 2024-09-05T12:15:08.835 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 124/160 2024-09-05T12:15:08.903 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 125/160 2024-09-05T12:15:08.932 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T12:15:09.126 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:09.166 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:09.187 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:10.020 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 150/160 2024-09-05T12:15:10.121 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T12:15:10.142 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 126/160 2024-09-05T12:15:10.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T12:15:10.184 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T12:15:10.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T12:15:10.273 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T12:15:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T12:15:10.299 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T12:15:10.392 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T12:15:10.417 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T12:15:10.422 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:10.423 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:10.610 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:10.611 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T12:15:10.702 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T12:15:10.703 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T12:15:10.885 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T12:15:10.885 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:15:10.885 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-09-05T12:15:10.885 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.886 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T12:15:10.888 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T12:15:10.889 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T12:15:10.890 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T12:15:10.891 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T12:15:10.892 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T12:15:10.893 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T12:15:10.894 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T12:15:10.895 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T12:15:10.896 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T12:15:10.897 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:15:10.898 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-05T12:15:11.263 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 127/160 2024-09-05T12:15:11.289 DEBUG:teuthology.parallel:result is None 2024-09-05T12:15:11.317 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T12:15:11.455 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 128/160 2024-09-05T12:15:11.630 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 129/160 2024-09-05T12:15:11.686 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T12:15:11.738 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 130/160 2024-09-05T12:15:11.738 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:11.738 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-05T12:15:11.738 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T12:15:11.739 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-05T12:15:11.739 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:11.795 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T12:15:11.822 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 131/160 2024-09-05T12:15:11.932 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 132/160 2024-09-05T12:15:12.003 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 133/160 2024-09-05T12:15:13.024 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-46.el9.x86_64 134/160 2024-09-05T12:15:13.056 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T12:15:13.107 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 135/160 2024-09-05T12:15:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-05T12:15:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T12:15:13.108 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-05T12:15:13.108 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:13.157 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T12:15:13.206 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-261.ge856d6 136/160 2024-09-05T12:15:13.206 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:13.207 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-05T12:15:13.207 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:13.274 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T12:15:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 137/160 2024-09-05T12:15:13.326 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:13.326 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-05T12:15:13.326 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T12:15:13.326 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-05T12:15:13.326 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:13.666 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T12:15:13.718 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 138/160 2024-09-05T12:15:13.718 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-05T12:15:13.718 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-05T12:15:13.718 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T12:15:13.718 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-05T12:15:13.718 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T12:15:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 1/160 2024-09-05T12:15:16.448 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 3/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 4/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 5/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 8/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 9/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 10/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 11/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:15:16.449 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 13/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 14/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 15/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 16/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 17/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 18/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 19/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 20/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 21/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 22/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 23/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 24/160 2024-09-05T12:15:16.450 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 25/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 26/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 27/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 28/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 29/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 30/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 31/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 32/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 33/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 35/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 36/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 37/160 2024-09-05T12:15:16.451 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 38/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 39/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T12:15:16.452 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T12:15:16.453 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T12:15:16.454 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T12:15:16.455 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T12:15:16.456 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T12:15:16.457 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T12:15:16.458 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T12:15:16.459 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 149/160 2024-09-05T12:15:16.460 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 151/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:16.461 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T12:15:18.637 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 139/160 2024-09-05T12:15:18.678 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 140/160 2024-09-05T12:15:18.721 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 141/160 2024-09-05T12:15:18.751 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 142/160 2024-09-05T12:15:18.778 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 143/160 2024-09-05T12:15:19.025 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 144/160 2024-09-05T12:15:19.075 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 145/160 2024-09-05T12:15:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 146/160 2024-09-05T12:15:19.136 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-team-1:1.48.10-1.el9.x86_64 147/160 2024-09-05T12:15:19.180 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : NetworkManager-tui-1:1.48.10-1.el9.x86_64 148/160 2024-09-05T12:15:19.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T12:15:19.209 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:19.209 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-05T12:15:19.209 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T12:15:19.210 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.211 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:19.212 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.213 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T12:15:19.214 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T12:15:19.215 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.216 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T12:15:19.217 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T12:15:19.218 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T12:15:19.219 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T12:15:19.220 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.221 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T12:15:19.222 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T12:15:19.223 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:15:19.223 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-05T12:15:19.630 DEBUG:teuthology.parallel:result is None 2024-09-05T12:15:19.956 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:20.044 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:20.086 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 149/160 2024-09-05T12:15:21.347 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 150/160 2024-09-05T12:15:21.469 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.2.2-4.el9.x86_64 151/160 2024-09-05T12:15:21.538 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/160 2024-09-05T12:15:21.644 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/160 2024-09-05T12:15:21.676 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/160 2024-09-05T12:15:21.705 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 155/160 2024-09-05T12:15:21.705 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T12:15:21.812 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 156/160 2024-09-05T12:15:21.853 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 157/160 2024-09-05T12:15:21.868 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:21.869 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:22.106 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:22.106 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T12:15:22.231 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 159/160 2024-09-05T12:15:22.231 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T12:15:22.585 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2024-09-05T12:15:22.586 INFO:teuthology.orchestra.run.smithi071.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-05T12:15:22.586 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:28.364 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 1/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 3/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 4/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-261.ge856d6 5/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 6/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 7/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 8/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 9/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 10/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 11/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 12/160 2024-09-05T12:15:28.365 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 13/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 14/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_6 15/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 16/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 17/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 18/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_ 19/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 20/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9 21/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x 22/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 23/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 24/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 25/160 2024-09-05T12:15:28.366 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 26/160 2024-09-05T12:15:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 27/160 2024-09-05T12:15:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 28/160 2024-09-05T12:15:28.367 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 29/160 2024-09-05T12:15:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.e 30/160 2024-09-05T12:15:28.378 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noar 31/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.no 32/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-261.ge856 33/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9 34/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 35/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el 36/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 37/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 38/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 39/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/160 2024-09-05T12:15:28.379 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/160 2024-09-05T12:15:28.380 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/160 2024-09-05T12:15:28.381 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.2.2-4.el9.x86_64 68/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/160 2024-09-05T12:15:28.382 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/160 2024-09-05T12:15:28.383 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/160 2024-09-05T12:15:28.384 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/160 2024-09-05T12:15:28.385 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/160 2024-09-05T12:15:28.386 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/160 2024-09-05T12:15:28.387 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 149/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 151/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.48.10-1.el9.x86_64 153/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.48.10-1.el9.x86_64 155/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.48.10-1.el9.x86_64 157/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/160 2024-09-05T12:15:28.388 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.48.10-1.el9.x86_64 159/160 2024-09-05T12:15:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/160 2024-09-05T12:15:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:30.643 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-09-05T12:15:30.643 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-libnm-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-team-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-tui-1:1.48.10-1.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.644 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.645 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.1.1-261.ge856d6b2.el9.noarch 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-05T12:15:30.646 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-05T12:15:30.647 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-05T12:15:30.648 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.2.2-4.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.16-1.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-05T12:15:30.649 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.650 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-05T12:15:30.651 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-05T12:15:30.652 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-05T12:15:30.653 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-09-05T12:15:30.656 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-05T12:15:30.657 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-05T12:15:30.658 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.1.1-261.ge856d6b2.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-20.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-05T12:15:30.659 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-05T12:15:30.660 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-05T12:15:30.660 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:15:30.660 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-09-05T12:15:31.272 DEBUG:teuthology.parallel:result is None 2024-09-05T12:15:31.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:15:31.419 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T12:15:31.468 INFO:teuthology.orchestra.run.smithi071.stdout:19.1.1-261.ge856d6b2.el9 2024-09-05T12:15:31.469 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-261.ge856d6b2.el9 2024-09-05T12:15:31.469 INFO:teuthology.task.install:The correct ceph version 19.1.1-261.ge856d6b2 is installed. 2024-09-05T12:15:31.471 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:15:31.613 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T12:15:31.639 INFO:teuthology.orchestra.run.smithi123.stdout:19.1.1-261.ge856d6b2.el9 2024-09-05T12:15:31.639 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-261.ge856d6b2.el9 2024-09-05T12:15:31.640 INFO:teuthology.task.install:The correct ceph version 19.1.1-261.ge856d6b2 is installed. 2024-09-05T12:15:31.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:15:31.783 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-05T12:15:31.809 INFO:teuthology.orchestra.run.smithi174.stdout:19.1.1-261.ge856d6b2.el9 2024-09-05T12:15:31.810 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-261.ge856d6b2.el9 2024-09-05T12:15:31.810 INFO:teuthology.task.install:The correct ceph version 19.1.1-261.ge856d6b2 is installed. 2024-09-05T12:15:31.812 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-05T12:15:31.812 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:15:31.812 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T12:15:31.870 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:15:31.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T12:15:31.902 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:15:31.902 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-05T12:15:31.934 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-05T12:15:31.935 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:15:31.935 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T12:15:31.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T12:15:32.091 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:15:32.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T12:15:32.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T12:15:32.206 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:15:32.206 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-09-05T12:15:32.237 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-05T12:15:32.305 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-05T12:15:32.305 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:15:32.305 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T12:15:32.361 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T12:15:32.457 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:15:32.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T12:15:32.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T12:15:32.548 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:15:32.548 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-05T12:15:32.573 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-05T12:15:32.638 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-05T12:15:32.638 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:15:32.639 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T12:15:32.695 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T12:15:32.790 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:15:32.790 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T12:15:32.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T12:15:32.881 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:15:32.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-09-05T12:15:32.908 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-05T12:15:32.974 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-05T12:15:33.071 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-05T12:15:33.071 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:15:33.072 INFO:tasks.cephadm:Cluster fsid is 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:15:33.072 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-05T12:15:33.072 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.71', 'mon.b': '172.21.15.123', 'mon.c': '172.21.15.174'} 2024-09-05T12:15:33.072 INFO:tasks.cephadm:First mon is mon.a on smithi071 2024-09-05T12:15:33.072 INFO:tasks.cephadm:First mgr is x 2024-09-05T12:15:33.072 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-05T12:15:33.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-09-05T12:15:33.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-09-05T12:15:33.155 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-09-05T12:15:33.181 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-05T12:15:33.181 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:15:33.324 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e856d6b2462509a377ef39149a05475ed006ebac', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82800/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-261-ge856d6b2', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-261.ge856d6b2'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-05 10:44:16.424921', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-bharath6-testing-2024-09-05-1413-squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-05T12:15:33.460 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-bharath6-testing-2024-09-05-1413-squid, sha1 e856d6b2462509a377ef39149a05475ed006ebac from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:15:33.461 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm 2024-09-05T12:15:33.461 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm 2024-09-05T12:15:33.461 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:33.879 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 784415 Sep 5 12:15 /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:33.879 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.242 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 784415 Sep 5 12:15 /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.243 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-bharath6-testing-2024-09-05-1413-squid/e856d6b2462509a377ef39149a05475ed006ebac/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.657 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 784415 Sep 5 12:15 /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.658 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.696 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.714 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-05T12:15:34.752 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac on all hosts... 2024-09-05T12:15:34.753 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac pull 2024-09-05T12:15:34.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac pull 2024-09-05T12:15:34.759 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac pull 2024-09-05T12:15:36.244 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T12:15:36.322 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T12:15:36.687 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T12:16:13.090 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-05T12:16:13.090 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T12:16:13.090 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232", 2024-09-05T12:16:13.090 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-09-05T12:16:13.091 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945" 2024-09-05T12:16:13.091 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-09-05T12:16:13.091 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232", 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945" 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-05T12:16:15.657 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232", 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945" 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:16:22.890 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:16:22.921 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-09-05T12:16:22.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-09-05T12:16:23.017 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-09-05T12:16:23.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-09-05T12:16:23.106 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-09-05T12:16:23.135 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-09-05T12:16:23.164 INFO:tasks.cephadm:Writing seed config... 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-05T12:16:23.166 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-05T12:16:23.167 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-05T12:16:23.168 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-05T12:16:23.169 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-05T12:16:23.170 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:16:23.170 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-05T12:16:23.204 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 8c9ae41c-6b80-11ef-bcd6-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-05T12:16:23.204 DEBUG:teuthology.orchestra.run.smithi071:mon.a> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service 2024-09-05T12:16:23.248 DEBUG:teuthology.orchestra.run.smithi071:mgr.x> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x.service 2024-09-05T12:16:23.293 INFO:tasks.cephadm:Bootstrapping... 2024-09-05T12:16:23.293 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac -v bootstrap --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.71 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:16:23.548 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-09-05T12:16:23.548 INFO:teuthology.orchestra.run.smithi071.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac', '-v', 'bootstrap', '--fsid', '8c9ae41c-6b80-11ef-bcd6-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.71', '--skip-admin-label'] 2024-09-05T12:16:23.549 INFO:teuthology.orchestra.run.smithi071.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-05T12:16:23.549 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying podman|docker is present... 2024-09-05T12:16:23.595 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-05T12:16:23.595 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying lvm2 is present... 2024-09-05T12:16:23.595 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying time synchronization is in place... 2024-09-05T12:16:23.609 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T12:16:23.609 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T12:16:23.621 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T12:16:23.621 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-05T12:16:23.635 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-09-05T12:16:23.647 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-09-05T12:16:23.647 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-09-05T12:16:23.647 INFO:teuthology.orchestra.run.smithi071.stdout:Repeating the final host check... 2024-09-05T12:16:23.690 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 5.2.2 2024-09-05T12:16:23.690 INFO:teuthology.orchestra.run.smithi071.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-05T12:16:23.690 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl is present 2024-09-05T12:16:23.690 INFO:teuthology.orchestra.run.smithi071.stdout:lvcreate is present 2024-09-05T12:16:23.703 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-05T12:16:23.703 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-05T12:16:23.716 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-05T12:16:23.716 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-05T12:16:23.730 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout enabled 2024-09-05T12:16:23.742 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout active 2024-09-05T12:16:23.742 INFO:teuthology.orchestra.run.smithi071.stdout:Unit chronyd.service is enabled and running 2024-09-05T12:16:23.742 INFO:teuthology.orchestra.run.smithi071.stdout:Host looks OK 2024-09-05T12:16:23.742 INFO:teuthology.orchestra.run.smithi071.stdout:Cluster fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:23.742 INFO:teuthology.orchestra.run.smithi071.stdout:Acquiring lock 139788298417152 on /run/cephadm/8c9ae41c-6b80-11ef-bcd6-c7b262605968.lock 2024-09-05T12:16:23.743 INFO:teuthology.orchestra.run.smithi071.stdout:Lock 139788298417152 acquired on /run/cephadm/8c9ae41c-6b80-11ef-bcd6-c7b262605968.lock 2024-09-05T12:16:23.743 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 3300 ... 2024-09-05T12:16:23.743 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying IP 172.21.15.71 port 6789 ... 2024-09-05T12:16:23.744 INFO:teuthology.orchestra.run.smithi071.stdout:Base mon IP(s) is [172.21.15.71:3300, 172.21.15.71:6789], mon addrv is [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-09-05T12:16:23.750 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.71 metric 100 2024-09-05T12:16:23.750 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.71 metric 100 2024-09-05T12:16:23.757 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-05T12:16:23.757 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-05T12:16:23.757 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-05T12:16:23.765 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-05T12:16:23.765 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-05T12:16:23.765 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T12:16:23.765 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:Mon IP `172.21.15.71` is in CIDR network `172.21.0.0/20` 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-05T12:16:23.766 INFO:teuthology.orchestra.run.smithi071.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stdout 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac... 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Getting image source signatures 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:57827424943d9f7b8fbbdf71efc0609494faf4d42198e71777a8add52eb57a7e 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Copying config sha256:1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 2024-09-05T12:16:24.307 INFO:teuthology.orchestra.run.smithi071.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-05T12:16:25.440 INFO:teuthology.orchestra.run.smithi071.stdout:ceph: stdout ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable) 2024-09-05T12:16:25.440 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph version: ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable) 2024-09-05T12:16:25.440 INFO:teuthology.orchestra.run.smithi071.stdout:Extracting ceph user uid/gid from container image... 2024-09-05T12:16:26.490 INFO:teuthology.orchestra.run.smithi071.stdout:stat: stdout 167 167 2024-09-05T12:16:26.490 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial keys... 2024-09-05T12:16:27.562 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQAaodlmIlT7NBAAZvFTNzcANf78j1ffuAfpow== 2024-09-05T12:16:28.659 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQAbodlm4ynIOhAAOiq44+A/8boJc1aOty3qPw== 2024-09-05T12:16:29.607 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph-authtool: stdout AQAdodlmywn2ARAAuSOT0u4hR9ZXx03227foYg== 2024-09-05T12:16:29.608 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial monmap... 2024-09-05T12:16:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T12:16:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-05T12:16:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T12:16:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:monmaptool for a [v2:172.21.15.71:3300,v1:172.21.15.71:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-05T12:16:30.724 INFO:teuthology.orchestra.run.smithi071.stdout:setting min_mon_release = quincy 2024-09-05T12:16:30.725 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: set fsid to 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:30.725 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-05T12:16:30.725 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:16:30.725 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mon... 2024-09-05T12:16:31.871 INFO:teuthology.orchestra.run.smithi071.stdout:create mon.a on 2024-09-05T12:16:32.184 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-05T12:16:32.403 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-05T12:16:32.677 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968.target. 2024-09-05T12:16:32.677 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968.target → /etc/systemd/system/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968.target. 2024-09-05T12:16:33.009 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a 2024-09-05T12:16:33.010 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service: Unit ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service not loaded. 2024-09-05T12:16:33.286 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968.target.wants/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service → /etc/systemd/system/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@.service. 2024-09-05T12:16:33.875 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:33 smithi071 podman[29468]: 2024-09-05 12:16:33.874809529 +0000 UTC m=+0.363803199 container start 34123dfaa86dd4d72b13f03fda4a0a414e3dbfa407f785bc3fcdb7953015c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:16:34.011 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T12:16:34.011 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-05T12:16:34.024 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T12:16:34.024 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-05T12:16:34.024 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-05T12:16:34.025 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T12:16:34.025 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon to start... 2024-09-05T12:16:34.025 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mon... 2024-09-05T12:16:34.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:33 smithi071 ceph-mon[29487]: mkfs 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:34.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:33 smithi071 ceph-mon[29487]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T12:16:34.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:33 smithi071 bash[29468]: 34123dfaa86dd4d72b13f03fda4a0a414e3dbfa407f785bc3fcdb7953015c48d 2024-09-05T12:16:34.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:33 smithi071 systemd[1]: Started Ceph mon.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:16:35.318 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: monmap epoch 1 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: last_changed 2024-09-05T12:16:29.995084+0000 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: min_mon_release 19 (squid) 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: election_strategy: 1 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: fsmap 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: mgrmap e1: no daemons active 2024-09-05T12:16:35.319 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:34 smithi071 ceph-mon[29487]: from='client.? 172.21.15.71:0/2013052235' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-05T12:16:35.325 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout cluster: 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout id: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout services: 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.721208s) 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-05T12:16:35.333 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout data: 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout pgs: 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.576+0000 7f4b92c00640 1 Processor -- start 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.576+0000 7f4b92c00640 1 -- start start 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.577+0000 7f4b92c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.577+0000 7f4b92c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c10a1e0 con 0x7f4b8c109810 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.578+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.578+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40820/0 (socket says 172.21.15.71:40820) 2024-09-05T12:16:35.334 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.578+0000 7f4b8be00640 1 -- 172.21.15.71:0/970491798 learned_addr learned my addr 172.21.15.71:0/970491798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.578+0000 7f4b8be00640 1 -- 172.21.15.71:0/970491798 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c10a950 con 0x7f4b8c109810 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.581+0000 7f4b8be00640 1 --2- 172.21.15.71:0/970491798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c109c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009820 tx=0x7f4b7c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b646aba0da4c7fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.581+0000 7f4b8aa00640 1 -- 172.21.15.71:0/970491798 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c037070 con 0x7f4b8c109810 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.581+0000 7f4b8aa00640 1 -- 172.21.15.71:0/970491798 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c032440 con 0x7f4b8c109810 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.581+0000 7f4b8aa00640 1 -- 172.21.15.71:0/970491798 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c032740 con 0x7f4b8c109810 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 -- 172.21.15.71:0/970491798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 msgr2=0x7f4b8c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 --2- 172.21.15.71:0/970491798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c109c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009820 tx=0x7f4b7c02a270 comp rx=0 tx=0).stop 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 -- 172.21.15.71:0/970491798 shutdown_connections 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 --2- 172.21.15.71:0/970491798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c109c10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 -- 172.21.15.71:0/970491798 >> 172.21.15.71:0/970491798 conn(0x7f4b8c07ccf0 msgr2=0x7f4b8c07d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:35.335 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 -- 172.21.15.71:0/970491798 shutdown_connections 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.582+0000 7f4b92c00640 1 -- 172.21.15.71:0/970491798 wait complete. 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.583+0000 7f4b92c00640 1 Processor -- start 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.583+0000 7f4b92c00640 1 -- start start 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.584+0000 7f4b92c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.584+0000 7f4b92c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c10aef0 con 0x7f4b8c109810 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.584+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.584+0000 7f4b8be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40836/0 (socket says 172.21.15.71:40836) 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.584+0000 7f4b8be00640 1 -- 172.21.15.71:0/2013052235 learned_addr learned my addr 172.21.15.71:0/2013052235 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.584+0000 7f4b8be00640 1 -- 172.21.15.71:0/2013052235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c19d480 con 0x7f4b8c109810 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b8be00640 1 --2- 172.21.15.71:0/2013052235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c19cf40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0090b0 tx=0x7f4b7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c041070 con 0x7f4b8c109810 2024-09-05T12:16:35.336 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b7c03c7a0 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c19d710 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b7c037040 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1a0400 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4b7c044070 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.585+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b7c03d3f0 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.586+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c10b560 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.589+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4b7c03cbe0 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.651+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4b8c1963b0 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.651+0000 7f4b88c00640 1 -- 172.21.15.71:0/2013052235 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4b8c1963b0 con 0x7f4b8c109810 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 msgr2=0x7f4b8c19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 --2- 172.21.15.71:0/2013052235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c19cf40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0090b0 tx=0x7f4b7c0046c0 comp rx=0 tx=0).stop 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 shutdown_connections 2024-09-05T12:16:35.337 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 --2- 172.21.15.71:0/2013052235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b8c109810 0x7f4b8c19cf40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:35.338 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 >> 172.21.15.71:0/2013052235 conn(0x7f4b8c07ccf0 msgr2=0x7f4b8c191110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:35.338 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 shutdown_connections 2024-09-05T12:16:35.338 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:34.652+0000 7f4b92c00640 1 -- 172.21.15.71:0/2013052235 wait complete. 2024-09-05T12:16:35.338 INFO:teuthology.orchestra.run.smithi071.stdout:mon is available 2024-09-05T12:16:35.338 INFO:teuthology.orchestra.run.smithi071.stdout:Assimilating anything we can from ceph.conf... 2024-09-05T12:16:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:35 smithi071 ceph-mon[29487]: from='client.? 172.21.15.71:0/1900222364' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T12:16:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:35 smithi071 ceph-mon[29487]: from='client.? 172.21.15.71:0/1900222364' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T12:16:36.496 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.823+0000 7fd1bbe00640 1 Processor -- start 2024-09-05T12:16:36.497 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.824+0000 7fd1bbe00640 1 -- start start 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.824+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc07c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.824+0000 7fd1bbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc07c870 con 0x7fd1bc07de40 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.824+0000 7fd1baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc07c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.824+0000 7fd1baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc07c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40852/0 (socket says 172.21.15.71:40852) 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.824+0000 7fd1baa00640 1 -- 172.21.15.71:0/1366614656 learned_addr learned my addr 172.21.15.71:0/1366614656 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.825+0000 7fd1baa00640 1 -- 172.21.15.71:0/1366614656 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc07c9f0 con 0x7fd1bc07de40 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1baa00640 1 --2- 172.21.15.71:0/1366614656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc07c2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009e80 tx=0x7fd1ac02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4750d7fefc2237a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1b9600640 1 -- 172.21.15.71:0/1366614656 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac036070 con 0x7fd1bc07de40 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1b9600640 1 -- 172.21.15.71:0/1366614656 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac031400 con 0x7fd1bc07de40 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1b9600640 1 -- 172.21.15.71:0/1366614656 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac031700 con 0x7fd1bc07de40 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1366614656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 msgr2=0x7fd1bc07c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1bbe00640 1 --2- 172.21.15.71:0/1366614656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc07c2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac009e80 tx=0x7fd1ac02a7a0 comp rx=0 tx=0).stop 2024-09-05T12:16:36.498 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1366614656 shutdown_connections 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1bbe00640 1 --2- 172.21.15.71:0/1366614656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc07c2a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.826+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1366614656 >> 172.21.15.71:0/1366614656 conn(0x7fd1bc102ae0 msgr2=0x7fd1bc104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.827+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1366614656 shutdown_connections 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.827+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1366614656 wait complete. 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.827+0000 7fd1bbe00640 1 Processor -- start 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1bbe00640 1 -- start start 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1bbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1bc07cf90 con 0x7fd1bc07de40 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40858/0 (socket says 172.21.15.71:40858) 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1baa00640 1 -- 172.21.15.71:0/1900222364 learned_addr learned my addr 172.21.15.71:0/1900222364 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:36.499 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.828+0000 7fd1baa00640 1 -- 172.21.15.71:0/1900222364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1bc19d2a0 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1baa00640 1 --2- 172.21.15.71:0/1900222364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc19cd60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac034040 tx=0x7fd1ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac041070 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1ac03c650 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1ac036040 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1bc19d530 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1bc1a0220 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd1ac044070 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.829+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd1ac046c10 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.830+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1bc07c320 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.833+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd1ac03c950 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.893+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fd1bc1968d0 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.897+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fd1bc1968d0 con 0x7fd1bc07de40 2024-09-05T12:16:36.500 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.898+0000 7fd1c0a00640 1 -- 172.21.15.71:0/1900222364 <== mon.0 v2:172.21.15.71:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd1ac031400 con 0x7fd1bc07de40 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 msgr2=0x7fd1bc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 --2- 172.21.15.71:0/1900222364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc19cd60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac034040 tx=0x7fd1ac004a40 comp rx=0 tx=0).stop 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 shutdown_connections 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 --2- 172.21.15.71:0/1900222364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1bc07de40 0x7fd1bc19cd60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 >> 172.21.15.71:0/1900222364 conn(0x7fd1bc102ae0 msgr2=0x7fd1bc103bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 shutdown_connections 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:35.899+0000 7fd1bbe00640 1 -- 172.21.15.71:0/1900222364 wait complete. 2024-09-05T12:16:36.501 INFO:teuthology.orchestra.run.smithi071.stdout:Generating new minimal ceph.conf... 2024-09-05T12:16:37.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:37 smithi071 ceph-mon[29487]: from='client.? 172.21.15.71:0/1729445636' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:16:37.747 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.004+0000 7f0ac1c00640 1 Processor -- start 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.004+0000 7f0ac1c00640 1 -- start start 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.005+0000 7f0ac1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.005+0000 7f0ac1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10a1e0 con 0x7f0abc109810 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.005+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.005+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40860/0 (socket says 172.21.15.71:40860) 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.005+0000 7f0abaa00640 1 -- 172.21.15.71:0/4086793591 learned_addr learned my addr 172.21.15.71:0/4086793591 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.005+0000 7f0abaa00640 1 -- 172.21.15.71:0/4086793591 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc10a950 con 0x7f0abc109810 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.006+0000 7f0abaa00640 1 --2- 172.21.15.71:0/4086793591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc109c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009e80 tx=0x7f0aac02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab6e7c9117b9aa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ab9600640 1 -- 172.21.15.71:0/4086793591 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aac036070 con 0x7f0abc109810 2024-09-05T12:16:37.748 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ab9600640 1 -- 172.21.15.71:0/4086793591 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0aac0318c0 con 0x7f0abc109810 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ab9600640 1 -- 172.21.15.71:0/4086793591 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aac007ab0 con 0x7f0abc109810 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 -- 172.21.15.71:0/4086793591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 msgr2=0x7f0abc109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 --2- 172.21.15.71:0/4086793591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc109c10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009e80 tx=0x7f0aac02a7a0 comp rx=0 tx=0).stop 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 -- 172.21.15.71:0/4086793591 shutdown_connections 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 --2- 172.21.15.71:0/4086793591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc109c10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 -- 172.21.15.71:0/4086793591 >> 172.21.15.71:0/4086793591 conn(0x7f0abc07ccf0 msgr2=0x7f0abc07d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 -- 172.21.15.71:0/4086793591 shutdown_connections 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.007+0000 7f0ac1c00640 1 -- 172.21.15.71:0/4086793591 wait complete. 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.008+0000 7f0ac1c00640 1 Processor -- start 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.008+0000 7f0ac1c00640 1 -- start start 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.009+0000 7f0ac1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.009+0000 7f0ac1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc10dd00 con 0x7f0abc109810 2024-09-05T12:16:37.749 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.009+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.009+0000 7f0abaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40876/0 (socket says 172.21.15.71:40876) 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.009+0000 7f0abaa00640 1 -- 172.21.15.71:0/1729445636 learned_addr learned my addr 172.21.15.71:0/1729445636 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.009+0000 7f0abaa00640 1 -- 172.21.15.71:0/1729445636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc19dd10 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0abaa00640 1 --2- 172.21.15.71:0/1729445636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc19d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009e50 tx=0x7f0aac031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aac036050 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0aac042070 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc19dfa0 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0aac03d8e0 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc1a0c90 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0aac03dbe0 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.010+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0aac0463c0 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.011+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0abc109c10 con 0x7f0abc109810 2024-09-05T12:16:37.750 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.014+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0aac007360 con 0x7f0abc109810 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.074+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f0abc1a1200 con 0x7f0abc109810 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.074+0000 7f0ab3400640 1 -- 172.21.15.71:0/1729445636 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f0aac03bbf0 con 0x7f0abc109810 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 msgr2=0x7f0abc19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 --2- 172.21.15.71:0/1729445636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc19d7d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0aac009e50 tx=0x7f0aac031ca0 comp rx=0 tx=0).stop 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 shutdown_connections 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 --2- 172.21.15.71:0/1729445636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0abc109810 0x7f0abc19d7d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 >> 172.21.15.71:0/1729445636 conn(0x7f0abc07ccf0 msgr2=0x7f0abc107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 shutdown_connections 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:37.075+0000 7f0ac1c00640 1 -- 172.21.15.71:0/1729445636 wait complete. 2024-09-05T12:16:37.751 INFO:teuthology.orchestra.run.smithi071.stdout:Restarting the monitor... 2024-09-05T12:16:38.154 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:37 smithi071 systemd[1]: Stopping Ceph mon.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:16:38.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a[29483]: 2024-09-05T12:16:38.152+0000 7fa0cbe00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:16:38.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a[29483]: 2024-09-05T12:16:38.152+0000 7fa0cbe00640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-05T12:16:38.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:38 smithi071 podman[29728]: 2024-09-05 12:16:38.327184687 +0000 UTC m=+0.342538426 container died 34123dfaa86dd4d72b13f03fda4a0a414e3dbfa407f785bc3fcdb7953015c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-05T12:16:38.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:38 smithi071 podman[29728]: 2024-09-05 12:16:38.739802948 +0000 UTC m=+0.755156675 container cleanup 34123dfaa86dd4d72b13f03fda4a0a414e3dbfa407f785bc3fcdb7953015c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T12:16:38.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:38 smithi071 bash[29728]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a 2024-09-05T12:16:39.267 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 podman[29740]: 2024-09-05 12:16:39.016392245 +0000 UTC m=+0.680437545 container remove 34123dfaa86dd4d72b13f03fda4a0a414e3dbfa407f785bc3fcdb7953015c48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.build-date=20240731) 2024-09-05T12:16:39.268 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-05T12:16:39.268 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 systemd[1]: Stopped Ceph mon.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:16:39.268 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service: Consumed 1.204s CPU time. 2024-09-05T12:16:39.577 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 systemd[1]: Starting Ceph mon.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:16:39.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 podman[29827]: 2024-09-05 12:16:39.481347631 +0000 UTC m=+0.026883905 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:16:39.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 podman[29827]: 2024-09-05 12:16:39.617114866 +0000 UTC m=+0.162651120 container create e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS) 2024-09-05T12:16:39.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 podman[29827]: 2024-09-05 12:16:39.819251049 +0000 UTC m=+0.364787312 container init e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:16:39.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 podman[29827]: 2024-09-05 12:16:39.825916338 +0000 UTC m=+0.371452586 container start e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T12:16:39.911 INFO:teuthology.orchestra.run.smithi071.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-05T12:16:40.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable), process ceph-mon, pid 2 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: pidfile_write: ignore empty --pid-file 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: load: jerasure load: lrc 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: RocksDB version: 7.9.2 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Git sha 0 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Compile date 2024-09-05 09:22:07 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: DB SUMMARY 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: DB Session ID: M7HUBJN5AF1ZWLBG9XIK 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: CURRENT file: CURRENT 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T12:16:40.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104688 ; 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.error_if_exists: 0 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.create_if_missing: 0 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.paranoid_checks: 1 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.env: 0x5621e2271940 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.info_log: 0x56221db3d6a0 2024-09-05T12:16:40.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.statistics: (nil) 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.use_fsync: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_log_file_size: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.allow_fallocate: 1 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.use_direct_reads: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T12:16:40.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.db_log_dir: 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.wal_dir: 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.write_buffer_manager: 0x56221db4cdc0 2024-09-05T12:16:40.165 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.unordered_write: 0 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.row_cache: None 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.wal_filter: None 2024-09-05T12:16:40.166 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.two_write_queues: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.wal_compression: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.atomic_flush: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.log_readahead_size: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T12:16:40.167 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T12:16:40.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T12:16:40.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T12:16:40.168 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T12:16:40.169 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_background_jobs: 2 2024-09-05T12:16:40.169 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_background_compactions: -1 2024-09-05T12:16:40.169 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_subcompactions: 1 2024-09-05T12:16:40.169 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_open_files: -1 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T12:16:40.170 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_background_flushes: -1 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Compression algorithms supported: 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kZSTD supported: 0 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kXpressCompression supported: 0 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kBZip2Compression supported: 0 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kLZ4Compression supported: 1 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kZlibCompression supported: 1 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: kSnappyCompression supported: 1 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T12:16:40.171 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.merge_operator: 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_filter: None 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56221db3d300) 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T12:16:40.172 INFO:journalctl@ceph.mon.a.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_type: 0 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_index_type: 0 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_shortening: 1 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: checksum: 4 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: no_block_cache: 0 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache: 0x56221db311f0 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_options: 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: capacity : 536870912 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: num_shard_bits : 4 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: strict_capacity_limit : 0 2024-09-05T12:16:40.173 INFO:journalctl@ceph.mon.a.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_cache_compressed: (nil) 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: persistent_cache: (nil) 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size: 4096 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_size_deviation: 10 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_restart_interval: 16 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: index_block_restart_interval: 1 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: metadata_block_size: 4096 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: partition_filters: 0 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: use_delta_encoding: 1 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: filter_policy: bloomfilter 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: whole_key_filtering: 1 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: verify_compression: 0 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-09-05T12:16:40.174 INFO:journalctl@ceph.mon.a.smithi071.stdout: format_version: 5 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_index_compression: 1 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout: block_align: 0 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout: max_auto_readahead_size: 262144 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout: prepopulate_block_cache: 0 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout: initial_auto_readahead_size: 8192 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression: NoCompression 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T12:16:40.175 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.num_levels: 7 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T12:16:40.176 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T12:16:40.177 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T12:16:40.178 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T12:16:40.179 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.inplace_update_support: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.bloom_locality: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.max_successive_merges: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.ttl: 2592000 2024-09-05T12:16:40.180 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.enable_blob_files: false 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.min_blob_size: 0 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T12:16:40.181 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 18415d5f-78a1-4e9f-96d6-079d2be52b9f 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725538599867580, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725538599869290, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99843, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97725, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14159, "raw_average_key_size": 49, "raw_value_size": 90034, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725538599, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "18415d5f-78a1-4e9f-96d6-079d2be52b9f", "db_session_id": "M7HUBJN5AF1ZWLBG9XIK", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725538599869578, "job": 1, "event": "recovery_finished"} 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56221db60e00 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: DB pointer 0x56221dc5c000 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T12:16:40.182 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** DB Stats ** 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: L0 2/0 99.36 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:16:40.183 INFO:journalctl@ceph.mon.a.smithi071.stdout: Sum 2/0 99.36 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 89.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** Compaction Stats [default] ** 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 89.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T12:16:40.184 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: Cumulative compaction: 0.00 GB write, 15.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: Interval compaction: 0.00 GB write, 15.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: Block cache BinnedLRUCache@0x56221db311f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: starting mon.a rank 0 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???) e1 preinit fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).mds e1 new map 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).mds e1 print_map 2024-09-05T12:16:40.185 INFO:journalctl@ceph.mon.a.smithi071.stdout: e1 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout: btime 2024-09-05T12:16:33:931477+0000 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout: legacy client fscid: -1 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout: 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout: No filesystems configured 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 bash[29827]: e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 systemd[1]: Started Ceph mon.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:16:40.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: monmap epoch 1 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: last_changed 2024-09-05T12:16:29.995084+0000 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: min_mon_release 19 (squid) 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: election_strategy: 1 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: fsmap 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: osdmap e1: 0 total, 0 up, 0 in 2024-09-05T12:16:40.187 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:39 smithi071 ceph-mon[29845]: mgrmap e1: no daemons active 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.396+0000 7f0223200640 1 Processor -- start 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.396+0000 7f0223200640 1 -- start start 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.396+0000 7f0223200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f0224109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.396+0000 7f0223200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f022410a1e0 con 0x7f0224109810 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.397+0000 7f0221e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f0224109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.397+0000 7f0221e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f0224109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40882/0 (socket says 172.21.15.71:40882) 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.397+0000 7f0221e00640 1 -- 172.21.15.71:0/3006368235 learned_addr learned my addr 172.21.15.71:0/3006368235 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.397+0000 7f0221e00640 1 -- 172.21.15.71:0/3006368235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022410a730 con 0x7f0224109810 2024-09-05T12:16:41.098 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.400+0000 7f0221e00640 1 --2- 172.21.15.71:0/3006368235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f0224109c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0214009820 tx=0x7f021402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5f954d070b54e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.400+0000 7f0221400640 1 -- 172.21.15.71:0/3006368235 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214037070 con 0x7f0224109810 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.400+0000 7f0221400640 1 -- 172.21.15.71:0/3006368235 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0214032900 con 0x7f0224109810 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.400+0000 7f0221400640 1 -- 172.21.15.71:0/3006368235 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214030850 con 0x7f0224109810 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.400+0000 7f0223200640 1 -- 172.21.15.71:0/3006368235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 msgr2=0x7f0224109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.400+0000 7f0223200640 1 --2- 172.21.15.71:0/3006368235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f0224109c10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0214009820 tx=0x7f021402a270 comp rx=0 tx=0).stop 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.401+0000 7f0223200640 1 -- 172.21.15.71:0/3006368235 shutdown_connections 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.401+0000 7f0223200640 1 --2- 172.21.15.71:0/3006368235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f0224109c10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.401+0000 7f0223200640 1 -- 172.21.15.71:0/3006368235 >> 172.21.15.71:0/3006368235 conn(0x7f022407cb00 msgr2=0x7f022407cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.401+0000 7f0223200640 1 -- 172.21.15.71:0/3006368235 shutdown_connections 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.401+0000 7f0223200640 1 -- 172.21.15.71:0/3006368235 wait complete. 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0223200640 1 Processor -- start 2024-09-05T12:16:41.099 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0223200640 1 -- start start 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0223200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f022419d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0223200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f022410d9f0 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0221e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f022419d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0221e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f022419d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40898/0 (socket says 172.21.15.71:40898) 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.402+0000 7f0221e00640 1 -- 172.21.15.71:0/3387166694 learned_addr learned my addr 172.21.15.71:0/3387166694 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f0221e00640 1 -- 172.21.15.71:0/3387166694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022419db10 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f0221e00640 1 --2- 172.21.15.71:0/3387166694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f022419d5d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f02140097f0 tx=0x7f0214030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214042070 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0214032ca0 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0214037040 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022419dda0 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.403+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02241a0a90 con 0x7f0224109810 2024-09-05T12:16:41.100 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.404+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0214045070 con 0x7f0224109810 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.404+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0214047af0 con 0x7f0224109810 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.404+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0224109c90 con 0x7f0224109810 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.407+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f021403db10 con 0x7f0224109810 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.467+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f02241a1070 con 0x7f0224109810 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.470+0000 7f021aa00640 1 -- 172.21.15.71:0/3387166694 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f021403d430 con 0x7f0224109810 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 msgr2=0x7f022419d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 --2- 172.21.15.71:0/3387166694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f022419d5d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f02140097f0 tx=0x7f0214030ec0 comp rx=0 tx=0).stop 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 shutdown_connections 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 --2- 172.21.15.71:0/3387166694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0224109810 0x7f022419d5d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 >> 172.21.15.71:0/3387166694 conn(0x7f022407cb00 msgr2=0x7f0224107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 shutdown_connections 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:40.471+0000 7f0223200640 1 -- 172.21.15.71:0/3387166694 wait complete. 2024-09-05T12:16:41.101 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-05T12:16:41.125 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:16:41.125 INFO:teuthology.orchestra.run.smithi071.stdout:Creating mgr... 2024-09-05T12:16:41.126 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-05T12:16:41.126 INFO:teuthology.orchestra.run.smithi071.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-05T12:16:41.456 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T12:16:41.457 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Failed to reset failed state of unit ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x.service: Unit ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x.service not loaded. 2024-09-05T12:16:41.683 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968.target.wants/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@.service. 2024-09-05T12:16:41.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:41 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3387166694' entity='client.admin' 2024-09-05T12:16:41.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:41 smithi071 systemd[1]: Starting Ceph mgr.x for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:16:42.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:42 smithi071 podman[30056]: 2024-09-05 12:16:41.926040914 +0000 UTC m=+0.032772081 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:16:46.661 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:46 smithi071 podman[30056]: 2024-09-05 12:16:46.190736852 +0000 UTC m=+4.297467991 container create 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-05T12:16:48.264 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T12:16:48.265 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-05T12:16:48.279 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T12:16:48.279 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-05T12:16:48.279 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-05T12:16:48.279 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to enable service . firewalld.service is not available 2024-09-05T12:16:48.296 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T12:16:48.296 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-05T12:16:48.310 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T12:16:48.310 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-05T12:16:48.310 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-05T12:16:48.310 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-05T12:16:48.310 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr to start... 2024-09-05T12:16:48.310 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr... 2024-09-05T12:16:48.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 podman[30056]: 2024-09-05 12:16:48.110330879 +0000 UTC m=+6.217062021 container init 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T12:16:48.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 podman[30056]: 2024-09-05 12:16:48.116389573 +0000 UTC m=+6.223120713 container start 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:16:48.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 bash[30056]: 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 2024-09-05T12:16:48.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 systemd[1]: Started Ceph mgr.x for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:16:48.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:48.285+0000 7f9651a30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:16:48.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:48.358+0000 7f9651a30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:16:48.767 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:48.550+0000 7f9651a30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:16:48.767 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:48.637+0000 7f9651a30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:16:49.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:49 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1450316131' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T12:16:49.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:48.813+0000 7f9651a30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:16:49.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:48.992+0000 7f9651a30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:16:49.625 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:49.375+0000 7f9651a30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:16:49.626 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:49.446+0000 7f9651a30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:16:49.636 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:49.636 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:16:49.636 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:16:49.636 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T12:16:49.636 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T12:16:49.636 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T12:16:49.637 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T12:16:49.638 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.638 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T12:16:49.638 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:49.638 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T12:16:49.638 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T12:16:49.638 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T12:16:49.647 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T12:16:49.647 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T12:16:49.648 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T12:16:33:931477+0000", 2024-09-05T12:16:49.649 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T12:16:49.650 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T12:16:33.932221+0000", 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T12:16:49.651 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.874+0000 7f1c2be00640 1 Processor -- start 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.874+0000 7f1c2be00640 1 -- start start 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.874+0000 7f1c2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.874+0000 7f1c2be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c07b740 con 0x7f1c2c07cd10 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.875+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:49.652 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.875+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c07b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52586/0 (socket says 172.21.15.71:52586) 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.875+0000 7f1c2aa00640 1 -- 172.21.15.71:0/1699033378 learned_addr learned my addr 172.21.15.71:0/1699033378 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.875+0000 7f1c2aa00640 1 -- 172.21.15.71:0/1699033378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c07b8c0 con 0x7f1c2c07cd10 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.876+0000 7f1c2aa00640 1 --2- 172.21.15.71:0/1699033378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c07b170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009820 tx=0x7f1c1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f550c59dbe7cc771 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.876+0000 7f1c29600640 1 -- 172.21.15.71:0/1699033378 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c1c037070 con 0x7f1c2c07cd10 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.876+0000 7f1c29600640 1 -- 172.21.15.71:0/1699033378 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c1c032900 con 0x7f1c2c07cd10 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.876+0000 7f1c29600640 1 -- 172.21.15.71:0/1699033378 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c1c030850 con 0x7f1c2c07cd10 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 -- 172.21.15.71:0/1699033378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 msgr2=0x7f1c2c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:49.653 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 --2- 172.21.15.71:0/1699033378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c07b170 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009820 tx=0x7f1c1c02a270 comp rx=0 tx=0).stop 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 -- 172.21.15.71:0/1699033378 shutdown_connections 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 --2- 172.21.15.71:0/1699033378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c07b170 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 -- 172.21.15.71:0/1699033378 >> 172.21.15.71:0/1699033378 conn(0x7f1c2c102b90 msgr2=0x7f1c2c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 -- 172.21.15.71:0/1699033378 shutdown_connections 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.877+0000 7f1c2be00640 1 -- 172.21.15.71:0/1699033378 wait complete. 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.878+0000 7f1c2be00640 1 Processor -- start 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.878+0000 7f1c2be00640 1 -- start start 2024-09-05T12:16:49.654 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.879+0000 7f1c2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.879+0000 7f1c2be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c10d9d0 con 0x7f1c2c07cd10 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.879+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.879+0000 7f1c2aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52600/0 (socket says 172.21.15.71:52600) 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.879+0000 7f1c2aa00640 1 -- 172.21.15.71:0/1450316131 learned_addr learned my addr 172.21.15.71:0/1450316131 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.879+0000 7f1c2aa00640 1 -- 172.21.15.71:0/1450316131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c19da80 con 0x7f1c2c07cd10 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.880+0000 7f1c2aa00640 1 --2- 172.21.15.71:0/1450316131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c19d540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097f0 tx=0x7f1c1c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.880+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c1c037040 con 0x7f1c2c07cd10 2024-09-05T12:16:49.655 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.880+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c1c040070 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.880+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c19dd10 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.880+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c1c03a390 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.880+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c1a0a00 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.881+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c1c03a630 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.881+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c1c0459a0 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.881+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c07b1f0 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.884+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1c1c045460 con 0x7f1c2c07cd10 2024-09-05T12:16:49.656 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.948+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1c2c1a0c20 con 0x7f1c2c07cd10 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.948+0000 7f1c30a00640 1 -- 172.21.15.71:0/1450316131 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f1c1c03b430 con 0x7f1c2c07cd10 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 msgr2=0x7f1c2c19d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 --2- 172.21.15.71:0/1450316131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c19d540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097f0 tx=0x7f1c1c030d80 comp rx=0 tx=0).stop 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 shutdown_connections 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 --2- 172.21.15.71:0/1450316131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c2c07cd10 0x7f1c2c19d540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 >> 172.21.15.71:0/1450316131 conn(0x7f1c2c102b90 msgr2=0x7f1c2c1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 shutdown_connections 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:48.950+0000 7f1c2be00640 1 -- 172.21.15.71:0/1450316131 wait complete. 2024-09-05T12:16:49.657 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (1/15)... 2024-09-05T12:16:49.910 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:49.658+0000 7f9651a30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:16:50.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:50 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:50.748+0000 7f9651a30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:16:50.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:50 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:50.826+0000 7f9651a30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:16:52.063 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:51 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:51.752+0000 7f9651a30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:16:52.063 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:51 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:51.892+0000 7f9651a30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:16:52.063 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.061+0000 7f9651a30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:16:52.322 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3959069734' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T12:16:52.322 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.139+0000 7f9651a30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:16:52.322 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.321+0000 7f9651a30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:16:52.645 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.404+0000 7f9651a30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:16:52.645 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.475+0000 7f9651a30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:16:52.910 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.664+0000 7f9651a30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:16:52.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.742+0000 7f9651a30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:16:52.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.822+0000 7f9651a30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:16:53.131 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:53.131 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:16:53.131 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-05T12:16:53.132 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T12:16:53.133 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T12:16:53.141 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T12:16:33:931477+0000", 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T12:16:53.142 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T12:16:33.932221+0000", 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:16:53.143 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8f5800640 1 Processor -- start 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8f5800640 1 -- start start 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8f5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f01077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8f5800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f0107d70 con 0x7fa8f01073a0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f01077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f01077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52608/0 (socket says 172.21.15.71:52608) 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.181+0000 7fa8efe00640 1 -- 172.21.15.71:0/1996757949 learned_addr learned my addr 172.21.15.71:0/1996757949 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.182+0000 7fa8efe00640 1 -- 172.21.15.71:0/1996757949 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f01085a0 con 0x7fa8f01073a0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.183+0000 7fa8efe00640 1 --2- 172.21.15.71:0/1996757949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f01077a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009820 tx=0x7fa8e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730d93d13494ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.183+0000 7fa8eea00640 1 -- 172.21.15.71:0/1996757949 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0037070 con 0x7fa8f01073a0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.183+0000 7fa8eea00640 1 -- 172.21.15.71:0/1996757949 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8e0032900 con 0x7fa8f01073a0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.183+0000 7fa8eea00640 1 -- 172.21.15.71:0/1996757949 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0030850 con 0x7fa8f01073a0 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 -- 172.21.15.71:0/1996757949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 msgr2=0x7fa8f01077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:53.144 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 --2- 172.21.15.71:0/1996757949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f01077a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009820 tx=0x7fa8e002a270 comp rx=0 tx=0).stop 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 -- 172.21.15.71:0/1996757949 shutdown_connections 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 --2- 172.21.15.71:0/1996757949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f01077a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 -- 172.21.15.71:0/1996757949 >> 172.21.15.71:0/1996757949 conn(0x7fa8f0102b90 msgr2=0x7fa8f0104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 -- 172.21.15.71:0/1996757949 shutdown_connections 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.184+0000 7fa8f5800640 1 -- 172.21.15.71:0/1996757949 wait complete. 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8f5800640 1 Processor -- start 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8f5800640 1 -- start start 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8f5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f007ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8f5800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f010b950 con 0x7fa8f01073a0 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f007ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f007ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52612/0 (socket says 172.21.15.71:52612) 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.185+0000 7fa8efe00640 1 -- 172.21.15.71:0/3959069734 learned_addr learned my addr 172.21.15.71:0/3959069734 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.186+0000 7fa8efe00640 1 -- 172.21.15.71:0/3959069734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f007cf40 con 0x7fa8f01073a0 2024-09-05T12:16:53.145 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.186+0000 7fa8efe00640 1 --2- 172.21.15.71:0/3959069734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f007ca00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009940 tx=0x7fa8e0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.186+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0042070 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.186+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8e00308c0 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.186+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0037040 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.186+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f007b130 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.187+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f007b670 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.187+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa8e0045070 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.187+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8e0047b70 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.188+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f010bad0 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.190+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fa8e00327b0 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.254+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa8f0107820 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.254+0000 7fa8ecc00640 1 -- 172.21.15.71:0/3959069734 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1292 (secure 0 0 0) 0x7fa8e003c760 con 0x7fa8f01073a0 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.255+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 msgr2=0x7fa8f007ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:53.146 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.255+0000 7fa8f5800640 1 --2- 172.21.15.71:0/3959069734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f007ca00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009940 tx=0x7fa8e0032ce0 comp rx=0 tx=0).stop 2024-09-05T12:16:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.255+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 shutdown_connections 2024-09-05T12:16:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.255+0000 7fa8f5800640 1 --2- 172.21.15.71:0/3959069734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa8f01073a0 0x7fa8f007ca00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.255+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 >> 172.21.15.71:0/3959069734 conn(0x7fa8f0102b90 msgr2=0x7fa8f0103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.256+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 shutdown_connections 2024-09-05T12:16:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:52.256+0000 7fa8f5800640 1 -- 172.21.15.71:0/3959069734 wait complete. 2024-09-05T12:16:53.147 INFO:teuthology.orchestra.run.smithi071.stdout:mgr not available, waiting (2/15)... 2024-09-05T12:16:53.410 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.918+0000 7f9651a30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:16:53.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:52 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:52.988+0000 7f9651a30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:16:53.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:53 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:53.058+0000 7f9651a30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:16:54.410 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:53 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:53.961+0000 7f9651a30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: Activating manager daemon x 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: mgrmap e2: x(active, starting, since 0.00426371s) 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: Manager daemon x is now available 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T12:16:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' 2024-09-05T12:16:54.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T12:16:54.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' 2024-09-05T12:16:54.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:54 smithi071 ceph-mon[29845]: from='mgr.14100 172.21.15.71:0/3369994056' entity='mgr.x' 2024-09-05T12:16:56.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:55 smithi071 ceph-mon[29845]: mgrmap e3: x(active, since 1.00976s) 2024-09-05T12:16:56.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:55 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3132797419' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-05T12:16:56.586 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "health": { 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 0 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:56.587 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "a" 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-05T12:16:56.588 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:56.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-05T12:16:56.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-05T12:16:56.589 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-05T12:16:56.598 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "btime": "2024-09-05T12:16:33:931477+0000", 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-05T12:16:56.599 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "restful" 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ], 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "modified": "2024-09-05T12:16:33.932221+0000", 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout }, 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.631+0000 7f9de3600640 1 Processor -- start 2024-09-05T12:16:56.600 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.631+0000 7f9de3600640 1 -- start start 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.631+0000 7f9de3600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.631+0000 7f9de3600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc081ee0 con 0x7f9ddc081510 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.632+0000 7f9de0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.632+0000 7f9de0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc081910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42034/0 (socket says 172.21.15.71:42034) 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.632+0000 7f9de0c00640 1 -- 172.21.15.71:0/2029555331 learned_addr learned my addr 172.21.15.71:0/2029555331 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.632+0000 7f9de0c00640 1 -- 172.21.15.71:0/2029555331 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc10ed70 con 0x7f9ddc081510 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.633+0000 7f9de0c00640 1 --2- 172.21.15.71:0/2029555331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc081910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009820 tx=0x7f9dd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf11e8d7656a24f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.633+0000 7f9ddb400640 1 -- 172.21.15.71:0/2029555331 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0037070 con 0x7f9ddc081510 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.633+0000 7f9ddb400640 1 -- 172.21.15.71:0/2029555331 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dd0032900 con 0x7f9ddc081510 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.633+0000 7f9ddb400640 1 -- 172.21.15.71:0/2029555331 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0030850 con 0x7f9ddc081510 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 -- 172.21.15.71:0/2029555331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 msgr2=0x7f9ddc081910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 --2- 172.21.15.71:0/2029555331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc081910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009820 tx=0x7f9dd002a270 comp rx=0 tx=0).stop 2024-09-05T12:16:56.601 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 -- 172.21.15.71:0/2029555331 shutdown_connections 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 --2- 172.21.15.71:0/2029555331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc081910 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 -- 172.21.15.71:0/2029555331 >> 172.21.15.71:0/2029555331 conn(0x7f9ddc07ccf0 msgr2=0x7f9ddc07f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 -- 172.21.15.71:0/2029555331 shutdown_connections 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.634+0000 7f9de3600640 1 -- 172.21.15.71:0/2029555331 wait complete. 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.635+0000 7f9de3600640 1 Processor -- start 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.635+0000 7f9de3600640 1 -- start start 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.635+0000 7f9de3600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc1a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.635+0000 7f9de3600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ddc112120 con 0x7f9ddc081510 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9de0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc1a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9de0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc1a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42044/0 (socket says 172.21.15.71:42044) 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9de0c00640 1 -- 172.21.15.71:0/3132797419 learned_addr learned my addr 172.21.15.71:0/3132797419 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9de0c00640 1 -- 172.21.15.71:0/3132797419 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ddc1a2130 con 0x7f9ddc081510 2024-09-05T12:16:56.602 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9de0c00640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc1a1bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097f0 tx=0x7f9dd0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0042070 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9dd0032ca0 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9dd0037040 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.636+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ddc1a23c0 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.637+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ddc1a50b0 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.638+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f9dd0045070 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.638+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ddc081990 con 0x7f9ddc081510 2024-09-05T12:16:56.603 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.638+0000 7f9dd9600640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f9dc4038e60 0x7f9dc403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.638+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9dd006bc40 con 0x7f9ddc081510 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.638+0000 7f9ddbe00640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f9dc4038e60 0x7f9dc403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.639+0000 7f9ddbe00640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f9dc4038e60 0x7f9dc403b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80098c0 tx=0x7f9dc8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.644+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9dd004b170 con 0x7f9ddc081510 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.884+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9ddc1a5730 con 0x7f9ddc081510 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.885+0000 7f9dd9600640 1 -- 172.21.15.71:0/3132797419 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f9dd003d430 con 0x7f9ddc081510 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f9dc4038e60 msgr2=0x7f9dc403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f9dc4038e60 0x7f9dc403b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9dc80098c0 tx=0x7f9dc8006db0 comp rx=0 tx=0).stop 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 msgr2=0x7f9ddc1a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc1a1bf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9dd00097f0 tx=0x7f9dd0030ec0 comp rx=0 tx=0).stop 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 shutdown_connections 2024-09-05T12:16:56.604 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f9dc4038e60 0x7f9dc403b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:56.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 --2- 172.21.15.71:0/3132797419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ddc081510 0x7f9ddc1a1bf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:56.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 >> 172.21.15.71:0/3132797419 conn(0x7f9ddc07ccf0 msgr2=0x7f9ddc07d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:56.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 shutdown_connections 2024-09-05T12:16:56.605 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:55.887+0000 7f9de3600640 1 -- 172.21.15.71:0/3132797419 wait complete. 2024-09-05T12:16:56.605 INFO:teuthology.orchestra.run.smithi071.stdout:mgr is available 2024-09-05T12:16:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:56 smithi071 ceph-mon[29845]: mgrmap e4: x(active, since 2s) 2024-09-05T12:16:58.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:58.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [global] 2024-09-05T12:16:58.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout fsid = 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.71:3300,v1:172.21.15.71:6789] 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout [osd] 2024-09-05T12:16:58.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.085+0000 7f96edc00640 1 Processor -- start 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.085+0000 7f96edc00640 1 -- start start 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.085+0000 7f96edc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e807c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.085+0000 7f96edc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96e807c850 con 0x7f96e807de20 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.086+0000 7f96e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e807c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.086+0000 7f96e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e807c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42054/0 (socket says 172.21.15.71:42054) 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.086+0000 7f96e6a00640 1 -- 172.21.15.71:0/3634487210 learned_addr learned my addr 172.21.15.71:0/3634487210 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.086+0000 7f96e6a00640 1 -- 172.21.15.71:0/3634487210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e807c9d0 con 0x7f96e807de20 2024-09-05T12:16:58.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.087+0000 7f96e6a00640 1 --2- 172.21.15.71:0/3634487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e807c280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009820 tx=0x7f96d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e04dcb9ca5e3ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.087+0000 7f96e5600640 1 -- 172.21.15.71:0/3634487210 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d8037070 con 0x7f96e807de20 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.087+0000 7f96e5600640 1 -- 172.21.15.71:0/3634487210 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96d8032900 con 0x7f96e807de20 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.087+0000 7f96e5600640 1 -- 172.21.15.71:0/3634487210 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d8030850 con 0x7f96e807de20 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 -- 172.21.15.71:0/3634487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 msgr2=0x7f96e807c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 --2- 172.21.15.71:0/3634487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e807c280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009820 tx=0x7f96d802a270 comp rx=0 tx=0).stop 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 -- 172.21.15.71:0/3634487210 shutdown_connections 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 --2- 172.21.15.71:0/3634487210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e807c280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 -- 172.21.15.71:0/3634487210 >> 172.21.15.71:0/3634487210 conn(0x7f96e8102cc0 msgr2=0x7f96e8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 -- 172.21.15.71:0/3634487210 shutdown_connections 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.088+0000 7f96edc00640 1 -- 172.21.15.71:0/3634487210 wait complete. 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.089+0000 7f96edc00640 1 Processor -- start 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.089+0000 7f96edc00640 1 -- start start 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.089+0000 7f96edc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:58.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.089+0000 7f96edc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96e810db60 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42064/0 (socket says 172.21.15.71:42064) 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96e6a00640 1 -- 172.21.15.71:0/2274102346 learned_addr learned my addr 172.21.15.71:0/2274102346 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96e6a00640 1 -- 172.21.15.71:0/2274102346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e819dc70 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96e6a00640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e819d730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097f0 tx=0x7f96d8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d8037040 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f96d8040070 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e819df00 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.090+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96d803a390 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.091+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a0bf0 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.092+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f96d8032900 con 0x7f96e807de20 2024-09-05T12:16:58.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.092+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e807c300 con 0x7f96e807de20 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.092+0000 7f96df400640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f96cc038f70 0x7f96cc03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.092+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f96d806bf90 con 0x7f96e807de20 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.092+0000 7f96e6000640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f96cc038f70 0x7f96cc03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.093+0000 7f96e6000640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f96cc038f70 0x7f96cc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96d40098c0 tx=0x7f96d4006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.097+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f96d8032bf0 con 0x7f96e807de20 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.274+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f96e819e380 con 0x7f96e807de20 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.275+0000 7f96df400640 1 -- 172.21.15.71:0/2274102346 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f96d8044ca0 con 0x7f96e807de20 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f96cc038f70 msgr2=0x7f96cc03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f96cc038f70 0x7f96cc03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f96d40098c0 tx=0x7f96d4006db0 comp rx=0 tx=0).stop 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 msgr2=0x7f96e819d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e819d730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f96d80097f0 tx=0x7f96d8030d80 comp rx=0 tx=0).stop 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 shutdown_connections 2024-09-05T12:16:58.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f96cc038f70 0x7f96cc03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 --2- 172.21.15.71:0/2274102346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96e807de20 0x7f96e819d730 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.277+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 >> 172.21.15.71:0/2274102346 conn(0x7f96e8102cc0 msgr2=0x7f96e8103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.278+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 shutdown_connections 2024-09-05T12:16:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:57.278+0000 7f96edc00640 1 -- 172.21.15.71:0/2274102346 wait complete. 2024-09-05T12:16:58.046 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling cephadm module... 2024-09-05T12:16:58.288 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:57 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2274102346' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-05T12:16:59.072 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:58 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1651309767' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-05T12:16:59.409 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ignoring --setuser ceph since I am not root 2024-09-05T12:16:59.409 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ignoring --setgroup ceph since I am not root 2024-09-05T12:16:59.409 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:59.193+0000 7f0b34c37040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:16:59.409 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:59.265+0000 7f0b34c37040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:16:59.661 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:59.461+0000 7f0b34c37040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:16:59.661 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:59.540+0000 7f0b34c37040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.517+0000 7fde6fe00640 1 Processor -- start 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.518+0000 7fde6fe00640 1 -- start start 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.518+0000 7fde6fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7007b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.518+0000 7fde6fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde7007b7a0 con 0x7fde7007cd70 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.518+0000 7fde6ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7007b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.518+0000 7fde6ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7007b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42066/0 (socket says 172.21.15.71:42066) 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.518+0000 7fde6ea00640 1 -- 172.21.15.71:0/783752274 learned_addr learned my addr 172.21.15.71:0/783752274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:59.802 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.519+0000 7fde6ea00640 1 -- 172.21.15.71:0/783752274 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7007b920 con 0x7fde7007cd70 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.520+0000 7fde6ea00640 1 --2- 172.21.15.71:0/783752274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7007b1d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde60009820 tx=0x7fde6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ceb007fbd3014ba server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.520+0000 7fde6d600640 1 -- 172.21.15.71:0/783752274 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde60037070 con 0x7fde7007cd70 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.520+0000 7fde6d600640 1 -- 172.21.15.71:0/783752274 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde60032900 con 0x7fde7007cd70 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.520+0000 7fde6d600640 1 -- 172.21.15.71:0/783752274 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde60030850 con 0x7fde7007cd70 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 -- 172.21.15.71:0/783752274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 msgr2=0x7fde7007b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 --2- 172.21.15.71:0/783752274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7007b1d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fde60009820 tx=0x7fde6002a270 comp rx=0 tx=0).stop 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 -- 172.21.15.71:0/783752274 shutdown_connections 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 --2- 172.21.15.71:0/783752274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7007b1d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 -- 172.21.15.71:0/783752274 >> 172.21.15.71:0/783752274 conn(0x7fde70102b90 msgr2=0x7fde70104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 -- 172.21.15.71:0/783752274 shutdown_connections 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.521+0000 7fde6fe00640 1 -- 172.21.15.71:0/783752274 wait complete. 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6fe00640 1 Processor -- start 2024-09-05T12:16:59.803 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6fe00640 1 -- start start 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7019d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde7010d9d0 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7019d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7019d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42080/0 (socket says 172.21.15.71:42080) 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.522+0000 7fde6ea00640 1 -- 172.21.15.71:0/1651309767 learned_addr learned my addr 172.21.15.71:0/1651309767 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde6ea00640 1 -- 172.21.15.71:0/1651309767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7019daf0 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde6ea00640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7019d5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde600097f0 tx=0x7fde60030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde60037040 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fde60040070 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7019dd80 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde6003a390 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.523+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde701a0a70 con 0x7fde7007cd70 2024-09-05T12:16:59.804 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.524+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fde6003b430 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.524+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7007b250 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.525+0000 7fde67400640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fde54038f20 0x7fde5403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.525+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fde6006c110 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.525+0000 7fde6e000640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fde54038f20 0x7fde5403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.526+0000 7fde6e000640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fde54038f20 0x7fde5403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0098c0 tx=0x7fde5c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.530+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde600371e0 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.748+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fde7019e200 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.976+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fde60032dd0 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.978+0000 7fde67400640 1 -- 172.21.15.71:0/1651309767 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fde6004acf0 con 0x7fde7007cd70 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fde54038f20 msgr2=0x7fde5403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fde54038f20 0x7fde5403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fde5c0098c0 tx=0x7fde5c006db0 comp rx=0 tx=0).stop 2024-09-05T12:16:59.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 msgr2=0x7fde7019d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7019d5b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fde600097f0 tx=0x7fde60030d80 comp rx=0 tx=0).stop 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 shutdown_connections 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fde54038f20 0x7fde5403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 --2- 172.21.15.71:0/1651309767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde7007cd70 0x7fde7019d5b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 >> 172.21.15.71:0/1651309767 conn(0x7fde70102b90 msgr2=0x7fde70103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 shutdown_connections 2024-09-05T12:16:59.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:16:58.980+0000 7fde6fe00640 1 -- 172.21.15.71:0/1651309767 wait complete. 2024-09-05T12:16:59.975 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:59.712+0000 7f0b34c37040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:16:59.975 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:16:59.880+0000 7f0b34c37040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:17:00.226 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1651309767' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-05T12:17:00.226 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:16:59 smithi071 ceph-mon[29845]: mgrmap e5: x(active, since 5s) 2024-09-05T12:17:00.536 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:00.255+0000 7f0b34c37040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:17:00.536 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:00.330+0000 7f0b34c37040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:17:00.910 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:00.563+0000 7f0b34c37040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:17:01.306 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:00 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1559585542' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T12:17:01.318 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:17:01.318 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-05T12:17:01.318 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.326+0000 7f93ff200640 1 Processor -- start 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.326+0000 7f93ff200640 1 -- start start 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.326+0000 7f93ff200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.326+0000 7f93ff200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f93f807c8b0 con 0x7f93f807de80 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.327+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.327+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42114/0 (socket says 172.21.15.71:42114) 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.327+0000 7f93f7e00640 1 -- 172.21.15.71:0/656460141 learned_addr learned my addr 172.21.15.71:0/656460141 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.327+0000 7f93f7e00640 1 -- 172.21.15.71:0/656460141 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f807ca30 con 0x7f93f807de80 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.328+0000 7f93f7e00640 1 --2- 172.21.15.71:0/656460141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f807c2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009820 tx=0x7f93e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad9f28e8a20cef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.328+0000 7f93f6a00640 1 -- 172.21.15.71:0/656460141 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93e8037070 con 0x7f93f807de80 2024-09-05T12:17:01.319 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.328+0000 7f93f6a00640 1 -- 172.21.15.71:0/656460141 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93e8032900 con 0x7f93f807de80 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.328+0000 7f93f6a00640 1 -- 172.21.15.71:0/656460141 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93e8030850 con 0x7f93f807de80 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.329+0000 7f93ff200640 1 -- 172.21.15.71:0/656460141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 msgr2=0x7f93f807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.329+0000 7f93ff200640 1 --2- 172.21.15.71:0/656460141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f807c2e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009820 tx=0x7f93e802a270 comp rx=0 tx=0).stop 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.329+0000 7f93ff200640 1 -- 172.21.15.71:0/656460141 shutdown_connections 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.329+0000 7f93ff200640 1 --2- 172.21.15.71:0/656460141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f807c2e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.329+0000 7f93ff200640 1 -- 172.21.15.71:0/656460141 >> 172.21.15.71:0/656460141 conn(0x7f93f8102cc0 msgr2=0x7f93f8105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.330+0000 7f93ff200640 1 -- 172.21.15.71:0/656460141 shutdown_connections 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.330+0000 7f93ff200640 1 -- 172.21.15.71:0/656460141 wait complete. 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.330+0000 7f93ff200640 1 Processor -- start 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.331+0000 7f93ff200640 1 -- start start 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.331+0000 7f93ff200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f81a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.331+0000 7f93ff200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8109750 con 0x7f93f807de80 2024-09-05T12:17:01.320 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.331+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f81a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.331+0000 7f93f7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f81a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42122/0 (socket says 172.21.15.71:42122) 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.331+0000 7f93f7e00640 1 -- 172.21.15.71:0/1559585542 learned_addr learned my addr 172.21.15.71:0/1559585542 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93f7e00640 1 -- 172.21.15.71:0/1559585542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f81a20c0 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93f7e00640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f81a1b80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f93e80097f0 tx=0x7f93e8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93e8037040 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f93e8040070 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93e803a390 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f81a2350 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.332+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f81a5040 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.333+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f93e8032900 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.334+0000 7f93f4c00640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 0x7f93dc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.334+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f93e806c0f0 con 0x7f93f807de80 2024-09-05T12:17:01.321 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.334+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f807c360 con 0x7f93f807de80 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.334+0000 7f93f7400640 1 -- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 msgr2=0x7f93dc03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.334+0000 7f93f7400640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 0x7f93dc03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.339+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93e806be50 con 0x7f93f807de80 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.535+0000 7f93f7400640 1 -- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 msgr2=0x7f93dc03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.535+0000 7f93f7400640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 0x7f93dc03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.583+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f93f802dac0 con 0x7f93f807de80 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.583+0000 7f93f4c00640 1 -- 172.21.15.71:0/1559585542 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f93e8040ce0 con 0x7f93f807de80 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 msgr2=0x7f93dc03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 0x7f93dc03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 msgr2=0x7f93f81a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f81a1b80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f93e80097f0 tx=0x7f93e8030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 shutdown_connections 2024-09-05T12:17:01.322 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7f93dc038fc0 0x7f93dc03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:01.323 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 --2- 172.21.15.71:0/1559585542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f807de80 0x7f93f81a1b80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:01.323 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 >> 172.21.15.71:0/1559585542 conn(0x7f93f8102cc0 msgr2=0x7f93f8198820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:01.323 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 shutdown_connections 2024-09-05T12:17:01.323 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:00.586+0000 7f93ff200640 1 -- 172.21.15.71:0/1559585542 wait complete. 2024-09-05T12:17:01.323 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-09-05T12:17:01.323 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 5... 2024-09-05T12:17:01.910 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:01 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:01.684+0000 7f0b34c37040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:17:01.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:01 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:01.767+0000 7f0b34c37040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:17:02.969 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:02.671+0000 7f0b34c37040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:17:02.969 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:02.805+0000 7f0b34c37040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:17:02.969 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:02.966+0000 7f0b34c37040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:17:03.220 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.037+0000 7f0b34c37040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:17:03.220 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.204+0000 7f0b34c37040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:17:03.537 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.285+0000 7f0b34c37040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:17:03.537 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.355+0000 7f0b34c37040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:17:03.537 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.535+0000 7f0b34c37040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:17:03.852 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.610+0000 7f0b34c37040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:17:03.852 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.686+0000 7f0b34c37040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:17:03.853 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.781+0000 7f0b34c37040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:17:04.160 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.850+0000 7f0b34c37040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:17:04.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:03.920+0000 7f0b34c37040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:17:05.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:04.813+0000 7f0b34c37040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:17:05.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: Active manager daemon x restarted 2024-09-05T12:17:05.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: Activating manager daemon x 2024-09-05T12:17:05.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: osdmap e2: 0 total, 0 up, 0 in 2024-09-05T12:17:05.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: mgrmap e6: x(active, starting, since 0.00616121s) 2024-09-05T12:17:05.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: Manager daemon x is now available 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:05.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:04 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:06.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:05 smithi071 ceph-mon[29845]: Found migration_current of "None". Setting to last migration. 2024-09-05T12:17:06.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:05 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T12:17:06.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:05 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T12:17:06.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:05 smithi071 ceph-mon[29845]: mgrmap e7: x(active, since 1.00653s) 2024-09-05T12:17:06.510 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.806+0000 7fd349000640 1 Processor -- start 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.807+0000 7fd349000640 1 -- start start 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.807+0000 7fd349000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd3441077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.807+0000 7fd349000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd344107d70 con 0x7fd3441073a0 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.807+0000 7fd343400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd3441077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.807+0000 7fd343400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd3441077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42130/0 (socket says 172.21.15.71:42130) 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.807+0000 7fd343400640 1 -- 172.21.15.71:0/2240005255 learned_addr learned my addr 172.21.15.71:0/2240005255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:06.511 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.808+0000 7fd343400640 1 -- 172.21.15.71:0/2240005255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3441085a0 con 0x7fd3441073a0 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.809+0000 7fd343400640 1 --2- 172.21.15.71:0/2240005255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd3441077a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd334009820 tx=0x7fd33402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6359dc0bb3952bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.809+0000 7fd342000640 1 -- 172.21.15.71:0/2240005255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd334037070 con 0x7fd3441073a0 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.809+0000 7fd342000640 1 -- 172.21.15.71:0/2240005255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd334032900 con 0x7fd3441073a0 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.809+0000 7fd342000640 1 -- 172.21.15.71:0/2240005255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd334030850 con 0x7fd3441073a0 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 -- 172.21.15.71:0/2240005255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 msgr2=0x7fd3441077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 --2- 172.21.15.71:0/2240005255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd3441077a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd334009820 tx=0x7fd33402a270 comp rx=0 tx=0).stop 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 -- 172.21.15.71:0/2240005255 shutdown_connections 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 --2- 172.21.15.71:0/2240005255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd3441077a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 -- 172.21.15.71:0/2240005255 >> 172.21.15.71:0/2240005255 conn(0x7fd344102b90 msgr2=0x7fd344104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 -- 172.21.15.71:0/2240005255 shutdown_connections 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.810+0000 7fd349000640 1 -- 172.21.15.71:0/2240005255 wait complete. 2024-09-05T12:17:06.512 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.811+0000 7fd349000640 1 Processor -- start 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.812+0000 7fd349000640 1 -- start start 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.812+0000 7fd349000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd34419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.812+0000 7fd349000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd34410b950 con 0x7fd3441073a0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.812+0000 7fd343400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd34419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.812+0000 7fd343400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd34419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42134/0 (socket says 172.21.15.71:42134) 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.812+0000 7fd343400640 1 -- 172.21.15.71:0/335064572 learned_addr learned my addr 172.21.15.71:0/335064572 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.813+0000 7fd343400640 1 -- 172.21.15.71:0/335064572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34419b8d0 con 0x7fd3441073a0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.813+0000 7fd343400640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd34419b390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd334009940 tx=0x7fd334032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.813+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd334042070 con 0x7fd3441073a0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.813+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd3340308c0 con 0x7fd3441073a0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.813+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34419bb60 con 0x7fd3441073a0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.813+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd334037040 con 0x7fd3441073a0 2024-09-05T12:17:06.513 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.814+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34419e850 con 0x7fd3441073a0 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.814+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd334045070 con 0x7fd3441073a0 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.815+0000 7fd33be00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.815+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 --> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd344107820 con 0x7fd328038fc0 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.815+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd33406c180 con 0x7fd3441073a0 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.815+0000 7fd342a00640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 msgr2=0x7fd32803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:01.815+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:02.016+0000 7fd342a00640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 msgr2=0x7fd32803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:02.016+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:02.417+0000 7fd342a00640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 msgr2=0x7fd32803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:02.417+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:03.218+0000 7fd342a00640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 msgr2=0x7fd32803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:03.218+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T12:17:06.514 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:04.819+0000 7fd342a00640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 msgr2=0x7fd32803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/1559694750 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:04.819+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:04.820+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7fd33403c780 con 0x7fd3441073a0 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:04.820+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 msgr2=0x7fd32803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:04.820+0000 7fd33be00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.819+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fd33406db40 con 0x7fd3441073a0 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.819+0000 7fd33be00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fd32803c700 0x7fd32803eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.819+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd344107820 con 0x7fd32803c700 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.820+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fd32803c700 0x7fd32803eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.821+0000 7fd342a00640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fd32803c700 0x7fd32803eaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd330003980 tx=0x7fd3300092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.822+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd344107820 con 0x7fd32803c700 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.829+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd344107820 con 0x7fd32803c700 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.829+0000 7fd33be00640 1 -- 172.21.15.71:0/335064572 <== mgr.14118 v2:172.21.15.71:6800/3648851306 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd344107820 con 0x7fd32803c700 2024-09-05T12:17:06.515 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fd32803c700 msgr2=0x7fd32803eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fd32803c700 0x7fd32803eaf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd330003980 tx=0x7fd3300092b0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 msgr2=0x7fd34419b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd34419b390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd334009940 tx=0x7fd334032ce0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 shutdown_connections 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fd32803c700 0x7fd32803eaf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:6800/1559694750,v1:172.21.15.71:6801/1559694750] conn(0x7fd328038fc0 0x7fd32803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 --2- 172.21.15.71:0/335064572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3441073a0 0x7fd34419b390 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 >> 172.21.15.71:0/335064572 conn(0x7fd344102b90 msgr2=0x7fd344103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 shutdown_connections 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:05.830+0000 7fd349000640 1 -- 172.21.15.71:0/335064572 wait complete. 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 5 is available 2024-09-05T12:17:06.516 INFO:teuthology.orchestra.run.smithi071.stdout:Setting orchestrator backend to cephadm... 2024-09-05T12:17:07.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:06 smithi071 ceph-mon[29845]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T12:17:07.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:06 smithi071 ceph-mon[29845]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T12:17:07.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:06 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:07.056 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:06 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:07.184+0000 7f0a93e00640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:12:17:07] ENGINE Error in HTTPServer.serve 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: Traceback (most recent call last): 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._connections.run(self.expiration_interval) 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._run(expiration_interval) 2024-09-05T12:17:07.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: s = self.context.wrap_socket( 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: return self.sslsocket_class._create( 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self.do_handshake() 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._sslobj.do_handshake() 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:17:07.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-mon[29845]: mgrmap e8: x(active, since 2s) 2024-09-05T12:17:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:07] ENGINE Bus STARTING 2024-09-05T12:17:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:07 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.054+0000 7f4c4d800640 1 Processor -- start 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.054+0000 7f4c4d800640 1 -- start start 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.054+0000 7f4c4d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c481099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.054+0000 7f4c4d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c48109f80 con 0x7f4c481095b0 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.054+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c481099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.055+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c481099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51670/0 (socket says 172.21.15.71:51670) 2024-09-05T12:17:08.012 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.055+0000 7f4c47e00640 1 -- 172.21.15.71:0/2546694195 learned_addr learned my addr 172.21.15.71:0/2546694195 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.055+0000 7f4c47e00640 1 -- 172.21.15.71:0/2546694195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4810a7b0 con 0x7f4c481095b0 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.056+0000 7f4c47e00640 1 --2- 172.21.15.71:0/2546694195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c481099b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009820 tx=0x7f4c3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77432891d72ffe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.056+0000 7f4c46a00640 1 -- 172.21.15.71:0/2546694195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c38037070 con 0x7f4c481095b0 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.056+0000 7f4c46a00640 1 -- 172.21.15.71:0/2546694195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c38032900 con 0x7f4c481095b0 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.056+0000 7f4c46a00640 1 -- 172.21.15.71:0/2546694195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c38030850 con 0x7f4c481095b0 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 -- 172.21.15.71:0/2546694195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 msgr2=0x7f4c481099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 --2- 172.21.15.71:0/2546694195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c481099b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4c38009820 tx=0x7f4c3802a270 comp rx=0 tx=0).stop 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 -- 172.21.15.71:0/2546694195 shutdown_connections 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 --2- 172.21.15.71:0/2546694195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c481099b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 -- 172.21.15.71:0/2546694195 >> 172.21.15.71:0/2546694195 conn(0x7f4c4807cbc0 msgr2=0x7f4c48107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:08.013 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 -- 172.21.15.71:0/2546694195 shutdown_connections 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.057+0000 7f4c4d800640 1 -- 172.21.15.71:0/2546694195 wait complete. 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.058+0000 7f4c4d800640 1 Processor -- start 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.059+0000 7f4c4d800640 1 -- start start 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.059+0000 7f4c4d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c4819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.059+0000 7f4c4d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4810db60 con 0x7f4c481095b0 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.059+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c4819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.059+0000 7f4c47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c4819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51684/0 (socket says 172.21.15.71:51684) 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.059+0000 7f4c47e00640 1 -- 172.21.15.71:0/2649962909 learned_addr learned my addr 172.21.15.71:0/2649962909 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.060+0000 7f4c47e00640 1 -- 172.21.15.71:0/2649962909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4819dab0 con 0x7f4c481095b0 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.060+0000 7f4c47e00640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c4819d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c380097f0 tx=0x7f4c38030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.060+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c38037040 con 0x7f4c481095b0 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.061+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4c38040070 con 0x7f4c481095b0 2024-09-05T12:17:08.014 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.061+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4819dd40 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.061+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c3803a390 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.061+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c481a0a30 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.062+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c48109a30 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.062+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4c3803b430 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.063+0000 7f4c44c00640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4c2c038f70 0x7f4c2c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.063+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4c3806c520 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.063+0000 7f4c47400640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4c2c038f70 0x7f4c2c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.064+0000 7f4c47400640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4c2c038f70 0x7f4c2c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c340098c0 tx=0x7f4c34006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.068+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c3806be70 con 0x7f4c481095b0 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.261+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f4c48107210 con 0x7f4c2c038f70 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.267+0000 7f4c44c00640 1 -- 172.21.15.71:0/2649962909 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c48107210 con 0x7f4c2c038f70 2024-09-05T12:17:08.015 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4c2c038f70 msgr2=0x7f4c2c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4c2c038f70 0x7f4c2c03b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4c340098c0 tx=0x7f4c34006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 msgr2=0x7f4c4819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c4819d570 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4c380097f0 tx=0x7f4c38030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 shutdown_connections 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4c2c038f70 0x7f4c2c03b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 --2- 172.21.15.71:0/2649962909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c481095b0 0x7f4c4819d570 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 >> 172.21.15.71:0/2649962909 conn(0x7f4c4807cbc0 msgr2=0x7f4c48106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 shutdown_connections 2024-09-05T12:17:08.016 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:07.269+0000 7f4c4d800640 1 -- 172.21.15.71:0/2649962909 wait complete. 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:08 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:07] ENGINE Serving on https://172.21.15.71:7150 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:08 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:07] ENGINE Error in HTTPServer.serve 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:17:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:08 smithi071 ceph-mon[29845]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:08 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:07] ENGINE Serving on http://172.21.15.71:8765 2024-09-05T12:17:09.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:08 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:07] ENGINE Bus STARTED 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.765+0000 7f8d2d400640 1 Processor -- start 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.765+0000 7f8d2d400640 1 -- start start 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.765+0000 7f8d2d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d281099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.765+0000 7f8d2d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d28109f80 con 0x7f8d281095b0 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.765+0000 7f8d27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d281099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.766+0000 7f8d27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d281099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51690/0 (socket says 172.21.15.71:51690) 2024-09-05T12:17:09.686 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.766+0000 7f8d27e00640 1 -- 172.21.15.71:0/1849735131 learned_addr learned my addr 172.21.15.71:0/1849735131 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.766+0000 7f8d27e00640 1 -- 172.21.15.71:0/1849735131 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2810a7b0 con 0x7f8d281095b0 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.767+0000 7f8d27e00640 1 --2- 172.21.15.71:0/1849735131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d281099b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009820 tx=0x7f8d1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5248d61f109b53 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.767+0000 7f8d27400640 1 -- 172.21.15.71:0/1849735131 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d18037070 con 0x7f8d281095b0 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.767+0000 7f8d27400640 1 -- 172.21.15.71:0/1849735131 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d18032900 con 0x7f8d281095b0 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.767+0000 7f8d27400640 1 -- 172.21.15.71:0/1849735131 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d18030850 con 0x7f8d281095b0 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 -- 172.21.15.71:0/1849735131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 msgr2=0x7f8d281099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 --2- 172.21.15.71:0/1849735131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d281099b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8d18009820 tx=0x7f8d1802a270 comp rx=0 tx=0).stop 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 -- 172.21.15.71:0/1849735131 shutdown_connections 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 --2- 172.21.15.71:0/1849735131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d281099b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 -- 172.21.15.71:0/1849735131 >> 172.21.15.71:0/1849735131 conn(0x7f8d2807cbc0 msgr2=0x7f8d28107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 -- 172.21.15.71:0/1849735131 shutdown_connections 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.768+0000 7f8d2d400640 1 -- 172.21.15.71:0/1849735131 wait complete. 2024-09-05T12:17:09.687 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.769+0000 7f8d2d400640 1 Processor -- start 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.770+0000 7f8d2d400640 1 -- start start 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.770+0000 7f8d2d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d2819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.770+0000 7f8d2d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2810db60 con 0x7f8d281095b0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.770+0000 7f8d27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d2819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d2819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51702/0 (socket says 172.21.15.71:51702) 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d27e00640 1 -- 172.21.15.71:0/2889417215 learned_addr learned my addr 172.21.15.71:0/2889417215 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d27e00640 1 -- 172.21.15.71:0/2889417215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2819dab0 con 0x7f8d281095b0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d27e00640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d2819d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d180097f0 tx=0x7f8d18030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d18037040 con 0x7f8d281095b0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8d18040070 con 0x7f8d281095b0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.771+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8d1803a390 con 0x7f8d281095b0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.772+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2819dd40 con 0x7f8d281095b0 2024-09-05T12:17:09.688 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.772+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d281a0a30 con 0x7f8d281095b0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.773+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8d18032900 con 0x7f8d281095b0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.773+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d28109a30 con 0x7f8d281095b0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.773+0000 7f8d25600640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8d0c038f70 0x7f8d0c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.773+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8d1806c540 con 0x7f8d281095b0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.773+0000 7f8d1f400640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8d0c038f70 0x7f8d0c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.774+0000 7f8d1f400640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8d0c038f70 0x7f8d0c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d100098c0 tx=0x7f8d10006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.778+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d1806be90 con 0x7f8d281095b0 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.956+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f8d28107210 con 0x7f8d0c038f70 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.957+0000 7f8d25600640 1 -- 172.21.15.71:0/2889417215 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f8d28107210 con 0x7f8d0c038f70 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.959+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8d0c038f70 msgr2=0x7f8d0c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.959+0000 7f8d2d400640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8d0c038f70 0x7f8d0c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d100098c0 tx=0x7f8d10006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.959+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 msgr2=0x7f8d2819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:09.689 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.959+0000 7f8d2d400640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d2819d570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8d180097f0 tx=0x7f8d18030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.960+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 shutdown_connections 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.960+0000 7f8d2d400640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8d0c038f70 0x7f8d0c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.960+0000 7f8d2d400640 1 --2- 172.21.15.71:0/2889417215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d281095b0 0x7f8d2819d570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.960+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 >> 172.21.15.71:0/2889417215 conn(0x7f8d2807cbc0 msgr2=0x7f8d28106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.960+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 shutdown_connections 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:08.960+0000 7f8d2d400640 1 -- 172.21.15.71:0/2889417215 wait complete. 2024-09-05T12:17:09.690 INFO:teuthology.orchestra.run.smithi071.stdout:Generating ssh key... 2024-09-05T12:17:09.947 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:09 smithi071 ceph-mon[29845]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: Generating public/private rsa key pair. 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: Your identification has been saved in /tmp/tmpcq242iji/key 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: Your public key has been saved in /tmp/tmpcq242iji/key.pub 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: The key fingerprint is: 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: SHA256:i0oZKJijiXPrnj5KTNWwFYBQ9UHtzmjvJAUqz67rLTw ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: The key's randomart image is: 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: +---[RSA 3072]----+ 2024-09-05T12:17:10.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |o.o+o++. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: | . =. .. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: | o .o. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |.... . .. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |=.o o +S | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |=+ + oooo. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |=+. =.o.o | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |.oE* . o. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: |.*X*+ .. | 2024-09-05T12:17:10.912 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: +----[SHA256]-----+ 2024-09-05T12:17:11.350 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4563e00640 1 Processor -- start 2024-09-05T12:17:11.350 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4563e00640 1 -- start start 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4563e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f45641099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4563e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4564109f80 con 0x7f45641095b0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4562a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f45641099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4562a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f45641099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51716/0 (socket says 172.21.15.71:51716) 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.246+0000 7f4562a00640 1 -- 172.21.15.71:0/3109429007 learned_addr learned my addr 172.21.15.71:0/3109429007 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.247+0000 7f4562a00640 1 -- 172.21.15.71:0/3109429007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456410a7b0 con 0x7f45641095b0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.248+0000 7f4562a00640 1 --2- 172.21.15.71:0/3109429007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f45641099b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4554009820 tx=0x7f455402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b382227b7b45ef85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.248+0000 7f4561600640 1 -- 172.21.15.71:0/3109429007 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554037070 con 0x7f45641095b0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.248+0000 7f4561600640 1 -- 172.21.15.71:0/3109429007 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4554032900 con 0x7f45641095b0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.248+0000 7f4561600640 1 -- 172.21.15.71:0/3109429007 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554030850 con 0x7f45641095b0 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 -- 172.21.15.71:0/3109429007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 msgr2=0x7f45641099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 --2- 172.21.15.71:0/3109429007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f45641099b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4554009820 tx=0x7f455402a270 comp rx=0 tx=0).stop 2024-09-05T12:17:11.351 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 -- 172.21.15.71:0/3109429007 shutdown_connections 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 --2- 172.21.15.71:0/3109429007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f45641099b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 -- 172.21.15.71:0/3109429007 >> 172.21.15.71:0/3109429007 conn(0x7f456407cbc0 msgr2=0x7f4564107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 -- 172.21.15.71:0/3109429007 shutdown_connections 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.249+0000 7f4563e00640 1 -- 172.21.15.71:0/3109429007 wait complete. 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.250+0000 7f4563e00640 1 Processor -- start 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.250+0000 7f4563e00640 1 -- start start 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4563e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f456419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4563e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f456410db60 con 0x7f45641095b0 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4562a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f456419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4562a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f456419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51722/0 (socket says 172.21.15.71:51722) 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4562a00640 1 -- 172.21.15.71:0/2406822378 learned_addr learned my addr 172.21.15.71:0/2406822378 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4562a00640 1 -- 172.21.15.71:0/2406822378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456419dab0 con 0x7f45641095b0 2024-09-05T12:17:11.352 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.251+0000 7f4562a00640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f456419d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45540097f0 tx=0x7f4554030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.252+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554037040 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.252+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4554040070 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.252+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456419dd40 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.252+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455403a390 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.252+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45641a0a30 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.253+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f455403b430 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.253+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4564109a30 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.253+0000 7f455b400640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4548038fc0 0x7f454803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.253+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f455406c4e0 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.254+0000 7f4562000640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4548038fc0 0x7f454803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.254+0000 7f4562000640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4548038fc0 0x7f454803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45500098c0 tx=0x7f4550006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.259+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f455406be30 con 0x7f45641095b0 2024-09-05T12:17:11.353 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.432+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f4564107210 con 0x7f4548038fc0 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.605+0000 7f455b400640 1 -- 172.21.15.71:0/2406822378 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4564107210 con 0x7f4548038fc0 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4548038fc0 msgr2=0x7f454803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4548038fc0 0x7f454803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f45500098c0 tx=0x7f4550006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 msgr2=0x7f456419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f456419d570 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f45540097f0 tx=0x7f4554030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 shutdown_connections 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4548038fc0 0x7f454803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 --2- 172.21.15.71:0/2406822378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45641095b0 0x7f456419d570 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.607+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 >> 172.21.15.71:0/2406822378 conn(0x7f456407cbc0 msgr2=0x7f4564106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.608+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 shutdown_connections 2024-09-05T12:17:11.354 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:10.608+0000 7f4563e00640 1 -- 172.21.15.71:0/2406822378 wait complete. 2024-09-05T12:17:11.863 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:11 smithi071 ceph-mon[29845]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:11.863 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:11 smithi071 ceph-mon[29845]: Generating ssh key... 2024-09-05T12:17:11.863 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:11 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:11.863 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:11 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:12.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:12 smithi071 ceph-mon[29845]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.861+0000 7fb6c9a00640 1 Processor -- start 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.861+0000 7fb6c9a00640 1 -- start start 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.861+0000 7fb6c9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c41099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.861+0000 7fb6c9a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4109f80 con 0x7fb6c41095b0 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.861+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c41099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.862+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c41099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51734/0 (socket says 172.21.15.71:51734) 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.862+0000 7fb6c3e00640 1 -- 172.21.15.71:0/276319755 learned_addr learned my addr 172.21.15.71:0/276319755 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.862+0000 7fb6c3e00640 1 -- 172.21.15.71:0/276319755 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c410a7b0 con 0x7fb6c41095b0 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.863+0000 7fb6c3e00640 1 --2- 172.21.15.71:0/276319755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c41099b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009820 tx=0x7fb6b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a98de3d29ea75af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.863+0000 7fb6c2a00640 1 -- 172.21.15.71:0/276319755 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4037070 con 0x7fb6c41095b0 2024-09-05T12:17:12.805 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.863+0000 7fb6c2a00640 1 -- 172.21.15.71:0/276319755 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6b4032900 con 0x7fb6c41095b0 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.863+0000 7fb6c2a00640 1 -- 172.21.15.71:0/276319755 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4030850 con 0x7fb6c41095b0 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 -- 172.21.15.71:0/276319755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 msgr2=0x7fb6c41099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 --2- 172.21.15.71:0/276319755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c41099b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009820 tx=0x7fb6b402a270 comp rx=0 tx=0).stop 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 -- 172.21.15.71:0/276319755 shutdown_connections 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 --2- 172.21.15.71:0/276319755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c41099b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 -- 172.21.15.71:0/276319755 >> 172.21.15.71:0/276319755 conn(0x7fb6c407cbc0 msgr2=0x7fb6c4107360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 -- 172.21.15.71:0/276319755 shutdown_connections 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.864+0000 7fb6c9a00640 1 -- 172.21.15.71:0/276319755 wait complete. 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.865+0000 7fb6c9a00640 1 Processor -- start 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.865+0000 7fb6c9a00640 1 -- start start 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.865+0000 7fb6c9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.865+0000 7fb6c9a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c410db60 con 0x7fb6c41095b0 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.866+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:12.806 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.866+0000 7fb6c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c419d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51746/0 (socket says 172.21.15.71:51746) 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.866+0000 7fb6c3e00640 1 -- 172.21.15.71:0/4138529845 learned_addr learned my addr 172.21.15.71:0/4138529845 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.866+0000 7fb6c3e00640 1 -- 172.21.15.71:0/4138529845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c419dab0 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.866+0000 7fb6c3e00640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c419d570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40097f0 tx=0x7fb6b4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.866+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4037040 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.867+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb6b4040070 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.867+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c419dd40 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.867+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6b403a390 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.867+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c41a0a30 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.868+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb6b4032900 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.868+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4109a30 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.868+0000 7fb6c0c00640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fb6a8038fc0 0x7fb6a803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.868+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb6b406c500 con 0x7fb6c41095b0 2024-09-05T12:17:12.807 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.868+0000 7fb6c3400640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fb6a8038fc0 0x7fb6a803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.869+0000 7fb6c3400640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fb6a8038fc0 0x7fb6a803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00098c0 tx=0x7fb6b0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:11.873+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6b406be50 con 0x7fb6c41095b0 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.050+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fb6c4107210 con 0x7fb6a8038fc0 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.051+0000 7fb6c0c00640 1 -- 172.21.15.71:0/4138529845 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fb6c4107210 con 0x7fb6a8038fc0 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.053+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fb6a8038fc0 msgr2=0x7fb6a803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.053+0000 7fb6c9a00640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fb6a8038fc0 0x7fb6a803b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb6b00098c0 tx=0x7fb6b0006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.053+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 msgr2=0x7fb6c419d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.053+0000 7fb6c9a00640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c419d570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40097f0 tx=0x7fb6b4030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.054+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 shutdown_connections 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.054+0000 7fb6c9a00640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fb6a8038fc0 0x7fb6a803b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.054+0000 7fb6c9a00640 1 --2- 172.21.15.71:0/4138529845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb6c41095b0 0x7fb6c419d570 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.054+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 >> 172.21.15.71:0/4138529845 conn(0x7fb6c407cbc0 msgr2=0x7fb6c4106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.054+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 shutdown_connections 2024-09-05T12:17:12.808 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:12.054+0000 7fb6c9a00640 1 -- 172.21.15.71:0/4138529845 wait complete. 2024-09-05T12:17:12.809 INFO:teuthology.orchestra.run.smithi071.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:17:12.809 INFO:teuthology.orchestra.run.smithi071.stdout:Adding key to root@localhost authorized_keys... 2024-09-05T12:17:12.809 INFO:teuthology.orchestra.run.smithi071.stdout:Adding host smithi071... 2024-09-05T12:17:14.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:14 smithi071 ceph-mon[29845]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:15.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:15 smithi071 ceph-mon[29845]: Deploying cephadm binary to smithi071 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Added host 'smithi071' with addr '172.21.15.71' 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.352+0000 7fb006800640 1 Processor -- start 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.352+0000 7fb006800640 1 -- start start 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.352+0000 7fb006800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb000109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.352+0000 7fb006800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb00010a1e0 con 0x7fb000109810 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.353+0000 7fafff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb000109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.353+0000 7fafff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb000109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48162/0 (socket says 172.21.15.71:48162) 2024-09-05T12:17:18.339 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.353+0000 7fafff400640 1 -- 172.21.15.71:0/3064277667 learned_addr learned my addr 172.21.15.71:0/3064277667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.353+0000 7fafff400640 1 -- 172.21.15.71:0/3064277667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00010a950 con 0x7fb000109810 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.354+0000 7fafff400640 1 --2- 172.21.15.71:0/3064277667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb000109c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faff0009820 tx=0x7faff002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac637cdebdaab1d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.354+0000 7faffe000640 1 -- 172.21.15.71:0/3064277667 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff0037070 con 0x7fb000109810 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.354+0000 7faffe000640 1 -- 172.21.15.71:0/3064277667 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faff0032900 con 0x7fb000109810 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.354+0000 7faffe000640 1 -- 172.21.15.71:0/3064277667 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff0030850 con 0x7fb000109810 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 -- 172.21.15.71:0/3064277667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 msgr2=0x7fb000109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 --2- 172.21.15.71:0/3064277667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb000109c10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faff0009820 tx=0x7faff002a270 comp rx=0 tx=0).stop 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 -- 172.21.15.71:0/3064277667 shutdown_connections 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 --2- 172.21.15.71:0/3064277667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb000109c10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 -- 172.21.15.71:0/3064277667 >> 172.21.15.71:0/3064277667 conn(0x7fb00007ccf0 msgr2=0x7fb00007d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 -- 172.21.15.71:0/3064277667 shutdown_connections 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.355+0000 7fb006800640 1 -- 172.21.15.71:0/3064277667 wait complete. 2024-09-05T12:17:18.340 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.356+0000 7fb006800640 1 Processor -- start 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.356+0000 7fb006800640 1 -- start start 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.356+0000 7fb006800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb00019d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.356+0000 7fb006800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb00010dd00 con 0x7fb000109810 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7fafff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb00019d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7fafff400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb00019d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48168/0 (socket says 172.21.15.71:48168) 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7fafff400640 1 -- 172.21.15.71:0/1244128514 learned_addr learned my addr 172.21.15.71:0/1244128514 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7fafff400640 1 -- 172.21.15.71:0/1244128514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00019dd10 con 0x7fb000109810 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7fafff400640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb00019d7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faff0030460 tx=0x7faff0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff0042070 con 0x7fb000109810 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faff0032be0 con 0x7fb000109810 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faff0037040 con 0x7fb000109810 2024-09-05T12:17:18.341 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.357+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00019dfa0 con 0x7fb000109810 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.358+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0001a0c90 con 0x7fb000109810 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.359+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faff0045070 con 0x7fb000109810 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.359+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb000109c10 con 0x7fb000109810 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.359+0000 7faff7e00640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fafe4038fc0 0x7fafe403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.359+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faff006bed0 con 0x7fb000109810 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.359+0000 7faffea00640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fafe4038fc0 0x7fafe403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.360+0000 7faffea00640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fafe4038fc0 0x7fafe403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafec0098c0 tx=0x7fafec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.364+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faff0045360 con 0x7fb000109810 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:13.549+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "addr": "172.21.15.71", "target": ["mon-mgr", ""]}) -- 0x7fb000107c20 con 0x7fafe4038fc0 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.481+0000 7faff7e00640 1 -- 172.21.15.71:0/1244128514 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fb000107c20 con 0x7fafe4038fc0 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.484+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fafe4038fc0 msgr2=0x7fafe403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.484+0000 7fb006800640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fafe4038fc0 0x7fafe403b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fafec0098c0 tx=0x7fafec006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:18.342 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 msgr2=0x7fb00019d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb00019d7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faff0030460 tx=0x7faff0030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 shutdown_connections 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fafe4038fc0 0x7fafe403b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 --2- 172.21.15.71:0/1244128514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb000109810 0x7fb00019d7d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 >> 172.21.15.71:0/1244128514 conn(0x7fb00007ccf0 msgr2=0x7fb000107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 shutdown_connections 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:17.485+0000 7fb006800640 1 -- 172.21.15.71:0/1244128514 wait complete. 2024-09-05T12:17:18.343 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mon service... 2024-09-05T12:17:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:18 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:18 smithi071 ceph-mon[29845]: Added host smithi071 2024-09-05T12:17:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:18 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:20.109 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-05T12:17:20.109 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.080+0000 7fbd61400640 1 Processor -- start 2024-09-05T12:17:20.109 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.081+0000 7fbd61400640 1 -- start start 2024-09-05T12:17:20.109 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.081+0000 7fbd61400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c07b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:20.109 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.081+0000 7fbd61400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c07b7a0 con 0x7fbd5c07cd70 2024-09-05T12:17:20.109 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.081+0000 7fbd5be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c07b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.081+0000 7fbd5be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c07b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48178/0 (socket says 172.21.15.71:48178) 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.081+0000 7fbd5be00640 1 -- 172.21.15.71:0/2758608373 learned_addr learned my addr 172.21.15.71:0/2758608373 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.082+0000 7fbd5be00640 1 -- 172.21.15.71:0/2758608373 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c07b920 con 0x7fbd5c07cd70 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.083+0000 7fbd5be00640 1 --2- 172.21.15.71:0/2758608373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c07b1d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009820 tx=0x7fbd4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee99968590a3670 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.083+0000 7fbd5aa00640 1 -- 172.21.15.71:0/2758608373 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd4c037070 con 0x7fbd5c07cd70 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.083+0000 7fbd5aa00640 1 -- 172.21.15.71:0/2758608373 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd4c032900 con 0x7fbd5c07cd70 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.083+0000 7fbd5aa00640 1 -- 172.21.15.71:0/2758608373 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd4c030850 con 0x7fbd5c07cd70 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.083+0000 7fbd61400640 1 -- 172.21.15.71:0/2758608373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 msgr2=0x7fbd5c07b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.083+0000 7fbd61400640 1 --2- 172.21.15.71:0/2758608373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c07b1d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009820 tx=0x7fbd4c02a270 comp rx=0 tx=0).stop 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.084+0000 7fbd61400640 1 -- 172.21.15.71:0/2758608373 shutdown_connections 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.084+0000 7fbd61400640 1 --2- 172.21.15.71:0/2758608373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c07b1d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.084+0000 7fbd61400640 1 -- 172.21.15.71:0/2758608373 >> 172.21.15.71:0/2758608373 conn(0x7fbd5c102b90 msgr2=0x7fbd5c104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:20.110 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.084+0000 7fbd61400640 1 -- 172.21.15.71:0/2758608373 shutdown_connections 2024-09-05T12:17:20.111 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.084+0000 7fbd61400640 1 -- 172.21.15.71:0/2758608373 wait complete. 2024-09-05T12:17:20.111 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd61400640 1 Processor -- start 2024-09-05T12:17:20.111 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd61400640 1 -- start start 2024-09-05T12:17:20.111 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd61400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c19d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:20.111 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd61400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c10d9d0 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd5be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c19d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd5be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c19d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48186/0 (socket says 172.21.15.71:48186) 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.085+0000 7fbd5be00640 1 -- 172.21.15.71:0/4106218282 learned_addr learned my addr 172.21.15.71:0/4106218282 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd5be00640 1 -- 172.21.15.71:0/4106218282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c19daf0 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd5be00640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c19d5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0097f0 tx=0x7fbd4c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd4c037040 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd4c040070 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c19dd80 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd4c03a390 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.086+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c1a0a70 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.088+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbd4c032900 con 0x7fbd5c07cd70 2024-09-05T12:17:20.112 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.088+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c07b250 con 0x7fbd5c07cd70 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.088+0000 7fbd58c00640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fbd40038fc0 0x7fbd4003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.088+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd4c06c500 con 0x7fbd5c07cd70 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.088+0000 7fbd5b400640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fbd40038fc0 0x7fbd4003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.089+0000 7fbd5b400640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fbd40038fc0 0x7fbd4003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd480098c0 tx=0x7fbd48006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.096+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd4c06be50 con 0x7fbd5c07cd70 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.308+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fbd5c103e70 con 0x7fbd40038fc0 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.313+0000 7fbd58c00640 1 -- 172.21.15.71:0/4106218282 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fbd5c103e70 con 0x7fbd40038fc0 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fbd40038fc0 msgr2=0x7fbd4003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fbd40038fc0 0x7fbd4003b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbd480098c0 tx=0x7fbd48006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 msgr2=0x7fbd5c19d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c19d5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0097f0 tx=0x7fbd4c030d80 comp rx=0 tx=0).stop 2024-09-05T12:17:20.113 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 shutdown_connections 2024-09-05T12:17:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7fbd40038fc0 0x7fbd4003b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 --2- 172.21.15.71:0/4106218282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd5c07cd70 0x7fbd5c19d5b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 >> 172.21.15.71:0/4106218282 conn(0x7fbd5c102b90 msgr2=0x7fbd5c103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 shutdown_connections 2024-09-05T12:17:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:19.315+0000 7fbd61400640 1 -- 172.21.15.71:0/4106218282 wait complete. 2024-09-05T12:17:20.114 INFO:teuthology.orchestra.run.smithi071.stdout:Deploying unmanaged mgr service... 2024-09-05T12:17:20.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:20 smithi071 ceph-mon[29845]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:20.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:20 smithi071 ceph-mon[29845]: Saving service mon spec with placement count:5 2024-09-05T12:17:20.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:20 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:22 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:22 smithi071 ceph-mon[29845]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:22 smithi071 ceph-mon[29845]: Saving service mgr spec with placement count:2 2024-09-05T12:17:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:22 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:22 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.516+0000 7f34ace00640 1 Processor -- start 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.516+0000 7f34ace00640 1 -- start start 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.516+0000 7f34ace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a81077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.516+0000 7f34ace00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34a8107d70 con 0x7f34a81073a0 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.517+0000 7f34a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a81077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.517+0000 7f34a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a81077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48196/0 (socket says 172.21.15.71:48196) 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.517+0000 7f34a7400640 1 -- 172.21.15.71:0/35891376 learned_addr learned my addr 172.21.15.71:0/35891376 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.517+0000 7f34a7400640 1 -- 172.21.15.71:0/35891376 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a81085a0 con 0x7f34a81073a0 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.519+0000 7f34a7400640 1 --2- 172.21.15.71:0/35891376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a81077a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3498009820 tx=0x7f349802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4db648e0be281c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.519+0000 7f34a6000640 1 -- 172.21.15.71:0/35891376 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3498037070 con 0x7f34a81073a0 2024-09-05T12:17:22.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.519+0000 7f34a6000640 1 -- 172.21.15.71:0/35891376 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498032920 con 0x7f34a81073a0 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.519+0000 7f34a6000640 1 -- 172.21.15.71:0/35891376 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3498030810 con 0x7f34a81073a0 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 -- 172.21.15.71:0/35891376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 msgr2=0x7f34a81077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 --2- 172.21.15.71:0/35891376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a81077a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3498009820 tx=0x7f349802a270 comp rx=0 tx=0).stop 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 -- 172.21.15.71:0/35891376 shutdown_connections 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 --2- 172.21.15.71:0/35891376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a81077a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 -- 172.21.15.71:0/35891376 >> 172.21.15.71:0/35891376 conn(0x7f34a8102b90 msgr2=0x7f34a8104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 -- 172.21.15.71:0/35891376 shutdown_connections 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.520+0000 7f34ace00640 1 -- 172.21.15.71:0/35891376 wait complete. 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.521+0000 7f34ace00640 1 Processor -- start 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.521+0000 7f34ace00640 1 -- start start 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.522+0000 7f34ace00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.522+0000 7f34ace00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34a810b950 con 0x7f34a81073a0 2024-09-05T12:17:22.463 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.522+0000 7f34a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.522+0000 7f34a7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a819d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48200/0 (socket says 172.21.15.71:48200) 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.522+0000 7f34a7400640 1 -- 172.21.15.71:0/3662907456 learned_addr learned my addr 172.21.15.71:0/3662907456 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.522+0000 7f34a7400640 1 -- 172.21.15.71:0/3662907456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a819db10 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.523+0000 7f34a7400640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a819d5d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f34980097f0 tx=0x7f34980327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.523+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3498037040 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.523+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498030cc0 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.523+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a819dda0 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.523+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f349803a640 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.523+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a81a0a90 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.524+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f349803a8f0 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.524+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a810bad0 con 0x7f34a81073a0 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.524+0000 7f349fe00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f348c038fc0 0x7f348c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:22.464 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.525+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f349806c5f0 con 0x7f34a81073a0 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.525+0000 7f34a6a00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f348c038fc0 0x7f348c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.525+0000 7f34a6a00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f348c038fc0 0x7f348c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34940098c0 tx=0x7f3494006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.530+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f349803ba40 con 0x7f34a81073a0 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.728+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f34a8103d90 con 0x7f348c038fc0 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.736+0000 7f349fe00640 1 -- 172.21.15.71:0/3662907456 <== mgr.14118 v2:172.21.15.71:6800/3648851306 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f34a8103d90 con 0x7f348c038fc0 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f348c038fc0 msgr2=0x7f348c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f348c038fc0 0x7f348c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34940098c0 tx=0x7f3494006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 msgr2=0x7f34a819d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:22.465 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a819d5d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f34980097f0 tx=0x7f34980327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 shutdown_connections 2024-09-05T12:17:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f348c038fc0 0x7f348c03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 --2- 172.21.15.71:0/3662907456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34a81073a0 0x7f34a819d5d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 >> 172.21.15.71:0/3662907456 conn(0x7f34a8102b90 msgr2=0x7f34a8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 shutdown_connections 2024-09-05T12:17:22.466 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:21.739+0000 7f34ace00640 1 -- 172.21.15.71:0/3662907456 wait complete. 2024-09-05T12:17:23.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.962+0000 7f87a3200640 1 Processor -- start 2024-09-05T12:17:23.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.962+0000 7f87a3200640 1 -- start start 2024-09-05T12:17:23.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.963+0000 7f87a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c081910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:23.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.963+0000 7f87a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f879c081ee0 con 0x7f879c081510 2024-09-05T12:17:23.911 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.963+0000 7f879be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c081910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.963+0000 7f879be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c081910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55514/0 (socket says 172.21.15.71:55514) 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.963+0000 7f879be00640 1 -- 172.21.15.71:0/204079506 learned_addr learned my addr 172.21.15.71:0/204079506 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.963+0000 7f879be00640 1 -- 172.21.15.71:0/204079506 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c10ed70 con 0x7f879c081510 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.964+0000 7f879be00640 1 --2- 172.21.15.71:0/204079506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c081910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f878c009820 tx=0x7f878c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b5452ecace786f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f879aa00640 1 -- 172.21.15.71:0/204079506 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c037070 con 0x7f879c081510 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f879aa00640 1 -- 172.21.15.71:0/204079506 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c032920 con 0x7f879c081510 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f879aa00640 1 -- 172.21.15.71:0/204079506 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c030810 con 0x7f879c081510 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f87a3200640 1 -- 172.21.15.71:0/204079506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 msgr2=0x7f879c081910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f87a3200640 1 --2- 172.21.15.71:0/204079506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c081910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f878c009820 tx=0x7f878c02a270 comp rx=0 tx=0).stop 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f87a3200640 1 -- 172.21.15.71:0/204079506 shutdown_connections 2024-09-05T12:17:23.912 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f87a3200640 1 --2- 172.21.15.71:0/204079506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c081910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.965+0000 7f87a3200640 1 -- 172.21.15.71:0/204079506 >> 172.21.15.71:0/204079506 conn(0x7f879c07ccf0 msgr2=0x7f879c07f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.966+0000 7f87a3200640 1 -- 172.21.15.71:0/204079506 shutdown_connections 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.966+0000 7f87a3200640 1 -- 172.21.15.71:0/204079506 wait complete. 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f87a3200640 1 Processor -- start 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f87a3200640 1 -- start start 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f87a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c1a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f87a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f879c112120 con 0x7f879c081510 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f879be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c1a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f879be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c1a1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55518/0 (socket says 172.21.15.71:55518) 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.967+0000 7f879be00640 1 -- 172.21.15.71:0/2424354194 learned_addr learned my addr 172.21.15.71:0/2424354194 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f879be00640 1 -- 172.21.15.71:0/2424354194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879c1a2140 con 0x7f879c081510 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f879be00640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c1a1c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097f0 tx=0x7f878c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:23.913 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c037040 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f878c030c40 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f878c03a5b0 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879c1a23d0 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.968+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f879c1a50c0 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.970+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f878c03b3f0 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.970+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f879c081910 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.970+0000 7f8798c00640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8780038fc0 0x7f878003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.970+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f878c06c430 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.970+0000 7f879b400640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8780038fc0 0x7f878003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.971+0000 7f879b400640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8780038fc0 0x7f878003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f87880098c0 tx=0x7f8788006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:22.975+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f878c044dc0 con 0x7f879c081510 2024-09-05T12:17:23.914 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.166+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f879c1a56b0 con 0x7f879c081510 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.169+0000 7f8798c00640 1 -- 172.21.15.71:0/2424354194 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f878c0313a0 con 0x7f879c081510 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8780038fc0 msgr2=0x7f878003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8780038fc0 0x7f878003b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f87880098c0 tx=0x7f8788006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 msgr2=0x7f879c1a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c1a1c00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097f0 tx=0x7f878c0327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 shutdown_connections 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f8780038fc0 0x7f878003b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 --2- 172.21.15.71:0/2424354194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879c081510 0x7f879c1a1c00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.172+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 >> 172.21.15.71:0/2424354194 conn(0x7f879c07ccf0 msgr2=0x7f879c07d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.173+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 shutdown_connections 2024-09-05T12:17:23.915 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:23.173+0000 7f87a3200640 1 -- 172.21.15.71:0/2424354194 wait complete. 2024-09-05T12:17:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:24 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2424354194' entity='client.admin' 2024-09-05T12:17:26.031 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:25 smithi071 ceph-mon[29845]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:17:26.032 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:25 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:26.032 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:25 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/696572652' entity='client.admin' 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.097+0000 7f613ce00640 1 Processor -- start 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.097+0000 7f613ce00640 1 -- start start 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.097+0000 7f613ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f61381077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.098+0000 7f613ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6138107d70 con 0x7f61381073a0 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.098+0000 7f6137400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f61381077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.098+0000 7f6137400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f61381077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55524/0 (socket says 172.21.15.71:55524) 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.098+0000 7f6137400640 1 -- 172.21.15.71:0/817471583 learned_addr learned my addr 172.21.15.71:0/817471583 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.098+0000 7f6137400640 1 -- 172.21.15.71:0/817471583 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61381085a0 con 0x7f61381073a0 2024-09-05T12:17:26.040 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.099+0000 7f6137400640 1 --2- 172.21.15.71:0/817471583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f61381077a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6128009820 tx=0x7f612802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb34d319054e93a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f6136000640 1 -- 172.21.15.71:0/817471583 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6128037070 con 0x7f61381073a0 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f6136000640 1 -- 172.21.15.71:0/817471583 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6128032920 con 0x7f61381073a0 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f6136000640 1 -- 172.21.15.71:0/817471583 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6128030810 con 0x7f61381073a0 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 -- 172.21.15.71:0/817471583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 msgr2=0x7f61381077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 --2- 172.21.15.71:0/817471583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f61381077a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6128009820 tx=0x7f612802a270 comp rx=0 tx=0).stop 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 -- 172.21.15.71:0/817471583 shutdown_connections 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 --2- 172.21.15.71:0/817471583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f61381077a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 -- 172.21.15.71:0/817471583 >> 172.21.15.71:0/817471583 conn(0x7f6138102b90 msgr2=0x7f6138104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 -- 172.21.15.71:0/817471583 shutdown_connections 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.100+0000 7f613ce00640 1 -- 172.21.15.71:0/817471583 wait complete. 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.101+0000 7f613ce00640 1 Processor -- start 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.101+0000 7f613ce00640 1 -- start start 2024-09-05T12:17:26.041 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.102+0000 7f613ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f613807ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.102+0000 7f613ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f613810b950 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.102+0000 7f6137400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f613807ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.102+0000 7f6137400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f613807ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55540/0 (socket says 172.21.15.71:55540) 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.102+0000 7f6137400640 1 -- 172.21.15.71:0/696572652 learned_addr learned my addr 172.21.15.71:0/696572652 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.102+0000 7f6137400640 1 -- 172.21.15.71:0/696572652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613807cf40 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.103+0000 7f6137400640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f613807ca00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61280097f0 tx=0x7f61280327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.103+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6128037040 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.103+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6128030cc0 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.103+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f612803a640 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.103+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613807b130 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.103+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613807b670 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.104+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f612803a8f0 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.104+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f613810bad0 con 0x7f61381073a0 2024-09-05T12:17:26.042 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.105+0000 7f612fe00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f611c038fc0 0x7f611c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.105+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f612806c750 con 0x7f61381073a0 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.105+0000 7f6136a00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f611c038fc0 0x7f611c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.106+0000 7f6136a00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f611c038fc0 0x7f611c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f61240098c0 tx=0x7f6124006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.110+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f61280328c0 con 0x7f61381073a0 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.307+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f6138107820 con 0x7f61381073a0 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.310+0000 7f612fe00640 1 -- 172.21.15.71:0/696572652 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f612806c0a0 con 0x7f61381073a0 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f611c038fc0 msgr2=0x7f611c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f611c038fc0 0x7f611c03b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f61240098c0 tx=0x7f6124006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 msgr2=0x7f613807ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f613807ca00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f61280097f0 tx=0x7f61280327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 shutdown_connections 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f611c038fc0 0x7f611c03b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:26.043 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 --2- 172.21.15.71:0/696572652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61381073a0 0x7f613807ca00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:26.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 >> 172.21.15.71:0/696572652 conn(0x7f6138102b90 msgr2=0x7f6138103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:26.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 shutdown_connections 2024-09-05T12:17:26.044 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:25.312+0000 7f613ce00640 1 -- 172.21.15.71:0/696572652 wait complete. 2024-09-05T12:17:26.044 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling the dashboard module... 2024-09-05T12:17:27.237 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-mon[29845]: from='mgr.14118 172.21.15.71:0/3710987649' entity='mgr.x' 2024-09-05T12:17:27.237 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-mon[29845]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:17:27.237 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/571008490' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-05T12:17:27.572 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ignoring --setuser ceph since I am not root 2024-09-05T12:17:27.573 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ignoring --setgroup ceph since I am not root 2024-09-05T12:17:27.573 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:27.367+0000 7f1742ded040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:17:27.573 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:27.435+0000 7f1742ded040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:17:27.874 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:27.619+0000 7f1742ded040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:17:27.874 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:27.698+0000 7f1742ded040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:17:27.875 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:27.872+0000 7f1742ded040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:17:28.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:28 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/571008490' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-05T12:17:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:28 smithi071 ceph-mon[29845]: mgrmap e9: x(active, since 22s) 2024-09-05T12:17:28.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:28.041+0000 7f1742ded040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:17:28.251 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.597+0000 7f1198200640 1 Processor -- start 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.597+0000 7f1198200640 1 -- start start 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.598+0000 7f1198200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f1190107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.598+0000 7f1198200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1190107f00 con 0x7f1190107530 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.598+0000 7f1195800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f1190107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.598+0000 7f1195800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f1190107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55546/0 (socket says 172.21.15.71:55546) 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.598+0000 7f1195800640 1 -- 172.21.15.71:0/848519932 learned_addr learned my addr 172.21.15.71:0/848519932 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.599+0000 7f1195800640 1 -- 172.21.15.71:0/848519932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1190108730 con 0x7f1190107530 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.599+0000 7f1195800640 1 --2- 172.21.15.71:0/848519932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f1190107930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1184009820 tx=0x7f118402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90001530aae050c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.600+0000 7f118fe00640 1 -- 172.21.15.71:0/848519932 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1184037070 con 0x7f1190107530 2024-09-05T12:17:28.252 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.600+0000 7f118fe00640 1 -- 172.21.15.71:0/848519932 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1184032920 con 0x7f1190107530 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.600+0000 7f118fe00640 1 -- 172.21.15.71:0/848519932 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1184030810 con 0x7f1190107530 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 -- 172.21.15.71:0/848519932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 msgr2=0x7f1190107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 --2- 172.21.15.71:0/848519932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f1190107930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1184009820 tx=0x7f118402a270 comp rx=0 tx=0).stop 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 -- 172.21.15.71:0/848519932 shutdown_connections 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 --2- 172.21.15.71:0/848519932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f1190107930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 -- 172.21.15.71:0/848519932 >> 172.21.15.71:0/848519932 conn(0x7f1190102cc0 msgr2=0x7f1190105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 -- 172.21.15.71:0/848519932 shutdown_connections 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.601+0000 7f1198200640 1 -- 172.21.15.71:0/848519932 wait complete. 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1198200640 1 Processor -- start 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1198200640 1 -- start start 2024-09-05T12:17:28.253 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1198200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f119007db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1198200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f119010bae0 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1195800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f119007db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1195800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f119007db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55550/0 (socket says 172.21.15.71:55550) 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.602+0000 7f1195800640 1 -- 172.21.15.71:0/571008490 learned_addr learned my addr 172.21.15.71:0/571008490 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.603+0000 7f1195800640 1 -- 172.21.15.71:0/571008490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119007e050 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.603+0000 7f1195800640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f119007db10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f11840097f0 tx=0x7f11840327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.603+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1184037040 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.603+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1184030cc0 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.603+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f118403a640 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.603+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119007c240 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.604+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119007c780 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.605+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119010bc60 con 0x7f1190107530 2024-09-05T12:17:28.254 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.605+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f118403b3f0 con 0x7f1190107530 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.606+0000 7f118e000640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f1178038fc0 0x7f117803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.606+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f118406c4c0 con 0x7f1190107530 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.606+0000 7f1194e00640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f1178038fc0 0x7f117803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.607+0000 7f1194e00640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f1178038fc0 0x7f117803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098c0 tx=0x7f117c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.611+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1184049020 con 0x7f1190107530 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:26.854+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f119007ce70 con 0x7f1190107530 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.120+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f118406be10 con 0x7f1190107530 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.122+0000 7f118e000640 1 -- 172.21.15.71:0/571008490 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f118403a7e0 con 0x7f1190107530 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f1178038fc0 msgr2=0x7f117803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f1178038fc0 0x7f117803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f117c0098c0 tx=0x7f117c006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 msgr2=0x7f119007db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f119007db10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f11840097f0 tx=0x7f11840327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:28.255 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 shutdown_connections 2024-09-05T12:17:28.256 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f1178038fc0 0x7f117803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:28.256 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 --2- 172.21.15.71:0/571008490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1190107530 0x7f119007db10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:28.256 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 >> 172.21.15.71:0/571008490 conn(0x7f1190102cc0 msgr2=0x7f11901038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:28.256 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 shutdown_connections 2024-09-05T12:17:28.256 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:27.124+0000 7f1198200640 1 -- 172.21.15.71:0/571008490 wait complete. 2024-09-05T12:17:28.660 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:28.415+0000 7f1742ded040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:17:28.661 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:28.486+0000 7f1742ded040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:17:28.954 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:28.697+0000 7f1742ded040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:17:29.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:29 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/78724399' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-05T12:17:30.160 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:29.826+0000 7f1742ded040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:17:30.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:29.907+0000 7f1742ded040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:17:30.215 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:17:30.222 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.161+0000 7f44b0a00640 1 Processor -- start 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.161+0000 7f44b0a00640 1 -- start start 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.162+0000 7f44b0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac1077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.162+0000 7f44b0a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac107d70 con 0x7f44ac1073a0 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.162+0000 7f44ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac1077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.162+0000 7f44ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac1077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55576/0 (socket says 172.21.15.71:55576) 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.162+0000 7f44ab400640 1 -- 172.21.15.71:0/1239680549 learned_addr learned my addr 172.21.15.71:0/1239680549 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.162+0000 7f44ab400640 1 -- 172.21.15.71:0/1239680549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac1085a0 con 0x7f44ac1073a0 2024-09-05T12:17:30.223 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.163+0000 7f44ab400640 1 --2- 172.21.15.71:0/1239680549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac1077a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f449c009820 tx=0x7f449c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733fc7d1f3b1f72a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.164+0000 7f44aa000640 1 -- 172.21.15.71:0/1239680549 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f449c037070 con 0x7f44ac1073a0 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.164+0000 7f44aa000640 1 -- 172.21.15.71:0/1239680549 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c032920 con 0x7f44ac1073a0 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.164+0000 7f44aa000640 1 -- 172.21.15.71:0/1239680549 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f449c030810 con 0x7f44ac1073a0 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.164+0000 7f44b0a00640 1 -- 172.21.15.71:0/1239680549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 msgr2=0x7f44ac1077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.164+0000 7f44b0a00640 1 --2- 172.21.15.71:0/1239680549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac1077a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f449c009820 tx=0x7f449c02a270 comp rx=0 tx=0).stop 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.165+0000 7f44b0a00640 1 -- 172.21.15.71:0/1239680549 shutdown_connections 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.165+0000 7f44b0a00640 1 --2- 172.21.15.71:0/1239680549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac1077a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.165+0000 7f44b0a00640 1 -- 172.21.15.71:0/1239680549 >> 172.21.15.71:0/1239680549 conn(0x7f44ac102b90 msgr2=0x7f44ac104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.165+0000 7f44b0a00640 1 -- 172.21.15.71:0/1239680549 shutdown_connections 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.165+0000 7f44b0a00640 1 -- 172.21.15.71:0/1239680549 wait complete. 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44b0a00640 1 Processor -- start 2024-09-05T12:17:30.224 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44b0a00640 1 -- start start 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44b0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44b0a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac10b950 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55592/0 (socket says 172.21.15.71:55592) 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.166+0000 7f44ab400640 1 -- 172.21.15.71:0/78724399 learned_addr learned my addr 172.21.15.71:0/78724399 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.167+0000 7f44ab400640 1 -- 172.21.15.71:0/78724399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac19b8e0 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.167+0000 7f44ab400640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac19b3a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097f0 tx=0x7f449c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.167+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f449c037040 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.167+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f449c030cc0 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.167+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44ac19bb70 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.167+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44ac19e860 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.168+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f449c03a640 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.168+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f449c03a8f0 con 0x7f44ac1073a0 2024-09-05T12:17:30.225 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.169+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f448c0052c0 con 0x7f44ac1073a0 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.169+0000 7f44a3e00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 0x7f449003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.169+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f449c03b800 con 0x7f44ac1073a0 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.169+0000 7f44aaa00640 1 -- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 msgr2=0x7f449003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3648851306 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.169+0000 7f44aaa00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 0x7f449003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.174+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f449c031e80 con 0x7f44ac1073a0 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.370+0000 7f44aaa00640 1 -- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 msgr2=0x7f449003b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3648851306 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.370+0000 7f44aaa00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 0x7f449003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.401+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f448c005880 con 0x7f44ac1073a0 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.402+0000 7f44a3e00640 1 -- 172.21.15.71:0/78724399 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f449c0328c0 con 0x7f44ac1073a0 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 msgr2=0x7f449003b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 0x7f449003b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 msgr2=0x7f44ac19b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:30.226 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac19b3a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097f0 tx=0x7f449c0327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 shutdown_connections 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f4490039020 0x7f449003b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 --2- 172.21.15.71:0/78724399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44ac1073a0 0x7f44ac19b3a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 >> 172.21.15.71:0/78724399 conn(0x7f44ac102b90 msgr2=0x7f44ac103740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 shutdown_connections 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:29.404+0000 7f44b0a00640 1 -- 172.21.15.71:0/78724399 wait complete. 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for the mgr to restart... 2024-09-05T12:17:30.227 INFO:teuthology.orchestra.run.smithi071.stdout:Waiting for mgr epoch 9... 2024-09-05T12:17:31.149 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:30.852+0000 7f1742ded040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:17:31.149 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:30.987+0000 7f1742ded040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:17:31.149 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.147+0000 7f1742ded040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:17:31.404 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.219+0000 7f1742ded040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:17:31.660 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.416+0000 7f1742ded040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:17:31.661 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.500+0000 7f1742ded040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:17:31.661 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.571+0000 7f1742ded040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:17:32.068 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.749+0000 7f1742ded040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:17:32.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.825+0000 7f1742ded040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:17:32.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.903+0000 7f1742ded040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:17:32.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:31.998+0000 7f1742ded040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:17:32.069 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:32.067+0000 7f1742ded040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:17:32.410 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:32.137+0000 7f1742ded040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:33.043+0000 7f1742ded040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: Active manager daemon x restarted 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: Activating manager daemon x 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: osdmap e3: 0 total, 0 up, 0 in 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: mgrmap e10: x(active, starting, since 0.0041671s) 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T12:17:33.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T12:17:33.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T12:17:33.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: Manager daemon x is now available 2024-09-05T12:17:33.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T12:17:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T12:17:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:34 smithi071 ceph-mon[29845]: mgrmap e11: x(active, since 1.00945s) 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout { 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout } 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.825+0000 7f56bbe00640 1 Processor -- start 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.825+0000 7f56bbe00640 1 -- start start 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.825+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc07b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.825+0000 7f56bbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc07b740 con 0x7f56bc07cd10 2024-09-05T12:17:35.045 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.825+0000 7f56baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.826+0000 7f56baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc07b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55608/0 (socket says 172.21.15.71:55608) 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.826+0000 7f56baa00640 1 -- 172.21.15.71:0/1058124805 learned_addr learned my addr 172.21.15.71:0/1058124805 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.826+0000 7f56baa00640 1 -- 172.21.15.71:0/1058124805 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc07b8c0 con 0x7f56bc07cd10 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.827+0000 7f56baa00640 1 --2- 172.21.15.71:0/1058124805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc07b170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009820 tx=0x7f56ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c211c40f333542 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.827+0000 7f56b9600640 1 -- 172.21.15.71:0/1058124805 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac037070 con 0x7f56bc07cd10 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.827+0000 7f56b9600640 1 -- 172.21.15.71:0/1058124805 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac032920 con 0x7f56bc07cd10 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.827+0000 7f56b9600640 1 -- 172.21.15.71:0/1058124805 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac030810 con 0x7f56bc07cd10 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 -- 172.21.15.71:0/1058124805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 msgr2=0x7f56bc07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 --2- 172.21.15.71:0/1058124805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc07b170 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009820 tx=0x7f56ac02a270 comp rx=0 tx=0).stop 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 -- 172.21.15.71:0/1058124805 shutdown_connections 2024-09-05T12:17:35.046 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 --2- 172.21.15.71:0/1058124805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc07b170 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 -- 172.21.15.71:0/1058124805 >> 172.21.15.71:0/1058124805 conn(0x7f56bc102b90 msgr2=0x7f56bc104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 -- 172.21.15.71:0/1058124805 shutdown_connections 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.828+0000 7f56bbe00640 1 -- 172.21.15.71:0/1058124805 wait complete. 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.829+0000 7f56bbe00640 1 Processor -- start 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.829+0000 7f56bbe00640 1 -- start start 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56bbe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc10da30 con 0x7f56bc07cd10 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56baa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55620/0 (socket says 172.21.15.71:55620) 2024-09-05T12:17:35.047 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56baa00640 1 -- 172.21.15.71:0/3636486665 learned_addr learned my addr 172.21.15.71:0/3636486665 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56baa00640 1 -- 172.21.15.71:0/3636486665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc19dab0 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.830+0000 7f56baa00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc19d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56ac030380 tx=0x7f56ac032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.831+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac037040 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.831+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac040070 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.831+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc19dd40 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.831+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56ac03a7b0 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.831+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1a0a30 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.832+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f56ac03aa30 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.833+0000 7f56c0a00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.833+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 --> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56a003bb90 con 0x7f56a0038fc0 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.833+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f56ac06c730 con 0x7f56bc07cd10 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.833+0000 7f56ba000640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 msgr2=0x7f56a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3648851306 2024-09-05T12:17:35.048 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:30.833+0000 7f56ba000640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:31.034+0000 7f56ba000640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 msgr2=0x7f56a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3648851306 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:31.034+0000 7f56ba000640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:31.435+0000 7f56ba000640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 msgr2=0x7f56a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3648851306 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:31.435+0000 7f56ba000640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:32.236+0000 7f56ba000640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 msgr2=0x7f56a003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/3648851306 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:32.236+0000 7f56ba000640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:33.047+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f56ac06b120 con 0x7f56bc07cd10 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:33.048+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 msgr2=0x7f56a003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:33.048+0000 7f56c0a00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.052+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f56ac045460 con 0x7f56bc07cd10 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.052+0000 7f56c0a00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56a003ca80 0x7f56a003ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:35.049 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.052+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f56a003bb90 con 0x7f56a003ca80 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.053+0000 7f56ba000640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56a003ca80 0x7f56a003ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.054+0000 7f56ba000640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56a003ca80 0x7f56a003ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f56a8003980 tx=0x7f56a8006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.055+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f56a003bb90 con 0x7f56a003ca80 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.067+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f56bc07b1f0 con 0x7f56a003ca80 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.068+0000 7f56c0a00640 1 -- 172.21.15.71:0/3636486665 <== mgr.14150 v2:172.21.15.71:6800/2050574497 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f56bc07b1f0 con 0x7f56a003ca80 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.068+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56a003ca80 msgr2=0x7f56a003ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.068+0000 7f56bbe00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56a003ca80 0x7f56a003ee70 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f56a8003980 tx=0x7f56a8006e10 comp rx=0 tx=0).stop 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.068+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 msgr2=0x7f56bc19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.068+0000 7f56bbe00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc19d570 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56ac030380 tx=0x7f56ac032670 comp rx=0 tx=0).stop 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 shutdown_connections 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56a003ca80 0x7f56a003ee70 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:6800/3648851306,v1:172.21.15.71:6801/3648851306] conn(0x7f56a0038fc0 0x7f56a003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:35.050 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 --2- 172.21.15.71:0/3636486665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56bc07cd10 0x7f56bc19d570 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:35.051 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 >> 172.21.15.71:0/3636486665 conn(0x7f56bc102b90 msgr2=0x7f56bc1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:35.051 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 shutdown_connections 2024-09-05T12:17:35.051 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:34.069+0000 7f56bbe00640 1 -- 172.21.15.71:0/3636486665 wait complete. 2024-09-05T12:17:35.051 INFO:teuthology.orchestra.run.smithi071.stdout:mgr epoch 9 is available 2024-09-05T12:17:35.051 INFO:teuthology.orchestra.run.smithi071.stdout:Generating a dashboard self-signed certificate... 2024-09-05T12:17:35.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-mon[29845]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-05T12:17:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-mon[29845]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-05T12:17:35.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:35.159+0000 7f1698800640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:12:17:35] ENGINE Error in HTTPServer.serve 2024-09-05T12:17:35.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: Traceback (most recent call last): 2024-09-05T12:17:35.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._connections.run(self.expiration_interval) 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._run(expiration_interval) 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:17:35.412 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: s = self.context.wrap_socket( 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: return self.sslsocket_class._create( 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self.do_handshake() 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._sslobj.do_handshake() 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:17:35.413 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:17:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:34] ENGINE Bus STARTING 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:35] ENGINE Serving on http://172.21.15.71:8765 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: mgrmap e12: x(active, since 2s) 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:35] ENGINE Serving on https://172.21.15.71:7150 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:35] ENGINE Bus STARTED 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: [05/Sep/2024:12:17:35] ENGINE Error in HTTPServer.serve 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:17:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:17:36.162 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-05T12:17:36.163 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:17:36.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:36.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-05T12:17:36.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.972+0000 7f28fd200640 1 Processor -- start 2024-09-05T12:17:36.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.972+0000 7f28fd200640 1 -- start start 2024-09-05T12:17:36.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.973+0000 7f28fd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f8109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:36.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.973+0000 7f28fd200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28f810a1e0 con 0x7f28f8109810 2024-09-05T12:17:36.903 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.973+0000 7f28f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f8109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.973+0000 7f28f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f8109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49494/0 (socket says 172.21.15.71:49494) 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.973+0000 7f28f5e00640 1 -- 172.21.15.71:0/2732710764 learned_addr learned my addr 172.21.15.71:0/2732710764 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.974+0000 7f28f5e00640 1 -- 172.21.15.71:0/2732710764 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f810a950 con 0x7f28f8109810 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.975+0000 7f28f5e00640 1 --2- 172.21.15.71:0/2732710764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f8109c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009820 tx=0x7f28e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0899cbb5335d9f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.975+0000 7f28f4a00640 1 -- 172.21.15.71:0/2732710764 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28e8037070 con 0x7f28f8109810 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.975+0000 7f28f4a00640 1 -- 172.21.15.71:0/2732710764 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e8032920 con 0x7f28f8109810 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.975+0000 7f28f4a00640 1 -- 172.21.15.71:0/2732710764 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28e8030810 con 0x7f28f8109810 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 -- 172.21.15.71:0/2732710764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 msgr2=0x7f28f8109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 --2- 172.21.15.71:0/2732710764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f8109c10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009820 tx=0x7f28e802a270 comp rx=0 tx=0).stop 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 -- 172.21.15.71:0/2732710764 shutdown_connections 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 --2- 172.21.15.71:0/2732710764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f8109c10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:36.904 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 -- 172.21.15.71:0/2732710764 >> 172.21.15.71:0/2732710764 conn(0x7f28f807ccf0 msgr2=0x7f28f807d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 -- 172.21.15.71:0/2732710764 shutdown_connections 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.976+0000 7f28fd200640 1 -- 172.21.15.71:0/2732710764 wait complete. 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.977+0000 7f28fd200640 1 Processor -- start 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.977+0000 7f28fd200640 1 -- start start 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.977+0000 7f28fd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.977+0000 7f28fd200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28f810dd00 con 0x7f28f8109810 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.977+0000 7f28f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49502/0 (socket says 172.21.15.71:49502) 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28f5e00640 1 -- 172.21.15.71:0/3740921214 learned_addr learned my addr 172.21.15.71:0/3740921214 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28f5e00640 1 -- 172.21.15.71:0/3740921214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f819dd00 con 0x7f28f8109810 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28f5e00640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f819d7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28e80097f0 tx=0x7f28e80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28e8037040 con 0x7f28f8109810 2024-09-05T12:17:36.905 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28e8030c40 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28e803a5d0 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.978+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f819df90 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.979+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f81a0c80 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.980+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f28e803b3f0 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.980+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f8109c90 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.980+0000 7f28eea00640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f28dc039020 0x7f28dc03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.981+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f28e806c610 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.982+0000 7f28f5400640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f28dc039020 0x7f28dc03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.982+0000 7f28f5400640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f28dc039020 0x7f28dc03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28e40098c0 tx=0x7f28e4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:35.986+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28e803b6e0 con 0x7f28f8109810 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.179+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f28f8107c80 con 0x7f28dc039020 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.246+0000 7f28eea00640 1 -- 172.21.15.71:0/3740921214 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f28f8107c80 con 0x7f28dc039020 2024-09-05T12:17:36.906 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f28dc039020 msgr2=0x7f28dc03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f28dc039020 0x7f28dc03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f28e40098c0 tx=0x7f28e4006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 msgr2=0x7f28f819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f819d7c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f28e80097f0 tx=0x7f28e80327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 shutdown_connections 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f28dc039020 0x7f28dc03b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 --2- 172.21.15.71:0/3740921214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28f8109810 0x7f28f819d7c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.248+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 >> 172.21.15.71:0/3740921214 conn(0x7f28f807ccf0 msgr2=0x7f28f8107570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.249+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 shutdown_connections 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:36.249+0000 7f28fd200640 1 -- 172.21.15.71:0/3740921214 wait complete. 2024-09-05T12:17:36.907 INFO:teuthology.orchestra.run.smithi071.stdout:Creating initial admin user... 2024-09-05T12:17:37.480 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:37 smithi071 ceph-mon[29845]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:37.480 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:37 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:37.481 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:37 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:37.481 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:37 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:39.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:38 smithi071 ceph-mon[29845]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:39.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:39.197 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$P5yNFnZtUBHRfH5XHU/rMOWB4GhTH4IFhrQjzcZ5JtuOsOZw5y4Zy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725538657, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.498+0000 7f0553000640 1 Processor -- start 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.498+0000 7f0553000640 1 -- start start 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.499+0000 7f0553000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c109c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.499+0000 7f0553000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f054c10a1e0 con 0x7f054c109810 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.499+0000 7f054be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.499+0000 7f054be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49510/0 (socket says 172.21.15.71:49510) 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.499+0000 7f054be00640 1 -- 172.21.15.71:0/2053217512 learned_addr learned my addr 172.21.15.71:0/2053217512 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.499+0000 7f054be00640 1 -- 172.21.15.71:0/2053217512 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c10a950 con 0x7f054c109810 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.500+0000 7f054be00640 1 --2- 172.21.15.71:0/2053217512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c109c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f053c009820 tx=0x7f053c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c009ca30879476 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f054b400640 1 -- 172.21.15.71:0/2053217512 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f053c037070 con 0x7f054c109810 2024-09-05T12:17:39.198 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f054b400640 1 -- 172.21.15.71:0/2053217512 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c032920 con 0x7f054c109810 2024-09-05T12:17:39.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f054b400640 1 -- 172.21.15.71:0/2053217512 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f053c030810 con 0x7f054c109810 2024-09-05T12:17:39.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f0553000640 1 -- 172.21.15.71:0/2053217512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 msgr2=0x7f054c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:39.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f0553000640 1 --2- 172.21.15.71:0/2053217512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c109c10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f053c009820 tx=0x7f053c02a270 comp rx=0 tx=0).stop 2024-09-05T12:17:39.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f0553000640 1 -- 172.21.15.71:0/2053217512 shutdown_connections 2024-09-05T12:17:39.199 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f0553000640 1 --2- 172.21.15.71:0/2053217512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c109c10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:39.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f0553000640 1 -- 172.21.15.71:0/2053217512 >> 172.21.15.71:0/2053217512 conn(0x7f054c07ccf0 msgr2=0x7f054c07d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:39.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.501+0000 7f0553000640 1 -- 172.21.15.71:0/2053217512 shutdown_connections 2024-09-05T12:17:39.200 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.502+0000 7f0553000640 1 -- 172.21.15.71:0/2053217512 wait complete. 2024-09-05T12:17:39.201 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.502+0000 7f0553000640 1 Processor -- start 2024-09-05T12:17:39.201 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.502+0000 7f0553000640 1 -- start start 2024-09-05T12:17:39.201 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.503+0000 7f0553000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c19d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:39.201 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.503+0000 7f0553000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f054c10dd00 con 0x7f054c109810 2024-09-05T12:17:39.201 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.503+0000 7f054be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c19d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:39.201 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.503+0000 7f054be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c19d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49514/0 (socket says 172.21.15.71:49514) 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.503+0000 7f054be00640 1 -- 172.21.15.71:0/2437804720 learned_addr learned my addr 172.21.15.71:0/2437804720 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.503+0000 7f054be00640 1 -- 172.21.15.71:0/2437804720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054c19dd10 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.504+0000 7f054be00640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c19d7d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f053c0097f0 tx=0x7f053c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.504+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f053c037040 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.504+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f053c030c40 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.504+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f053c03a5d0 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.504+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f054c19dfa0 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.504+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f054c1a0c90 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.505+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f053c03b3f0 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.505+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f054c109c90 con 0x7f054c109810 2024-09-05T12:17:39.202 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.506+0000 7f0549600640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0530039020 0x7f053003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.506+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f053c06c610 con 0x7f054c109810 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.506+0000 7f0543400640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0530039020 0x7f053003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.506+0000 7f0543400640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0530039020 0x7f053003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05340098c0 tx=0x7f0534006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.512+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f053c03b6e0 con 0x7f054c109810 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.713+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f054c19e480 con 0x7f0530039020 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.939+0000 7f0549600640 1 -- 172.21.15.71:0/2437804720 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f054c19e480 con 0x7f0530039020 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0530039020 msgr2=0x7f053003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:39.203 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0530039020 0x7f053003b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f05340098c0 tx=0x7f0534006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:39.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 msgr2=0x7f054c19d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:39.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c19d7d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f053c0097f0 tx=0x7f053c0327b0 comp rx=0 tx=0).stop 2024-09-05T12:17:39.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 shutdown_connections 2024-09-05T12:17:39.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0530039020 0x7f053003b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:39.204 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 --2- 172.21.15.71:0/2437804720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f054c109810 0x7f054c19d7d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:39.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.941+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 >> 172.21.15.71:0/2437804720 conn(0x7f054c07ccf0 msgr2=0x7f054c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:39.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.942+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 shutdown_connections 2024-09-05T12:17:39.205 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:37.942+0000 7f0553000640 1 -- 172.21.15.71:0/2437804720 wait complete. 2024-09-05T12:17:39.205 INFO:teuthology.orchestra.run.smithi071.stdout:Fetching dashboard port number... 2024-09-05T12:17:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stdout 8443 2024-09-05T12:17:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.036+0000 7ff256a00640 1 Processor -- start 2024-09-05T12:17:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.036+0000 7ff256a00640 1 -- start start 2024-09-05T12:17:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.036+0000 7ff256a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25007c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.036+0000 7ff256a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff25007c7a0 con 0x7ff25007de00 2024-09-05T12:17:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.037+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25007c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.037+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25007c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49520/0 (socket says 172.21.15.71:49520) 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.037+0000 7ff24fe00640 1 -- 172.21.15.71:0/1986475442 learned_addr learned my addr 172.21.15.71:0/1986475442 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.037+0000 7ff24fe00640 1 -- 172.21.15.71:0/1986475442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25007c920 con 0x7ff25007de00 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.038+0000 7ff24fe00640 1 --2- 172.21.15.71:0/1986475442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25007c260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff240009820 tx=0x7ff24002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff15fe2e7d711cbb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.038+0000 7ff24ea00640 1 -- 172.21.15.71:0/1986475442 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff240037070 con 0x7ff25007de00 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.038+0000 7ff24ea00640 1 -- 172.21.15.71:0/1986475442 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240032920 con 0x7ff25007de00 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.038+0000 7ff24ea00640 1 -- 172.21.15.71:0/1986475442 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff240030810 con 0x7ff25007de00 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 -- 172.21.15.71:0/1986475442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 msgr2=0x7ff25007c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 --2- 172.21.15.71:0/1986475442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25007c260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff240009820 tx=0x7ff24002a270 comp rx=0 tx=0).stop 2024-09-05T12:17:40.897 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 -- 172.21.15.71:0/1986475442 shutdown_connections 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 --2- 172.21.15.71:0/1986475442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25007c260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 -- 172.21.15.71:0/1986475442 >> 172.21.15.71:0/1986475442 conn(0x7ff250102cc0 msgr2=0x7ff250105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 -- 172.21.15.71:0/1986475442 shutdown_connections 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.039+0000 7ff256a00640 1 -- 172.21.15.71:0/1986475442 wait complete. 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.040+0000 7ff256a00640 1 Processor -- start 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.040+0000 7ff256a00640 1 -- start start 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.040+0000 7ff256a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25019d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.040+0000 7ff256a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff25010db80 con 0x7ff25007de00 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.041+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25019d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:40.898 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.041+0000 7ff24fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25019d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49532/0 (socket says 172.21.15.71:49532) 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.041+0000 7ff24fe00640 1 -- 172.21.15.71:0/3975634565 learned_addr learned my addr 172.21.15.71:0/3975634565 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.041+0000 7ff24fe00640 1 -- 172.21.15.71:0/3975634565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25019dc90 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.041+0000 7ff24fe00640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25019d750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff240030380 tx=0x7ff240032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.042+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff240037040 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.042+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff240040070 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.042+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff24003a7b0 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.042+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25019df20 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.042+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2501a0c10 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.043+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25007c2e0 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.043+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff24003bc00 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.043+0000 7ff24cc00640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff2340390c0 0x7ff23403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.043+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff24006c800 con 0x7ff25007de00 2024-09-05T12:17:40.899 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.044+0000 7ff24f400640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff2340390c0 0x7ff23403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.044+0000 7ff24f400640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff2340390c0 0x7ff23403b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0098c0 tx=0x7ff23c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.049+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff240043360 con 0x7ff25007de00 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.225+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ff2501a0de0 con 0x7ff25007de00 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.225+0000 7ff24cc00640 1 -- 172.21.15.71:0/3975634565 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ff240040220 con 0x7ff25007de00 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff2340390c0 msgr2=0x7ff23403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff2340390c0 0x7ff23403b580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff23c0098c0 tx=0x7ff23c006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 msgr2=0x7ff25019d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25019d750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff240030380 tx=0x7ff240032670 comp rx=0 tx=0).stop 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 shutdown_connections 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff2340390c0 0x7ff23403b580 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 --2- 172.21.15.71:0/3975634565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25007de00 0x7ff25019d750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 >> 172.21.15.71:0/3975634565 conn(0x7ff250102cc0 msgr2=0x7ff250103910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 shutdown_connections 2024-09-05T12:17:40.900 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:40.228+0000 7ff256a00640 1 -- 172.21.15.71:0/3975634565 wait complete. 2024-09-05T12:17:40.910 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-05T12:17:40.910 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout disabled 2024-09-05T12:17:40.923 INFO:teuthology.orchestra.run.smithi071.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-05T12:17:40.923 INFO:teuthology.orchestra.run.smithi071.stdout:systemctl: stdout inactive 2024-09-05T12:17:40.924 INFO:teuthology.orchestra.run.smithi071.stdout:firewalld.service is not enabled 2024-09-05T12:17:40.924 INFO:teuthology.orchestra.run.smithi071.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-05T12:17:40.926 INFO:teuthology.orchestra.run.smithi071.stdout:Ceph Dashboard is now available at: 2024-09-05T12:17:40.926 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:40.926 INFO:teuthology.orchestra.run.smithi071.stdout: URL: https://smithi071.front.sepia.ceph.com:8443/ 2024-09-05T12:17:40.926 INFO:teuthology.orchestra.run.smithi071.stdout: User: admin 2024-09-05T12:17:40.926 INFO:teuthology.orchestra.run.smithi071.stdout: Password: pa4n7u40gn 2024-09-05T12:17:40.926 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:40.927 INFO:teuthology.orchestra.run.smithi071.stdout:Saving cluster configuration to /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config directory 2024-09-05T12:17:41.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:40 smithi071 ceph-mon[29845]: mgrmap e13: x(active, since 6s) 2024-09-05T12:17:41.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3975634565' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a3200640 1 Processor -- start 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a3200640 1 -- start start 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a407b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a407b780 con 0x7ff4a407cd50 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a407b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a407b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49540/0 (socket says 172.21.15.71:49540) 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.483+0000 7ff4a1e00640 1 -- 172.21.15.71:0/3747466208 learned_addr learned my addr 172.21.15.71:0/3747466208 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.484+0000 7ff4a1e00640 1 -- 172.21.15.71:0/3747466208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a407b900 con 0x7ff4a407cd50 2024-09-05T12:17:42.458 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.485+0000 7ff4a1e00640 1 --2- 172.21.15.71:0/3747466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a407b1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff494009e80 tx=0x7ff49402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205391b71bb9685b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.485+0000 7ff4a1400640 1 -- 172.21.15.71:0/3747466208 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff494036070 con 0x7ff4a407cd50 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.485+0000 7ff4a1400640 1 -- 172.21.15.71:0/3747466208 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4940318e0 con 0x7ff4a407cd50 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.485+0000 7ff4a1400640 1 -- 172.21.15.71:0/3747466208 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff494007a00 con 0x7ff4a407cd50 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 -- 172.21.15.71:0/3747466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 msgr2=0x7ff4a407b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 --2- 172.21.15.71:0/3747466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a407b1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff494009e80 tx=0x7ff49402a7a0 comp rx=0 tx=0).stop 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 -- 172.21.15.71:0/3747466208 shutdown_connections 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 --2- 172.21.15.71:0/3747466208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a407b1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 -- 172.21.15.71:0/3747466208 >> 172.21.15.71:0/3747466208 conn(0x7ff4a4102b90 msgr2=0x7ff4a4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 -- 172.21.15.71:0/3747466208 shutdown_connections 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.486+0000 7ff4a3200640 1 -- 172.21.15.71:0/3747466208 wait complete. 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.487+0000 7ff4a3200640 1 Processor -- start 2024-09-05T12:17:42.459 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.487+0000 7ff4a3200640 1 -- start start 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.487+0000 7ff4a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.487+0000 7ff4a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a410da50 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff4a1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a419d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49554/0 (socket says 172.21.15.71:49554) 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff4a1e00640 1 -- 172.21.15.71:0/1000720572 learned_addr learned my addr 172.21.15.71:0/1000720572 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff4a1e00640 1 -- 172.21.15.71:0/1000720572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a419db00 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff4a1e00640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a419d5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff494031cc0 tx=0x7ff494031cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff494036050 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff494042070 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff49403d8e0 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.488+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a419dd90 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.489+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a41a0a80 con 0x7ff4a407cd50 2024-09-05T12:17:42.460 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.490+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff49403bc80 con 0x7ff4a407cd50 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.490+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a407b230 con 0x7ff4a407cd50 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.490+0000 7ff49b400640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4880390c0 0x7ff48803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.490+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff49406d650 con 0x7ff4a407cd50 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.490+0000 7ff499400640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4880390c0 0x7ff48803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.491+0000 7ff499400640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4880390c0 0x7ff48803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0098c0 tx=0x7ff48c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.495+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4940477e0 con 0x7ff4a407cd50 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.754+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff4a41a1070 con 0x7ff4a407cd50 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.757+0000 7ff49b400640 1 -- 172.21.15.71:0/1000720572 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7ff494038260 con 0x7ff4a407cd50 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4880390c0 msgr2=0x7ff48803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4880390c0 0x7ff48803b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0098c0 tx=0x7ff48c006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 msgr2=0x7ff4a419d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:42.461 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a419d5c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff494031cc0 tx=0x7ff494031cf0 comp rx=0 tx=0).stop 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 shutdown_connections 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4880390c0 0x7ff48803b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 --2- 172.21.15.71:0/1000720572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a407cd50 0x7ff4a419d5c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 >> 172.21.15.71:0/1000720572 conn(0x7ff4a4102b90 msgr2=0x7ff4a41036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.759+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 shutdown_connections 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:/usr/bin/ceph: stderr 2024-09-05T12:17:41.760+0000 7ff4a3200640 1 -- 172.21.15.71:0/1000720572 wait complete. 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout:Or, if you are only running a single cluster on this host: 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.462 INFO:teuthology.orchestra.run.smithi071.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: ceph telemetry on 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout:For more information see: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:42.463 INFO:teuthology.orchestra.run.smithi071.stdout:Bootstrap complete. 2024-09-05T12:17:42.512 INFO:tasks.cephadm:Fetching config... 2024-09-05T12:17:42.512 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:17:42.513 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-05T12:17:42.549 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-05T12:17:42.550 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:17:42.550 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-05T12:17:42.630 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-05T12:17:42.630 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:17:42.631 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-05T12:17:42.727 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-05T12:17:42.727 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:17:42.727 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-05T12:17:42.802 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-05T12:17:42.802 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T12:17:42.924 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:17:42.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T12:17:42.994 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:17:43.007 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-05T12:17:43.048 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD9THxow+IlPvlPL8L3tfhbqYY3jgKTK9BeRBKP5z+rT5SaG9vhHUebRZszj/66M9qb2URG2OZE6GOZodVhEki7rhpMr0lNhVz/T57EDF1h/KbupwGqvqXXXBGZZfDm0GfR7QGvHd2pRhVglDy3sGDDP6kzGtaffl7r6Hc4tfhag8qc4ajaytoWbVWVhURlhDD3wIu7DOMnk/QxdO0ryIeWKAPMjAJwhjj4Y1jVwZq2LC6PB+oZuU3ARvXwwr0SLeSWLzY3bj0Xblnn3HSwmljQfoxESX/UrQz1x+lU+1vHRpkNlP/l2Ti+bkpQmBeGVXxvNX3A6bLION8SpLjssRoHVKMaNnQ/1BAQOGWdNKkFokT5lAIv7fAYpUe66gMOXAcUs4+gvnMAWdqrow2lojcod9uj/P55P/BXFDC8JsYzucN8r2iusCdAYtaTn0jhF6p/vX3CjCadKeZ/J5Jv3hjM5BSaGpXWGKYMJRvNAXOJITn8ub0unc5tmoWCvsVQlzc= ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:17:43.059 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:42 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1000720572' entity='client.admin' 2024-09-05T12:17:43.061 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-05T12:17:43.423 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 -- 172.21.15.71:0/3957888121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 msgr2=0x7f20fc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 --2- 172.21.15.71:0/3957888121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc101a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20f0009880 tx=0x7f20f002a5d0 comp rx=0 tx=0).stop 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 -- 172.21.15.71:0/3957888121 shutdown_connections 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 --2- 172.21.15.71:0/3957888121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc101a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 -- 172.21.15.71:0/3957888121 >> 172.21.15.71:0/3957888121 conn(0x7f20fc0fcdb0 msgr2=0x7f20fc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 -- 172.21.15.71:0/3957888121 shutdown_connections 2024-09-05T12:17:43.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.997+0000 7f2103c00640 1 -- 172.21.15.71:0/3957888121 wait complete. 2024-09-05T12:17:44.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.998+0000 7f2103c00640 1 Processor -- start 2024-09-05T12:17:44.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.998+0000 7f2103c00640 1 -- start start 2024-09-05T12:17:44.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.999+0000 7f2103c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:44.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.999+0000 7f2103c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20fc107710 con 0x7f20fc101600 2024-09-05T12:17:44.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.999+0000 7f2101200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:44.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.999+0000 7f2101200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35790/0 (socket says 172.21.15.71:35790) 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.999+0000 7f2101200640 1 -- 172.21.15.71:0/3416351003 learned_addr learned my addr 172.21.15.71:0/3416351003 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:43.999+0000 7f2101200640 1 -- 172.21.15.71:0/3416351003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20fc1972e0 con 0x7f20fc101600 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.000+0000 7f2101200640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f20f0004f60 tx=0x7f20f00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.000+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20f003f070 con 0x7f20fc101600 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.000+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20f0004d40 con 0x7f20fc101600 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.000+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20f0035030 con 0x7f20fc101600 2024-09-05T12:17:44.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.000+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20fc197570 con 0x7f20fc101600 2024-09-05T12:17:44.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.000+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20fc19a220 con 0x7f20fc101600 2024-09-05T12:17:44.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.001+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20fc107e90 con 0x7f20fc101600 2024-09-05T12:17:44.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.002+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f20f0042070 con 0x7f20fc101600 2024-09-05T12:17:44.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.002+0000 7f20f9600640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f20e40390c0 0x7f20e403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:44.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.002+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f20f006cdd0 con 0x7f20fc101600 2024-09-05T12:17:44.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.003+0000 7f20fbe00640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f20e40390c0 0x7f20e403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:44.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.003+0000 7f20fbe00640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f20e40390c0 0x7f20e403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f20e80098c0 tx=0x7f20e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:44.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.009+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20f0049030 con 0x7f20fc101600 2024-09-05T12:17:44.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.194+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f20fc02d940 con 0x7f20fc101600 2024-09-05T12:17:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.197+0000 7f20f9600640 1 -- 172.21.15.71:0/3416351003 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f20f00303a0 con 0x7f20fc101600 2024-09-05T12:17:44.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f20e40390c0 msgr2=0x7f20e403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:44.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f20e40390c0 0x7f20e403b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f20e80098c0 tx=0x7f20e8006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:44.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 msgr2=0x7f20fc196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:44.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc196da0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f20f0004f60 tx=0x7f20f00048f0 comp rx=0 tx=0).stop 2024-09-05T12:17:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 shutdown_connections 2024-09-05T12:17:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f20e40390c0 0x7f20e403b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 --2- 172.21.15.71:0/3416351003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc101600 0x7f20fc196da0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.199+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 >> 172.21.15.71:0/3416351003 conn(0x7f20fc0fcdb0 msgr2=0x7f20fc0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.200+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 shutdown_connections 2024-09-05T12:17:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:44.200+0000 7f2103c00640 1 -- 172.21.15.71:0/3416351003 wait complete. 2024-09-05T12:17:45.056 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-05T12:17:45.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-05T12:17:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:45 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3416351003' entity='client.admin' 2024-09-05T12:17:45.647 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:17:46.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.410+0000 7fcf49c00640 1 -- 172.21.15.71:0/3113696475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 msgr2=0x7fcf44074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:46.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.410+0000 7fcf49c00640 1 --2- 172.21.15.71:0/3113696475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf44074680 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009880 tx=0x7fcf3402a5d0 comp rx=0 tx=0).stop 2024-09-05T12:17:46.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.411+0000 7fcf49c00640 1 -- 172.21.15.71:0/3113696475 shutdown_connections 2024-09-05T12:17:46.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.411+0000 7fcf49c00640 1 --2- 172.21.15.71:0/3113696475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf44074680 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.411+0000 7fcf49c00640 1 -- 172.21.15.71:0/3113696475 >> 172.21.15.71:0/3113696475 conn(0x7fcf440fcd70 msgr2=0x7fcf440ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.411+0000 7fcf49c00640 1 -- 172.21.15.71:0/3113696475 shutdown_connections 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.411+0000 7fcf49c00640 1 -- 172.21.15.71:0/3113696475 wait complete. 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf49c00640 1 Processor -- start 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf49c00640 1 -- start start 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf49c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf4419d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf49c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4410dd40 con 0x7fcf44076220 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf4419d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf42a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf4419d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35806/0 (socket says 172.21.15.71:35806) 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.412+0000 7fcf42a00640 1 -- 172.21.15.71:0/2453822967 learned_addr learned my addr 172.21.15.71:0/2453822967 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.413+0000 7fcf42a00640 1 -- 172.21.15.71:0/2453822967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4419d8f0 con 0x7fcf44076220 2024-09-05T12:17:46.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.413+0000 7fcf42a00640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf4419d3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf34004f60 tx=0x7fcf340048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.413+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf3403f070 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.413+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf34004cc0 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.413+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf34035030 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.413+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4419db80 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.414+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf441a0870 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.415+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fcf34042070 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.415+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4410e480 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.415+0000 7fcf3b400640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcf28039060 0x7fcf2803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.415+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf3406cf10 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.415+0000 7fcf42000640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcf28039060 0x7fcf2803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.416+0000 7fcf42000640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcf28039060 0x7fcf2803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf300098c0 tx=0x7fcf30006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.421+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf34003dd0 con 0x7fcf44076220 2024-09-05T12:17:46.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.609+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fcf4402d830 con 0x7fcf28039060 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.612+0000 7fcf3b400640 1 -- 172.21.15.71:0/2453822967 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcf4402d830 con 0x7fcf28039060 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcf28039060 msgr2=0x7fcf2803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcf28039060 0x7fcf2803b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf300098c0 tx=0x7fcf30006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 msgr2=0x7fcf4419d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf4419d3b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf34004f60 tx=0x7fcf340048f0 comp rx=0 tx=0).stop 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 shutdown_connections 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcf28039060 0x7fcf2803b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 --2- 172.21.15.71:0/2453822967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf44076220 0x7fcf4419d3b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 >> 172.21.15.71:0/2453822967 conn(0x7fcf440fcd70 msgr2=0x7fcf440fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 shutdown_connections 2024-09-05T12:17:46.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:46.615+0000 7fcf49c00640 1 -- 172.21.15.71:0/2453822967 wait complete. 2024-09-05T12:17:47.595 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-09-05T12:17:47.595 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:17:47.595 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-09-05T12:17:47.615 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:17:47.615 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:17:47.673 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-09-05T12:17:47.673 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch host add smithi123 2024-09-05T12:17:47.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:47 smithi071 ceph-mon[29845]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:47.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.919+0000 7fe232a00640 1 -- 172.21.15.71:0/550638092 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe224031760 con 0x7fe2341037f0 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 -- 172.21.15.71:0/550638092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 msgr2=0x7fe234103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 --2- 172.21.15.71:0/550638092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234103bf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe224009880 tx=0x7fe22402a5d0 comp rx=0 tx=0).stop 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 -- 172.21.15.71:0/550638092 shutdown_connections 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 --2- 172.21.15.71:0/550638092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234103bf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 -- 172.21.15.71:0/550638092 >> 172.21.15.71:0/550638092 conn(0x7fe2340fefc0 msgr2=0x7fe234101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 -- 172.21.15.71:0/550638092 shutdown_connections 2024-09-05T12:17:48.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.920+0000 7fe23ae00640 1 -- 172.21.15.71:0/550638092 wait complete. 2024-09-05T12:17:48.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.921+0000 7fe23ae00640 1 Processor -- start 2024-09-05T12:17:48.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.921+0000 7fe23ae00640 1 -- start start 2024-09-05T12:17:48.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.921+0000 7fe23ae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:48.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.921+0000 7fe23ae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe234109900 con 0x7fe2341037f0 2024-09-05T12:17:48.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.922+0000 7fe233e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:48.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.922+0000 7fe233e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35828/0 (socket says 172.21.15.71:35828) 2024-09-05T12:17:48.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.922+0000 7fe233e00640 1 -- 172.21.15.71:0/3022170291 learned_addr learned my addr 172.21.15.71:0/3022170291 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:48.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.922+0000 7fe233e00640 1 -- 172.21.15.71:0/3022170291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2341994b0 con 0x7fe2341037f0 2024-09-05T12:17:48.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.922+0000 7fe233e00640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234198f70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe224004f60 tx=0x7fe2240048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:48.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.923+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe22403f070 con 0x7fe2341037f0 2024-09-05T12:17:48.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.923+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe224004d40 con 0x7fe2341037f0 2024-09-05T12:17:48.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.923+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe224035030 con 0x7fe2341037f0 2024-09-05T12:17:48.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.923+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe234199740 con 0x7fe2341037f0 2024-09-05T12:17:48.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.923+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23419c430 con 0x7fe2341037f0 2024-09-05T12:17:48.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.924+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe224042070 con 0x7fe2341037f0 2024-09-05T12:17:48.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.924+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23410a080 con 0x7fe2341037f0 2024-09-05T12:17:48.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.924+0000 7fe230c00640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe2180390c0 0x7fe21803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:48.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.924+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe22406cd80 con 0x7fe2341037f0 2024-09-05T12:17:48.926 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.925+0000 7fe233400640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe2180390c0 0x7fe21803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:48.927 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.926+0000 7fe233400640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe2180390c0 0x7fe21803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098c0 tx=0x7fe220006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:48.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:48.929+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe22404a260 con 0x7fe2341037f0 2024-09-05T12:17:49.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:49.141+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) -- 0x7fe23402d830 con 0x7fe2180390c0 2024-09-05T12:17:49.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:49.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:49.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:17:49.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:17:49.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:17:49.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:17:49.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:17:49.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:49 smithi071 ceph-mon[29845]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:51.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: Deploying cephadm binary to smithi123 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:17:51.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:52.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.358+0000 7fe230c00640 1 -- 172.21.15.71:0/3022170291 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe23402d830 con 0x7fe2180390c0 2024-09-05T12:17:52.359 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.360+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe2180390c0 msgr2=0x7fe21803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.360+0000 7fe23ae00640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe2180390c0 0x7fe21803b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098c0 tx=0x7fe220006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 msgr2=0x7fe234198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234198f70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe224004f60 tx=0x7fe2240048f0 comp rx=0 tx=0).stop 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 shutdown_connections 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe2180390c0 0x7fe21803b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 --2- 172.21.15.71:0/3022170291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe2341037f0 0x7fe234198f70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 >> 172.21.15.71:0/3022170291 conn(0x7fe2340fefc0 msgr2=0x7fe2340ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 shutdown_connections 2024-09-05T12:17:52.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:52.361+0000 7fe23ae00640 1 -- 172.21.15.71:0/3022170291 wait complete. 2024-09-05T12:17:53.353 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T12:17:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:53 smithi071 ceph-mon[29845]: Added host smithi123 2024-09-05T12:17:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:17:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:53 smithi071 ceph-mon[29845]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:17:53.710 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:17:54.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 -- 172.21.15.71:0/3851168132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 msgr2=0x7fe570076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:54.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 --2- 172.21.15.71:0/3851168132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe570076b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe560009880 tx=0x7fe56002a5d0 comp rx=0 tx=0).stop 2024-09-05T12:17:54.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 -- 172.21.15.71:0/3851168132 shutdown_connections 2024-09-05T12:17:54.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 --2- 172.21.15.71:0/3851168132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe570076b00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:54.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 -- 172.21.15.71:0/3851168132 >> 172.21.15.71:0/3851168132 conn(0x7fe5700ff290 msgr2=0x7fe5701016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:54.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 -- 172.21.15.71:0/3851168132 shutdown_connections 2024-09-05T12:17:54.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.251+0000 7fe56fe00640 1 -- 172.21.15.71:0/3851168132 wait complete. 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.252+0000 7fe56fe00640 1 Processor -- start 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.252+0000 7fe56fe00640 1 -- start start 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.253+0000 7fe56fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe57019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.253+0000 7fe56fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5701120e0 con 0x7fe570076720 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.253+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe57019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.253+0000 7fe56ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe57019d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51478/0 (socket says 172.21.15.71:51478) 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.253+0000 7fe56ea00640 1 -- 172.21.15.71:0/500978639 learned_addr learned my addr 172.21.15.71:0/500978639 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:54.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.253+0000 7fe56ea00640 1 -- 172.21.15.71:0/500978639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57019db20 con 0x7fe570076720 2024-09-05T12:17:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.254+0000 7fe56ea00640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe57019d5e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe560009230 tx=0x7fe560003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.254+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe56003f070 con 0x7fe570076720 2024-09-05T12:17:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.254+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57019ddb0 con 0x7fe570076720 2024-09-05T12:17:54.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.254+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe56003bd40 con 0x7fe570076720 2024-09-05T12:17:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.254+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe560035030 con 0x7fe570076720 2024-09-05T12:17:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.254+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe57019e210 con 0x7fe570076720 2024-09-05T12:17:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.255+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe560042070 con 0x7fe570076720 2024-09-05T12:17:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.255+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe570077e60 con 0x7fe570076720 2024-09-05T12:17:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.256+0000 7fe567400640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe554039060 0x7fe55403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:54.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.256+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe56006cc60 con 0x7fe570076720 2024-09-05T12:17:54.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.256+0000 7fe56e000640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe554039060 0x7fe55403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:54.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.257+0000 7fe56e000640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe554039060 0x7fe55403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098c0 tx=0x7fe55c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:54.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.261+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe560030d60 con 0x7fe570076720 2024-09-05T12:17:54.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.440+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe57019e9a0 con 0x7fe554039060 2024-09-05T12:17:54.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.442+0000 7fe567400640 1 -- 172.21.15.71:0/500978639 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fe57019e9a0 con 0x7fe554039060 2024-09-05T12:17:54.443 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:17:54.443 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-09-05T12:17:54.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe554039060 msgr2=0x7fe55403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:54.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe554039060 0x7fe55403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098c0 tx=0x7fe55c006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:54.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 msgr2=0x7fe57019d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:54.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe57019d5e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe560009230 tx=0x7fe560003d80 comp rx=0 tx=0).stop 2024-09-05T12:17:54.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 shutdown_connections 2024-09-05T12:17:54.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe554039060 0x7fe55403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:54.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 --2- 172.21.15.71:0/500978639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe570076720 0x7fe57019d5e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:54.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.444+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 >> 172.21.15.71:0/500978639 conn(0x7fe5700ff290 msgr2=0x7fe57010dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:54.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.445+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 shutdown_connections 2024-09-05T12:17:54.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:54.445+0000 7fe56fe00640 1 -- 172.21.15.71:0/500978639 wait complete. 2024-09-05T12:17:55.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:55.221 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-09-05T12:17:55.221 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:17:55.221 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-09-05T12:17:55.240 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:17:55.241 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:17:55.299 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-09-05T12:17:55.299 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch host add smithi174 2024-09-05T12:17:55.651 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:17:55.959 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:55 smithi071 ceph-mon[29845]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:17:55.959 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:55 smithi071 ceph-mon[29845]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:17:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 -- 172.21.15.71:0/772023112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 msgr2=0x7f45d8109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 --2- 172.21.15.71:0/772023112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d8109b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009880 tx=0x7f45cc02a5d0 comp rx=0 tx=0).stop 2024-09-05T12:17:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 -- 172.21.15.71:0/772023112 shutdown_connections 2024-09-05T12:17:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 --2- 172.21.15.71:0/772023112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d8109b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:56.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 -- 172.21.15.71:0/772023112 >> 172.21.15.71:0/772023112 conn(0x7f45d80ff450 msgr2=0x7f45d8101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:56.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 -- 172.21.15.71:0/772023112 shutdown_connections 2024-09-05T12:17:56.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.160+0000 7f45df600640 1 -- 172.21.15.71:0/772023112 wait complete. 2024-09-05T12:17:56.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.161+0000 7f45df600640 1 Processor -- start 2024-09-05T12:17:56.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.161+0000 7f45df600640 1 -- start start 2024-09-05T12:17:56.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.161+0000 7f45df600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:56.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.161+0000 7f45df600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f45d810de30 con 0x7f45d8109770 2024-09-05T12:17:56.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.162+0000 7f45dcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:56.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.162+0000 7f45dcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d81993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51492/0 (socket says 172.21.15.71:51492) 2024-09-05T12:17:56.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.162+0000 7f45dcc00640 1 -- 172.21.15.71:0/3712395801 learned_addr learned my addr 172.21.15.71:0/3712395801 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.162+0000 7f45dcc00640 1 -- 172.21.15.71:0/3712395801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d81998e0 con 0x7f45d8109770 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.162+0000 7f45dcc00640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d81993a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009230 tx=0x7f45cc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.162+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45cc03f070 con 0x7f45d8109770 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.163+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d8199bd0 con 0x7f45d8109770 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.163+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45cc03bc80 con 0x7f45d8109770 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.163+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d819dc70 con 0x7f45d8109770 2024-09-05T12:17:56.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.163+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45cc049060 con 0x7f45d8109770 2024-09-05T12:17:56.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.164+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f45cc042070 con 0x7f45d8109770 2024-09-05T12:17:56.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.164+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d8104eb0 con 0x7f45d8109770 2024-09-05T12:17:56.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.164+0000 7f45d5600640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45c0039060 0x7f45c003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:17:56.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.164+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f45cc06cd20 con 0x7f45d8109770 2024-09-05T12:17:56.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.164+0000 7f45d7e00640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45c0039060 0x7f45c003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:17:56.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.165+0000 7f45d7e00640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45c0039060 0x7f45c003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45c40098c0 tx=0x7f45c4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:17:56.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.170+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45cc040260 con 0x7f45d8109770 2024-09-05T12:17:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:56.346+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7f45d8109c30 con 0x7f45c0039060 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:17:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: Deploying cephadm binary to smithi174 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:57.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:17:57 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:17:59.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.490+0000 7f45d5600640 1 -- 172.21.15.71:0/3712395801 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f45d8109c30 con 0x7f45c0039060 2024-09-05T12:17:59.493 INFO:teuthology.orchestra.run.smithi071.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-09-05T12:17:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45c0039060 msgr2=0x7f45c003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45c0039060 0x7f45c003b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45c40098c0 tx=0x7f45c4006db0 comp rx=0 tx=0).stop 2024-09-05T12:17:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 msgr2=0x7f45d81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:17:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d81993a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009230 tx=0x7f45cc003d40 comp rx=0 tx=0).stop 2024-09-05T12:17:59.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 shutdown_connections 2024-09-05T12:17:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45c0039060 0x7f45c003b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 --2- 172.21.15.71:0/3712395801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45d8109770 0x7f45d81993a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:17:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 >> 172.21.15.71:0/3712395801 conn(0x7f45d80ff450 msgr2=0x7f45d8107280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:17:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 shutdown_connections 2024-09-05T12:17:59.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:17:59.493+0000 7f45df600640 1 -- 172.21.15.71:0/3712395801 wait complete. 2024-09-05T12:18:00.242 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch host ls --format=json 2024-09-05T12:18:00.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:00 smithi071 ceph-mon[29845]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:00.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:00.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:00.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:00.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:00.600 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.105+0000 7fe910000640 1 -- 172.21.15.71:0/3805969887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 msgr2=0x7fe908076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.105+0000 7fe910000640 1 --2- 172.21.15.71:0/3805969887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe908076b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009880 tx=0x7fe8fc02a5d0 comp rx=0 tx=0).stop 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.105+0000 7fe910000640 1 -- 172.21.15.71:0/3805969887 shutdown_connections 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.105+0000 7fe910000640 1 --2- 172.21.15.71:0/3805969887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe908076b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.105+0000 7fe910000640 1 -- 172.21.15.71:0/3805969887 >> 172.21.15.71:0/3805969887 conn(0x7fe9080ff450 msgr2=0x7fe908101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.106+0000 7fe910000640 1 -- 172.21.15.71:0/3805969887 shutdown_connections 2024-09-05T12:18:01.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.106+0000 7fe910000640 1 -- 172.21.15.71:0/3805969887 wait complete. 2024-09-05T12:18:01.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe910000640 1 Processor -- start 2024-09-05T12:18:01.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe910000640 1 -- start start 2024-09-05T12:18:01.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe910000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe9081a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:01.109 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe910000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe908112280 con 0x7fe908076720 2024-09-05T12:18:01.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe90d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe9081a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:01.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe90d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe9081a33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51512/0 (socket says 172.21.15.71:51512) 2024-09-05T12:18:01.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.107+0000 7fe90d600640 1 -- 172.21.15.71:0/3621487706 learned_addr learned my addr 172.21.15.71:0/3621487706 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:18:01.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.108+0000 7fe90d600640 1 -- 172.21.15.71:0/3621487706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9081a3930 con 0x7fe908076720 2024-09-05T12:18:01.110 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.108+0000 7fe90d600640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe9081a33f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009230 tx=0x7fe8fc003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:01.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.109+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8fc03f070 con 0x7fe908076720 2024-09-05T12:18:01.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.109+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8fc03bd40 con 0x7fe908076720 2024-09-05T12:18:01.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.109+0000 7fe910000640 1 -- 172.21.15.71:0/3621487706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9081a3bc0 con 0x7fe908076720 2024-09-05T12:18:01.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.109+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8fc035030 con 0x7fe908076720 2024-09-05T12:18:01.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.109+0000 7fe910000640 1 -- 172.21.15.71:0/3621487706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90819d5b0 con 0x7fe908076720 2024-09-05T12:18:01.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.111+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe8fc042070 con 0x7fe908076720 2024-09-05T12:18:01.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.111+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe908077e60 con 0x7fe908076720 2024-09-05T12:18:01.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.111+0000 7fe906000640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe8f00390c0 0x7fe8f003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:01.113 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.111+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe8fc06ccd0 con 0x7fe908076720 2024-09-05T12:18:01.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.111+0000 7fe90cc00640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe8f00390c0 0x7fe8f003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:01.114 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.112+0000 7fe90cc00640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe8f00390c0 0x7fe8f003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40098c0 tx=0x7fe8f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:01.118 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.116+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8fc03a3c0 con 0x7fe908076720 2024-09-05T12:18:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.296+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9081a4020 con 0x7fe8f00390c0 2024-09-05T12:18:01.298 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:01 smithi071 ceph-mon[29845]: Added host smithi174 2024-09-05T12:18:01.298 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:01 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.297+0000 7fe906000640 1 -- 172.21.15.71:0/3621487706 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fe8f40094d0 con 0x7fe8f00390c0 2024-09-05T12:18:01.299 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:18:01.299 INFO:teuthology.orchestra.run.smithi071.stdout:[{"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-09-05T12:18:01.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe8f00390c0 msgr2=0x7fe8f003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:01.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe8f00390c0 0x7fe8f003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40098c0 tx=0x7fe8f4006db0 comp rx=0 tx=0).stop 2024-09-05T12:18:01.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 msgr2=0x7fe9081a33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe9081a33f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009230 tx=0x7fe8fc003d80 comp rx=0 tx=0).stop 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 shutdown_connections 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe8f00390c0 0x7fe8f003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 --2- 172.21.15.71:0/3621487706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe908076720 0x7fe9081a33f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 >> 172.21.15.71:0/3621487706 conn(0x7fe9080ff450 msgr2=0x7fe90810dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.299+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 shutdown_connections 2024-09-05T12:18:01.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:01.300+0000 7fe8fb400640 1 -- 172.21.15.71:0/3621487706 wait complete. 2024-09-05T12:18:01.946 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-05T12:18:01.947 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd crush tunables default 2024-09-05T12:18:02.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:02 smithi071 ceph-mon[29845]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:02.300 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:18:02.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 -- 172.21.15.71:0/76497074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 msgr2=0x7fdcdc109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:02.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 --2- 172.21.15.71:0/76497074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc109b60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009880 tx=0x7fdccc02a5d0 comp rx=0 tx=0).stop 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 -- 172.21.15.71:0/76497074 shutdown_connections 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 --2- 172.21.15.71:0/76497074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc109b60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 -- 172.21.15.71:0/76497074 >> 172.21.15.71:0/76497074 conn(0x7fdcdc0ff480 msgr2=0x7fdcdc1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 -- 172.21.15.71:0/76497074 shutdown_connections 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.812+0000 7fdce1a00640 1 -- 172.21.15.71:0/76497074 wait complete. 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.813+0000 7fdce1a00640 1 Processor -- start 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.813+0000 7fdce1a00640 1 -- start start 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.813+0000 7fdce1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.813+0000 7fdce1a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdcdc10de40 con 0x7fdcdc109780 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcdaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:02.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcdaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51540/0 (socket says 172.21.15.71:51540) 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcdaa00640 1 -- 172.21.15.71:0/958420675 learned_addr learned my addr 172.21.15.71:0/958420675 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcdaa00640 1 -- 172.21.15.71:0/958420675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcdc199870 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcdaa00640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc199330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009230 tx=0x7fdccc003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc03f070 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdccc03bd40 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.814+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcdc199b60 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.815+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdccc035030 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.815+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcdc19dc00 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.816+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fdccc042070 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.816+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcdc104ec0 con 0x7fdcdc109780 2024-09-05T12:18:02.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.816+0000 7fdcd3400640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fdcc00390c0 0x7fdcc003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.816+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdccc06cc50 con 0x7fdcdc109780 2024-09-05T12:18:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.816+0000 7fdcda000640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fdcc00390c0 0x7fdcc003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.817+0000 7fdcda000640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fdcc00390c0 0x7fdcc003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80098c0 tx=0x7fdcc8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:02.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:02.821+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdccc036df0 con 0x7fdcdc109780 2024-09-05T12:18:03.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.004+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fdcdc19a690 con 0x7fdcdc109780 2024-09-05T12:18:03.112 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.110+0000 7fdcd3400640 1 -- 172.21.15.71:0/958420675 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fdccc030d60 con 0x7fdcdc109780 2024-09-05T12:18:03.112 INFO:teuthology.orchestra.run.smithi071.stderr:adjusted tunables profile to default 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.113+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fdcc00390c0 msgr2=0x7fdcc003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.113+0000 7fdce1a00640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fdcc00390c0 0x7fdcc003b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdcc80098c0 tx=0x7fdcc8006db0 comp rx=0 tx=0).stop 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.113+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 msgr2=0x7fdcdc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.113+0000 7fdce1a00640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc199330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009230 tx=0x7fdccc003d80 comp rx=0 tx=0).stop 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.114+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 shutdown_connections 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.114+0000 7fdce1a00640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fdcc00390c0 0x7fdcc003b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.114+0000 7fdce1a00640 1 --2- 172.21.15.71:0/958420675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcdc109780 0x7fdcdc199330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.114+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 >> 172.21.15.71:0/958420675 conn(0x7fdcdc0ff480 msgr2=0x7fdcdc100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.114+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 shutdown_connections 2024-09-05T12:18:03.116 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:03.114+0000 7fdce1a00640 1 -- 172.21.15.71:0/958420675 wait complete. 2024-09-05T12:18:03.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:03 smithi071 ceph-mon[29845]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:18:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:03 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/958420675' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T12:18:03.805 INFO:tasks.cephadm:Adding mon.a on smithi071 2024-09-05T12:18:03.805 INFO:tasks.cephadm:Adding mon.b on smithi123 2024-09-05T12:18:03.806 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-09-05T12:18:03.806 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch apply mon '3;smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c' 2024-09-05T12:18:04.049 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-05T12:18:04.094 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/958420675' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:04.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- 172.21.15.174:0/1043430383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 msgr2=0x7fd5a4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:04.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 --2- 172.21.15.174:0/1043430383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4106940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd598009880 tx=0x7fd59802a5d0 comp rx=0 tx=0).stop 2024-09-05T12:18:04.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- 172.21.15.174:0/1043430383 shutdown_connections 2024-09-05T12:18:04.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 --2- 172.21.15.174:0/1043430383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4106940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- 172.21.15.174:0/1043430383 >> 172.21.15.174:0/1043430383 conn(0x7fd5a40fc1b0 msgr2=0x7fd5a40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- 172.21.15.174:0/1043430383 shutdown_connections 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- 172.21.15.174:0/1043430383 wait complete. 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 Processor -- start 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- start start 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.593+0000 7fd5aae21640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a410ab90 con 0x7fd5a4106560 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a8bbd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a8bbd640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.174:52254/0 (socket says 172.21.15.174:52254) 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a8bbd640 1 -- 172.21.15.174:0/335710563 learned_addr learned my addr 172.21.15.174:0/335710563 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a8bbd640 1 -- 172.21.15.174:0/335710563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a41976a0 con 0x7fd5a4106560 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a8bbd640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4197160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd598009230 tx=0x7fd598003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:04.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd59803f070 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a4197930 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.594+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd59803bc80 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.595+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd598035030 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.595+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a419ba30 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.596+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd598042070 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.596+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a4101bd0 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.596+0000 7fd5a1ffb640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd5940390c0 0x7fd59403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.596+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd59806cdc0 con 0x7fd5a4106560 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.597+0000 7fd5a3fff640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd5940390c0 0x7fd59403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.597+0000 7fd5a3fff640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd5940390c0 0x7fd59403b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd5900098c0 tx=0x7fd590006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:04.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.602+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd598031e20 con 0x7fd5a4106560 2024-09-05T12:18:04.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.783+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) -- 0x7fd5a41984c0 con 0x7fd5940390c0 2024-09-05T12:18:04.789 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.789+0000 7fd5a1ffb640 1 -- 172.21.15.174:0/335710563 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd5a41984c0 con 0x7fd5940390c0 2024-09-05T12:18:04.790 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-09-05T12:18:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd5940390c0 msgr2=0x7fd59403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd5940390c0 0x7fd59403b580 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd5900098c0 tx=0x7fd590006db0 comp rx=0 tx=0).stop 2024-09-05T12:18:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 msgr2=0x7fd5a4197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4197160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd598009230 tx=0x7fd598003d40 comp rx=0 tx=0).stop 2024-09-05T12:18:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 shutdown_connections 2024-09-05T12:18:04.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd5940390c0 0x7fd59403b580 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 --2- 172.21.15.174:0/335710563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a4106560 0x7fd5a4197160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 >> 172.21.15.174:0/335710563 conn(0x7fd5a40fc1b0 msgr2=0x7fd5a4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 shutdown_connections 2024-09-05T12:18:04.793 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:04.791+0000 7fd5aae21640 1 -- 172.21.15.174:0/335710563 wait complete. 2024-09-05T12:18:05.371 DEBUG:teuthology.orchestra.run.smithi123:mon.b> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.b.service 2024-09-05T12:18:05.374 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.c.service 2024-09-05T12:18:05.377 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T12:18:05.377 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T12:18:05.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:05.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:05.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:05.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:06.128 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:06 smithi071 ceph-mon[29845]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:18:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:06 smithi071 ceph-mon[29845]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:06 smithi071 ceph-mon[29845]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c;count:3 2024-09-05T12:18:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:06 smithi071 ceph-mon[29845]: Deploying daemon mon.c on smithi174 2024-09-05T12:18:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:06 smithi071 ceph-mon[29845]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 -- 172.21.15.174:0/3780556727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 msgr2=0x7f86e4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 --2- 172.21.15.174:0/3780556727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4106920 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009810 tx=0x7f86e002a540 comp rx=0 tx=0).stop 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 -- 172.21.15.174:0/3780556727 shutdown_connections 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 --2- 172.21.15.174:0/3780556727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4106920 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 -- 172.21.15.174:0/3780556727 >> 172.21.15.174:0/3780556727 conn(0x7f86e4077310 msgr2=0x7f86e40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 -- 172.21.15.174:0/3780556727 shutdown_connections 2024-09-05T12:18:06.551 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.549+0000 7f86ebf0e640 1 -- 172.21.15.174:0/3780556727 wait complete. 2024-09-05T12:18:06.552 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.550+0000 7f86ebf0e640 1 Processor -- start 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.550+0000 7f86ebf0e640 1 -- start start 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.550+0000 7f86ebf0e640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.550+0000 7f86ebf0e640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f86e410ab70 con 0x7f86e4106540 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.550+0000 7f86e9caa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86e9caa640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.174:52266/0 (socket says 172.21.15.174:52266) 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86e9caa640 1 -- 172.21.15.174:0/1056987195 learned_addr learned my addr 172.21.15.174:0/1056987195 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86e9caa640 1 -- 172.21.15.174:0/1056987195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e4199890 con 0x7f86e4106540 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86e9caa640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4199350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009930 tx=0x7f86e0004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86e0031400 con 0x7f86e4106540 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e4199b20 con 0x7f86e4106540 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e00035f0 con 0x7f86e4106540 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86e003c440 con 0x7f86e4106540 2024-09-05T12:18:06.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.551+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e419dc20 con 0x7f86e4106540 2024-09-05T12:18:06.554 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.552+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f86e0039040 con 0x7f86e4106540 2024-09-05T12:18:06.554 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.553+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e4101bb0 con 0x7f86e4106540 2024-09-05T12:18:06.555 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.553+0000 7f86daffd640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86d0039060 0x7f86d003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:06.555 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.553+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f86e006d390 con 0x7f86e4106540 2024-09-05T12:18:06.555 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.553+0000 7f86e94a9640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86d0039060 0x7f86d003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:06.555 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.554+0000 7f86e94a9640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86d0039060 0x7f86d003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f86d40098c0 tx=0x7f86d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:06.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.558+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f86e0040030 con 0x7f86e4106540 2024-09-05T12:18:06.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.795+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f86e419a6b0 con 0x7f86e4106540 2024-09-05T12:18:06.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.795+0000 7f86daffd640 1 -- 172.21.15.174:0/1056987195 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f86e0044dc0 con 0x7f86e4106540 2024-09-05T12:18:06.797 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:18:06.797 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","modified":"2024-09-05T12:16:29.995084Z","created":"2024-09-05T12:16:29.995084Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T12:18:06.797 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-05T12:18:06.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86d0039060 msgr2=0x7f86d003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:06.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86d0039060 0x7f86d003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f86d40098c0 tx=0x7f86d4006db0 comp rx=0 tx=0).stop 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 msgr2=0x7f86e4199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4199350 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009930 tx=0x7f86e0004570 comp rx=0 tx=0).stop 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 shutdown_connections 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86d0039060 0x7f86d003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 --2- 172.21.15.174:0/1056987195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86e4106540 0x7f86e4199350 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 >> 172.21.15.174:0/1056987195 conn(0x7f86e4077310 msgr2=0x7f86e41908a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 shutdown_connections 2024-09-05T12:18:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:06.798+0000 7f86ebf0e640 1 -- 172.21.15.174:0/1056987195 wait complete. 2024-09-05T12:18:07.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/1056987195' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:08 smithi071 ceph-mon[29845]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:09.063 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T12:18:09.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T12:18:09.445 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:18:09.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.897+0000 7fe101ac1640 1 -- 172.21.15.174:0/1680927518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 msgr2=0x7fe0fc107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:09.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 --2- 172.21.15.174:0/1680927518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc107a40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009880 tx=0x7fe0ec02a5d0 comp rx=0 tx=0).stop 2024-09-05T12:18:09.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 -- 172.21.15.174:0/1680927518 shutdown_connections 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 --2- 172.21.15.174:0/1680927518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc107a40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 -- 172.21.15.174:0/1680927518 >> 172.21.15.174:0/1680927518 conn(0x7fe0fc0fd320 msgr2=0x7fe0fc0ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 -- 172.21.15.174:0/1680927518 shutdown_connections 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 -- 172.21.15.174:0/1680927518 wait complete. 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 Processor -- start 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.898+0000 7fe101ac1640 1 -- start start 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.899+0000 7fe101ac1640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.899+0000 7fe101ac1640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc10bd20 con 0x7fe0fc107660 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.899+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.899+0000 7fe0fb7fe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.174:60712/0 (socket says 172.21.15.174:60712) 2024-09-05T12:18:09.901 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.899+0000 7fe0fb7fe640 1 -- 172.21.15.174:0/2724800905 learned_addr learned my addr 172.21.15.174:0/2724800905 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.899+0000 7fe0fb7fe640 1 -- 172.21.15.174:0/2724800905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc1977e0 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.900+0000 7fe0fb7fe640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc1972a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009230 tx=0x7fe0ec003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.900+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0ec03f070 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.900+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc197ad0 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.900+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec03bc80 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.900+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0ec035030 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.900+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc19bb70 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.901+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe0ec042070 con 0x7fe0fc107660 2024-09-05T12:18:09.903 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.901+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0fc102da0 con 0x7fe0fc107660 2024-09-05T12:18:09.904 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.901+0000 7fe0f8ff9640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe0e4039060 0x7fe0e403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:09.904 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.901+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe0ec06cd80 con 0x7fe0fc107660 2024-09-05T12:18:09.904 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.901+0000 7fe0faffd640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe0e4039060 0x7fe0e403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:09.904 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.902+0000 7fe0faffd640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe0e4039060 0x7fe0e403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00098c0 tx=0x7fe0f0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:09.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:09.907+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe0ec031570 con 0x7fe0fc107660 2024-09-05T12:18:10.068 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 podman[34281]: 2024-09-05 12:18:10.068007832 +0000 UTC m=+0.129192302 container create 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731) 2024-09-05T12:18:10.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.154+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe0fc198600 con 0x7fe0fc107660 2024-09-05T12:18:10.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.155+0000 7fe0f8ff9640 1 -- 172.21.15.174:0/2724800905 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fe0ec0303a0 con 0x7fe0fc107660 2024-09-05T12:18:10.156 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:18:10.157 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","modified":"2024-09-05T12:16:29.995084Z","created":"2024-09-05T12:16:29.995084Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-05T12:18:10.157 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-05T12:18:10.158 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe0e4039060 msgr2=0x7fe0e403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:10.158 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe0e4039060 0x7fe0e403b520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00098c0 tx=0x7fe0f0006db0 comp rx=0 tx=0).stop 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 msgr2=0x7fe0fc1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc1972a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009230 tx=0x7fe0ec003d40 comp rx=0 tx=0).stop 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 shutdown_connections 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe0e4039060 0x7fe0e403b520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 --2- 172.21.15.174:0/2724800905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe0fc107660 0x7fe0fc1972a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 >> 172.21.15.174:0/2724800905 conn(0x7fe0fc0fd320 msgr2=0x7fe0fc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 shutdown_connections 2024-09-05T12:18:10.159 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:10.157+0000 7fe101ac1640 1 -- 172.21.15.174:0/2724800905 wait complete. 2024-09-05T12:18:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:10 smithi071 ceph-mon[29845]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 podman[34281]: 2024-09-05 12:18:10.46797425 +0000 UTC m=+0.529158723 container init 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 podman[34281]: 2024-09-05 12:18:10.471319904 +0000 UTC m=+0.532504384 container start 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: set uid:gid to 167:167 (ceph:ceph) 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable), process ceph-mon, pid 2 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: pidfile_write: ignore empty --pid-file 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: load: jerasure load: lrc 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: RocksDB version: 7.9.2 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Git sha 0 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Compile date 2024-09-05 09:22:07 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: DB SUMMARY 2024-09-05T12:18:10.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: DB Session ID: 2KKBU4NHN3E72RAVVIER 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: CURRENT file: CURRENT 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: IDENTITY file: IDENTITY 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.error_if_exists: 0 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.create_if_missing: 0 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.paranoid_checks: 1 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.env: 0x55d0da6d8940 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.fs: PosixFileSystem 2024-09-05T12:18:10.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.info_log: 0x55d0dc76b6e0 2024-09-05T12:18:10.763 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_file_opening_threads: 16 2024-09-05T12:18:10.764 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.statistics: (nil) 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.use_fsync: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_log_file_size: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.keep_log_file_num: 1000 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.recycle_log_file_num: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.allow_fallocate: 1 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.allow_mmap_reads: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.allow_mmap_writes: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.use_direct_reads: 0 2024-09-05T12:18:10.765 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.create_missing_column_families: 0 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.db_log_dir: 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.wal_dir: 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.advise_random_on_open: 1 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.db_write_buffer_size: 0 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.write_buffer_manager: 0x55d0dc77adc0 2024-09-05T12:18:10.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.rate_limiter: (nil) 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.wal_recovery_mode: 2 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.enable_thread_tracking: 0 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.enable_pipelined_write: 0 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.unordered_write: 0 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-05T12:18:10.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.row_cache: None 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.wal_filter: None 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.allow_ingest_behind: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.two_write_queues: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.manual_wal_flush: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.wal_compression: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.atomic_flush: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.log_readahead_size: 0 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-05T12:18:10.768 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.best_efforts_recovery: 0 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.allow_data_in_errors: 0 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.db_host_id: __hostname__ 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_background_jobs: 2 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_background_compactions: -1 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_subcompactions: 1 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_total_wal_size: 0 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-05T12:18:10.769 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_open_files: -1 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bytes_per_sync: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_readahead_size: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_background_flushes: -1 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Compression algorithms supported: 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kZSTD supported: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kXpressCompression supported: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kBZip2Compression supported: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kLZ4Compression supported: 1 2024-09-05T12:18:10.770 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kZlibCompression supported: 1 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kLZ4HCCompression supported: 1 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: kSnappyCompression supported: 1 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.merge_operator: 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_filter: None 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_filter_factory: None 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.sst_partitioner_factory: None 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-05T12:18:10.771 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d0dc76b320) 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x55d0dc75f1f0 2024-09-05T12:18:10.772 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-09-05T12:18:10.773 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.write_buffer_size: 33554432 2024-09-05T12:18:10.774 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_write_buffer_number: 2 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression: NoCompression 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression: Disabled 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.prefix_extractor: nullptr 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.num_levels: 7 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-05T12:18:10.775 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.level: 32767 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.strategy: 0 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.enabled: false 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-05T12:18:10.776 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.target_file_size_base: 67108864 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-05T12:18:10.777 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.arena_block_size: 1048576 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.disable_auto_compactions: 0 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-05T12:18:10.778 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.inplace_update_support: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.bloom_locality: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.max_successive_merges: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-05T12:18:10.779 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.paranoid_file_checks: 0 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.force_consistency_checks: 1 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.report_bg_io_stats: 0 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.ttl: 2592000 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.enable_blob_files: false 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.min_blob_size: 0 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.blob_file_size: 268435456 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-05T12:18:10.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.blob_file_starting_level: 0 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cdf16420-9645-42b6-9b4e-ce075fa396eb 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725538690501324, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725538690501896, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725538690, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cdf16420-9645-42b6-9b4e-ce075fa396eb", "db_session_id": "2KKBU4NHN3E72RAVVIER", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725538690502035, "job": 1, "event": "recovery_finished"} 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-05T12:18:10.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d0dc78ce00 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: DB pointer 0x55d0dc888000 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-05T12:18:10.782 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-05T12:18:10.783 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x55d0dc75f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-05T12:18:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(???) e0 preinit fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).mds e1 new map 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: btime 2024-09-05T12:16:33:931477+0000 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-05T12:18:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Added host smithi174 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/958420675' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T12:18:10.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/958420675' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c;count:3 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: Deploying daemon mon.c on smithi174 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:10.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/1056987195' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:10.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:10.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:10.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 ceph-mon[34313]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-05T12:18:10.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 bash[34281]: 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 2024-09-05T12:18:10.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:10 smithi174 systemd[1]: Started Ceph mon.c for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:18:12.132 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T12:18:12.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T12:18:12.390 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:18:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.841+0000 7f9abc540640 1 -- 172.21.15.174:0/2547162756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 msgr2=0x7f9aa00059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.841+0000 7f9abc540640 1 --2- 172.21.15.174:0/2547162756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9aa00059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9ab41016a0 tx=0x7f9ab002bce0 comp rx=0 tx=0).stop 2024-09-05T12:18:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.842+0000 7f9abc540640 1 -- 172.21.15.174:0/2547162756 shutdown_connections 2024-09-05T12:18:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.842+0000 7f9abc540640 1 --2- 172.21.15.174:0/2547162756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9aa00059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.842+0000 7f9abc540640 1 -- 172.21.15.174:0/2547162756 >> 172.21.15.174:0/2547162756 conn(0x7f9ab40fc220 msgr2=0x7f9ab40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:15.845 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.842+0000 7f9abc540640 1 -- 172.21.15.174:0/2547162756 shutdown_connections 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.842+0000 7f9abc540640 1 -- 172.21.15.174:0/2547162756 wait complete. 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.842+0000 7f9abc540640 1 Processor -- start 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9abc540640 1 -- start start 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9abc540640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9ab41021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9abc540640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9abc540640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4108670 con 0x7f9aa0003530 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9abc540640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41084f0 con 0x7f9ab4102730 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53358/0 (socket says 172.21.15.174:53358) 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 -- 172.21.15.174:0/4097715593 learned_addr learned my addr 172.21.15.174:0/4097715593 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 -- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 msgr2=0x7f9ab4102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T12:18:15.846 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 -- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 msgr2=0x7f9ab4102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9ab9adb640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.843+0000 7f9aba2dc640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9ab41021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aba2dc640 1 -- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 msgr2=0x7f9ab4102b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aba2dc640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aba2dc640 1 -- 172.21.15.174:0/4097715593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab41040e0 con 0x7f9aa0003530 2024-09-05T12:18:15.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aba2dc640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9ab41021f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ab002c2d0 tx=0x7f9ab002da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9ab0005400 con 0x7f9aa0003530 2024-09-05T12:18:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ab0003ad0 con 0x7f9aa0003530 2024-09-05T12:18:15.848 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab41a34c0 con 0x7f9aa0003530 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.844+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f9ab0004cd0 con 0x7f9aa0003530 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.845+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab41a3a50 con 0x7f9aa0003530 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.846+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9ab0037480 con 0x7f9aa0003530 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.846+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab4101ca0 con 0x7f9aa0003530 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.846+0000 7f9aab7fe640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9a98038e50 0x7f9a9803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.846+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ab006d9c0 con 0x7f9aa0003530 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.846+0000 7f9ab9adb640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9a98038e50 0x7f9a9803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:15.849 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.847+0000 7f9ab9adb640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9a98038e50 0x7f9a9803b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009100 tx=0x7f9aa4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:15.852 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:15.851+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ab0040600 con 0x7f9aa0003530 2024-09-05T12:18:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:18:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: mon.a calling monitor election 2024-09-05T12:18:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: mon.c calling monitor election 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: monmap epoch 2 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: last_changed 2024-09-05T12:18:10.531576+0000 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: min_mon_release 19 (squid) 2024-09-05T12:18:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: election_strategy: 1 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: fsmap 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: mgrmap e13: x(active, since 42s) 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: overall HEALTH_OK 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:15.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:18:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: mon.a calling monitor election 2024-09-05T12:18:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: mon.c calling monitor election 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: monmap epoch 2 2024-09-05T12:18:16.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: last_changed 2024-09-05T12:18:10.531576+0000 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: min_mon_release 19 (squid) 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: election_strategy: 1 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: fsmap 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: mgrmap e13: x(active, since 42s) 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: overall HEALTH_OK 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:16.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:16.013 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:16.090 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.089+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9ab41a3d30 con 0x7f9aa0003530 2024-09-05T12:18:16.090 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.090+0000 7f9aab7fe640 1 -- 172.21.15.174:0/4097715593 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f9ab00403a0 con 0x7f9aa0003530 2024-09-05T12:18:16.091 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:18:16.091 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","modified":"2024-09-05T12:18:10.531576Z","created":"2024-09-05T12:16:29.995084Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T12:18:16.091 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9a98038e50 msgr2=0x7f9a9803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9a98038e50 0x7f9a9803b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009100 tx=0x7f9aa4008040 comp rx=0 tx=0).stop 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 msgr2=0x7f9ab41021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9ab41021f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ab002c2d0 tx=0x7f9ab002da80 comp rx=0 tx=0).stop 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 shutdown_connections 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9a98038e50 0x7f9a9803b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ab4102730 0x7f9ab4102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:16.093 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 --2- 172.21.15.174:0/4097715593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9aa0003530 0x7f9ab41021f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:16.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 >> 172.21.15.174:0/4097715593 conn(0x7f9ab40fc220 msgr2=0x7f9ab406a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:16.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 shutdown_connections 2024-09-05T12:18:16.094 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:16.092+0000 7f9abc540640 1 -- 172.21.15.174:0/4097715593 wait complete. 2024-09-05T12:18:16.655 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:16 smithi174 ceph-mon[34313]: Deploying daemon mon.b on smithi123 2024-09-05T12:18:16.655 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:16 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/4097715593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:16.656 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:16 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:16 smithi071 ceph-mon[29845]: Deploying daemon mon.b on smithi123 2024-09-05T12:18:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:16 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/4097715593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:16 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:17.684 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T12:18:17.684 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T12:18:17.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:17 smithi071 ceph-mon[29845]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:17.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:18:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:18:17.530+0000 7f16dfc00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-05T12:18:17.933 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:18:18.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:17 smithi174 ceph-mon[34313]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:18.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 -- 172.21.15.174:0/397802334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 msgr2=0x7fb0b8102c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:18.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/397802334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8102c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac009880 tx=0x7fb0ac02a670 comp rx=0 tx=0).stop 2024-09-05T12:18:18.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 -- 172.21.15.174:0/397802334 shutdown_connections 2024-09-05T12:18:18.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/397802334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8102c00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:18.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 -- 172.21.15.174:0/397802334 >> 172.21.15.174:0/397802334 conn(0x7fb0b80fc220 msgr2=0x7fb0b80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:18.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 -- 172.21.15.174:0/397802334 shutdown_connections 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.338+0000 7fb0bed7d640 1 -- 172.21.15.174:0/397802334 wait complete. 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.339+0000 7fb0bed7d640 1 Processor -- start 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.339+0000 7fb0bed7d640 1 -- start start 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.339+0000 7fb0bed7d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.339+0000 7fb0bed7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 0x7fb0b819bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.339+0000 7fb0bed7d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810bd70 con 0x7fb0b8102820 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.339+0000 7fb0bed7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b810bbf0 con 0x7fb0b8197980 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.174:53282/0 (socket says 172.21.15.174:53282) 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 -- 172.21.15.174:0/2616438867 learned_addr learned my addr 172.21.15.174:0/2616438867 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b7fff640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 0x7fb0b819bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:18.360 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b7fff640 1 -- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 msgr2=0x7fb0b819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b7fff640 1 -- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 msgr2=0x7fb0b819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b7fff640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 0x7fb0b819bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 -- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 msgr2=0x7fb0b819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b7fff640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 0x7fb0b819bcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 0x7fb0b819bcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 -- 172.21.15.174:0/2616438867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b8197fa0 con 0x7fb0b8102820 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bcb19640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8197440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac032c50 tx=0x7fb0ac032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb0ac035070 con 0x7fb0b8102820 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b819c200 con 0x7fb0b8102820 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0ac003ce0 con 0x7fb0b8102820 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.340+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fb0ac03bc00 con 0x7fb0b8102820 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.341+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b819c6e0 con 0x7fb0b8102820 2024-09-05T12:18:18.361 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.341+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb0ac042030 con 0x7fb0b8102820 2024-09-05T12:18:18.362 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.342+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0b806ad10 con 0x7fb0b8102820 2024-09-05T12:18:18.362 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.342+0000 7fb0b5ffb640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb0a8038e50 0x7fb0a803b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:18.362 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.342+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb0ac040070 con 0x7fb0b8102820 2024-09-05T12:18:18.362 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.342+0000 7fb0b7fff640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb0a8038e50 0x7fb0a803b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:18.362 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.343+0000 7fb0b7fff640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb0a8038e50 0x7fb0a803b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4006fd0 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:18.362 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.347+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb0ac04c350 con 0x7fb0b8102820 2024-09-05T12:18:18.574 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.573+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb0b819c990 con 0x7fb0b8102820 2024-09-05T12:18:18.574 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.573+0000 7fb0b5ffb640 1 -- 172.21.15.174:0/2616438867 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fb0ac03b460 con 0x7fb0b8102820 2024-09-05T12:18:18.575 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:18:18.575 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","modified":"2024-09-05T12:18:10.531576Z","created":"2024-09-05T12:16:29.995084Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T12:18:18.575 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-05T12:18:18.576 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb0a8038e50 msgr2=0x7fb0a803b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb0a8038e50 0x7fb0a803b310 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4006fd0 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 msgr2=0x7fb0b8197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8197440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac032c50 tx=0x7fb0ac032c80 comp rx=0 tx=0).stop 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 shutdown_connections 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb0a8038e50 0x7fb0a803b310 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8197980 0x7fb0b819bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 --2- 172.21.15.174:0/2616438867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0b8102820 0x7fb0b8197440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 >> 172.21.15.174:0/2616438867 conn(0x7fb0b80fc220 msgr2=0x7fb0b81055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 shutdown_connections 2024-09-05T12:18:18.577 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:18.575+0000 7fb0bed7d640 1 -- 172.21.15.174:0/2616438867 wait complete. 2024-09-05T12:18:18.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:18 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/2616438867' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:18 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/2616438867' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).mds e1 new map 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: e1 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: btime 2024-09-05T12:16:33:931477+0000 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: legacy client fscid: -1 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: 2024-09-05T12:18:19.170 INFO:journalctl@ceph.mon.b.smithi123.stdout: No filesystems configured 2024-09-05T12:18:19.171 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:18:19.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Added host smithi174 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/958420675' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/958420675' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:19.432 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Saving service mon spec with placement smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c;count:3 2024-09-05T12:18:19.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Deploying daemon mon.c on smithi174 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/1056987195' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.a calling monitor election 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.c calling monitor election 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.434 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: monmap epoch 2 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: last_changed 2024-09-05T12:18:10.531576+0000 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: min_mon_release 19 (squid) 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: election_strategy: 1 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: fsmap 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:19.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mgrmap e13: x(active, since 42s) 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: overall HEALTH_OK 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: Deploying daemon mon.b on smithi123 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/4097715593' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/2616438867' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 ceph-mon[34321]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 bash[34302]: 1dcd4e6c34fc11d7c08497c125fa1f9e52a23c8f0f89841be25347ebdfd27f7a 2024-09-05T12:18:19.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:19 smithi123 systemd[1]: Started Ceph mon.b for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:18:20.170 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-05T12:18:20.170 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph mon dump -f json 2024-09-05T12:18:20.419 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:18:24.440 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.436+0000 7f2313644640 1 -- 172.21.15.174:0/1701065076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f4003530 msgr2=0x7f22f40059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.436+0000 7f2313644640 1 --2- 172.21.15.174:0/1701065076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f4003530 0x7f22f40059c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f230c069dc0 tx=0x7f230402e600 comp rx=0 tx=0).stop 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.436+0000 7f2313644640 1 -- 172.21.15.174:0/1701065076 shutdown_connections 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.436+0000 7f2313644640 1 --2- 172.21.15.174:0/1701065076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f4003530 0x7f22f40059c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.436+0000 7f2313644640 1 -- 172.21.15.174:0/1701065076 >> 172.21.15.174:0/1701065076 conn(0x7f230c0fb9a0 msgr2=0x7f230c0fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.437+0000 7f2313644640 1 -- 172.21.15.174:0/1701065076 shutdown_connections 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.437+0000 7f2313644640 1 -- 172.21.15.174:0/1701065076 wait complete. 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.437+0000 7f2313644640 1 Processor -- start 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.437+0000 7f2313644640 1 -- start start 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2313644640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 0x7f230c195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2313644640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2313644640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c199f90 0x7f230c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:24.441 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2313644640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1075b0 con 0x7f230c199f90 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2313644640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f230c107430 con 0x7f22f4003530 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2313644640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f230c107730 con 0x7f230c1956c0 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:42134/0 (socket says 172.21.15.174:42134) 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 -- 172.21.15.174:0/675357928 learned_addr learned my addr 172.21.15.174:0/675357928 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f23113e0640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 0x7f230c195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 msgr2=0x7f230c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 msgr2=0x7f230c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2311be1640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c199f90 0x7f230c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f23113e0640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 msgr2=0x7f230c195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f23113e0640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 msgr2=0x7f230c195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-05T12:18:24.442 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f23113e0640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 0x7f230c195180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f23113e0640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 0x7f230c195180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2310bdf640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2311be1640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 msgr2=0x7f230c199a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2311be1640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2311be1640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 msgr2=0x7f230c195180 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.438+0000 7f2311be1640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 0x7f230c195180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f2311be1640 1 -- 172.21.15.174:0/675357928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c1a1d00 con 0x7f230c199f90 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f2311be1640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c199f90 0x7f230c19a440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009fd0 tx=0x7f22fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc015020 con 0x7f230c199f90 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22fc011d10 con 0x7f230c199f90 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c1a1f30 con 0x7f230c199f90 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc00f2e0 con 0x7f230c199f90 2024-09-05T12:18:24.443 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.439+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1a2410 con 0x7f230c199f90 2024-09-05T12:18:24.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.440+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f22fc018020 con 0x7f230c199f90 2024-09-05T12:18:24.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.440+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c1086a0 con 0x7f230c199f90 2024-09-05T12:18:24.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.440+0000 7f230a7fc640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22f80394f0 0x7f22f803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:24.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.440+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f22fc04bd80 con 0x7f230c199f90 2024-09-05T12:18:24.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.441+0000 7f23113e0640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22f80394f0 0x7f22f803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:24.445 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.441+0000 7f23113e0640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22f80394f0 0x7f22f803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f230402ebf0 tx=0x7f230402eb00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:24.446 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.445+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22fc01f020 con 0x7f230c199f90 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: mon.c calling monitor election 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: mon.a calling monitor election 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: monmap epoch 3 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: last_changed 2024-09-05T12:18:19.175980+0000 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: min_mon_release 19 (squid) 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: election_strategy: 1 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-05T12:18:24.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: fsmap 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: mgrmap e13: x(active, since 51s) 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: overall HEALTH_OK 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: mon.c calling monitor election 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: mon.a calling monitor election 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: monmap epoch 3 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: last_changed 2024-09-05T12:18:19.175980+0000 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: min_mon_release 19 (squid) 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: election_strategy: 1 2024-09-05T12:18:24.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: fsmap 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: mgrmap e13: x(active, since 51s) 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: overall HEALTH_OK 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.663 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:24.699 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.697+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f230c1a2860 con 0x7f230c199f90 2024-09-05T12:18:24.699 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.698+0000 7f230a7fc640 1 -- 172.21.15.174:0/675357928 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f22fc018310 con 0x7f230c199f90 2024-09-05T12:18:24.699 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:18:24.699 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","modified":"2024-09-05T12:18:19.175980Z","created":"2024-09-05T12:16:29.995084Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-05T12:18:24.700 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-09-05T12:18:24.701 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22f80394f0 msgr2=0x7f22f803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:24.701 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22f80394f0 0x7f22f803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f230402ebf0 tx=0x7f230402eb00 comp rx=0 tx=0).stop 2024-09-05T12:18:24.701 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c199f90 msgr2=0x7f230c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c199f90 0x7f230c19a440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009fd0 tx=0x7f22fc00b870 comp rx=0 tx=0).stop 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 shutdown_connections 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22f80394f0 0x7f22f803b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c199f90 0x7f230c19a440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f230c1956c0 0x7f230c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 --2- 172.21.15.174:0/675357928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f4003530 0x7f230c195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 >> 172.21.15.174:0/675357928 conn(0x7f230c0fb9a0 msgr2=0x7f230c0fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 shutdown_connections 2024-09-05T12:18:24.702 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:24.700+0000 7f2313644640 1 -- 172.21.15.174:0/675357928 wait complete. 2024-09-05T12:18:25.250 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:25.250 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:25 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/675357928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:25.250 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:25.303 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-05T12:18:25.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph config generate-minimal-conf 2024-09-05T12:18:25.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:25.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:25 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/675357928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:25.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:25.674 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:18:26.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 -- 172.21.15.71:0/237357761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f011410dca0 msgr2=0x7f011410e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 --2- 172.21.15.71:0/237357761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f011410dca0 0x7f011410e080 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0104009880 tx=0x7f010402a5d0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 -- 172.21.15.71:0/237357761 shutdown_connections 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 --2- 172.21.15.71:0/237357761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f011410dca0 0x7f011410e080 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 -- 172.21.15.71:0/237357761 >> 172.21.15.71:0/237357761 conn(0x7f01140775c0 msgr2=0x7f01140799e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 -- 172.21.15.71:0/237357761 shutdown_connections 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.261+0000 7f0119200640 1 -- 172.21.15.71:0/237357761 wait complete. 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.262+0000 7f0119200640 1 Processor -- start 2024-09-05T12:18:26.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 -- start start 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 0x7f01141a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01141a3b90 0x7f011419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f011419dd10 0x7f011419e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f01141122c0 con 0x7f01141a3b90 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0114112140 con 0x7f011410dca0 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0119200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0114112440 con 0x7f011419dd10 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0111e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 0x7f01141a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0111e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 0x7f01141a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:41708/0 (socket says 172.21.15.71:41708) 2024-09-05T12:18:26.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0111400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01141a3b90 0x7f011419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.263+0000 7f0111e00640 1 -- 172.21.15.71:0/1927199468 learned_addr learned my addr 172.21.15.71:0/1927199468 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0112800640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f011419dd10 0x7f011419e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0111e00640 1 -- 172.21.15.71:0/1927199468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f011419dd10 msgr2=0x7f011419e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0111e00640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f011419dd10 0x7f011419e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0111e00640 1 -- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01141a3b90 msgr2=0x7f011419d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0111e00640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01141a3b90 0x7f011419d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0111e00640 1 -- 172.21.15.71:0/1927199468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01141aa4b0 con 0x7f011410dca0 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0111e00640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 0x7f01141a3650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0104006010 tx=0x7f0104003c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010403f070 con 0x7f011410dca0 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f010403b460 con 0x7f011410dca0 2024-09-05T12:18:26.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01141aa6e0 con 0x7f011410dca0 2024-09-05T12:18:26.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.264+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104035030 con 0x7f011410dca0 2024-09-05T12:18:26.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.265+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01141aac70 con 0x7f011410dca0 2024-09-05T12:18:26.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.265+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0104042070 con 0x7f011410dca0 2024-09-05T12:18:26.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.266+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0114109330 con 0x7f011410dca0 2024-09-05T12:18:26.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.266+0000 7f010aa00640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f00f80394f0 0x7f00f803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:26.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.266+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f010403ba00 con 0x7f011410dca0 2024-09-05T12:18:26.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.266+0000 7f0111400640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f00f80394f0 0x7f00f803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:26.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.267+0000 7f0111400640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f00f80394f0 0x7f00f803b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0100009a30 tx=0x7f0100009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:26.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.271+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0104039510 con 0x7f011410dca0 2024-09-05T12:18:26.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.460+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f01141aaed0 con 0x7f011410dca0 2024-09-05T12:18:26.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.461+0000 7f010aa00640 1 -- 172.21.15.71:0/1927199468 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f0104039770 con 0x7f011410dca0 2024-09-05T12:18:26.462 INFO:teuthology.orchestra.run.smithi071.stdout:# minimal ceph.conf for 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:26.462 INFO:teuthology.orchestra.run.smithi071.stdout:[global] 2024-09-05T12:18:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: fsid = 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:26.462 INFO:teuthology.orchestra.run.smithi071.stdout: mon_host = [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-05T12:18:26.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f00f80394f0 msgr2=0x7f00f803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:26.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f00f80394f0 0x7f00f803b9b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0100009a30 tx=0x7f0100009190 comp rx=0 tx=0).stop 2024-09-05T12:18:26.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 msgr2=0x7f01141a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:26.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 0x7f01141a3650 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0104006010 tx=0x7f0104003c00 comp rx=0 tx=0).stop 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 shutdown_connections 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f00f80394f0 0x7f00f803b9b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f011419dd10 0x7f011419e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01141a3b90 0x7f011419d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 --2- 172.21.15.71:0/1927199468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f011410dca0 0x7f01141a3650 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 >> 172.21.15.71:0/1927199468 conn(0x7f01140775c0 msgr2=0x7f01140779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.463+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 shutdown_connections 2024-09-05T12:18:26.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:26.464+0000 7f0119200640 1 -- 172.21.15.71:0/1927199468 wait complete. 2024-09-05T12:18:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:18:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.c calling monitor election 2024-09-05T12:18:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.a calling monitor election 2024-09-05T12:18:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: monmap epoch 3 2024-09-05T12:18:26.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: last_changed 2024-09-05T12:18:19.175980+0000 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: min_mon_release 19 (squid) 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: election_strategy: 1 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: fsmap 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mgrmap e13: x(active, since 51s) 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: overall HEALTH_OK 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:26.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:26.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:26.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:26.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:26.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/675357928' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-05T12:18:26.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: mon.b calling monitor election 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: mon.b calling monitor election 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: mon.c calling monitor election 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: mon.a calling monitor election 2024-09-05T12:18:27.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: monmap epoch 3 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: last_changed 2024-09-05T12:18:19.175980+0000 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: min_mon_release 19 (squid) 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: election_strategy: 1 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: fsmap 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: mgrmap e13: x(active, since 53s) 2024-09-05T12:18:27.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: overall HEALTH_OK 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:26 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1927199468' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.b calling monitor election 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.b calling monitor election 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.c calling monitor election 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.a calling monitor election 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: monmap epoch 3 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:27.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: last_changed 2024-09-05T12:18:19.175980+0000 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: min_mon_release 19 (squid) 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: election_strategy: 1 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: fsmap 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: mgrmap e13: x(active, since 53s) 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: overall HEALTH_OK 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.182 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.182 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.182 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:26 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1927199468' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: mon.b calling monitor election 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: mon.b calling monitor election 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: mon.c calling monitor election 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: mon.a calling monitor election 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: monmap epoch 3 2024-09-05T12:18:27.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: last_changed 2024-09-05T12:18:19.175980+0000 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: created 2024-09-05T12:16:29.995084+0000 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: min_mon_release 19 (squid) 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: election_strategy: 1 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: 0: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.a 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: fsmap 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: osdmap e4: 0 total, 0 up, 0 in 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: mgrmap e13: x(active, since 53s) 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: overall HEALTH_OK 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:27.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.264 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.264 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.264 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:27.264 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:26 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1927199468' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:27.304 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-05T12:18:27.305 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:18:27.305 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T12:18:27.365 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:18:27.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:18:27.462 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:18:27.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T12:18:27.490 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:18:27.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:18:27.555 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:18:27.555 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-09-05T12:18:27.585 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:18:27.585 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:18:27.653 INFO:tasks.cephadm:Adding mgr.x on smithi071 2024-09-05T12:18:27.653 INFO:tasks.cephadm:Adding mgr.y on smithi123 2024-09-05T12:18:27.654 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-09-05T12:18:27.654 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch apply mgr '3;smithi071=x;smithi123=y;smithi174=z' 2024-09-05T12:18:27.931 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.036 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: Reconfiguring daemon mon.a on smithi071 2024-09-05T12:18:28.037 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:28.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:28.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: Reconfiguring daemon mon.a on smithi071 2024-09-05T12:18:28.258 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:28.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 -- 172.21.15.174:0/3080858260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 msgr2=0x7f5bcc1009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:28.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/3080858260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc1009f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009c20 tx=0x7f5bbc02aa90 comp rx=0 tx=0).stop 2024-09-05T12:18:28.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 -- 172.21.15.174:0/3080858260 shutdown_connections 2024-09-05T12:18:28.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/3080858260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc1009f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.340 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 -- 172.21.15.174:0/3080858260 >> 172.21.15.174:0/3080858260 conn(0x7f5bcc0fbda0 msgr2=0x7f5bcc0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:28.341 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 -- 172.21.15.174:0/3080858260 shutdown_connections 2024-09-05T12:18:28.341 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 -- 172.21.15.174:0/3080858260 wait complete. 2024-09-05T12:18:28.343 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.339+0000 7f5bd16b8640 1 Processor -- start 2024-09-05T12:18:28.343 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 -- start start 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bcc197500 0x7f5bcc19c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 0x7f5bcc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc106700 con 0x7f5bcc1005f0 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc106580 con 0x7f5bcc197500 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bd16b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc106880 con 0x7f5bcc197980 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 0x7f5bcc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 0x7f5bcc19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56318/0 (socket says 172.21.15.174:56318) 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 -- 172.21.15.174:0/1970774702 learned_addr learned my addr 172.21.15.174:0/1970774702 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcaffd640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bca7fc640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bcc197500 0x7f5bcc19c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:28.344 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 -- 172.21.15.174:0/1970774702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bcc197500 msgr2=0x7f5bcc19c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bcc197500 0x7f5bcc19c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 -- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 msgr2=0x7f5bcc196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc196fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 -- 172.21.15.174:0/1970774702 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc19f3c0 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcaffd640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.340+0000 7f5bcb7fe640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 0x7f5bcc19eb00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400b450 tx=0x7f5bc400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.341+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4015020 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.341+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc19f6b0 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.341+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4011d70 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.341+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc400f340 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.341+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc19fc20 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.342+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5bc4018020 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.342+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bcc1907e0 con 0x7f5bcc197980 2024-09-05T12:18:28.345 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.342+0000 7f5bb3fff640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5bb40394f0 0x7f5bb403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:28.346 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.342+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5bc404bf00 con 0x7f5bcc197980 2024-09-05T12:18:28.346 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.342+0000 7f5bcaffd640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5bb40394f0 0x7f5bb403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:28.346 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.343+0000 7f5bcaffd640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5bb40394f0 0x7f5bb403b9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc000f80 tx=0x7f5bbc03a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:28.348 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.347+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bc401f020 con 0x7f5bcc197980 2024-09-05T12:18:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: Reconfiguring mon.a (unknown last config time)... 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: Reconfiguring daemon mon.a on smithi071 2024-09-05T12:18:28.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:18:28.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.525+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}) -- 0x7f5bcc0626c0 con 0x7f5bb40394f0 2024-09-05T12:18:28.533 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.532+0000 7f5bb3fff640 1 -- 172.21.15.174:0/1970774702 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5bcc0626c0 con 0x7f5bb40394f0 2024-09-05T12:18:28.534 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5bb40394f0 msgr2=0x7f5bb403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5bb40394f0 0x7f5bb403b9b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc000f80 tx=0x7f5bbc03a040 comp rx=0 tx=0).stop 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 msgr2=0x7f5bcc19eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 0x7f5bcc19eb00 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400b450 tx=0x7f5bc400b910 comp rx=0 tx=0).stop 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 shutdown_connections 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5bb40394f0 0x7f5bb403b9b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bcc197980 0x7f5bcc19eb00 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bcc197500 0x7f5bcc19c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.536 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 --2- 172.21.15.174:0/1970774702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bcc1005f0 0x7f5bcc196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:28.537 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 >> 172.21.15.174:0/1970774702 conn(0x7f5bcc0fbda0 msgr2=0x7f5bcc06a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:28.537 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 shutdown_connections 2024-09-05T12:18:28.537 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:18:28.535+0000 7f5bd16b8640 1 -- 172.21.15.174:0/1970774702 wait complete. 2024-09-05T12:18:29.145 DEBUG:teuthology.orchestra.run.smithi123:mgr.y> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.y.service 2024-09-05T12:18:29.147 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.z.service 2024-09-05T12:18:29.150 INFO:tasks.cephadm:Deploying OSDs... 2024-09-05T12:18:29.150 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:18:29.150 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T12:18:29.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T12:18:29.188 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 639 Links: 1 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-05 12:17:47.105205587 +0000 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-05 12:13:16.301180055 +0000 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-05 12:13:16.301180055 +0000 2024-09-05T12:18:29.266 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-05 12:13:16.301180055 +0000 2024-09-05T12:18:29.267 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T12:18:29.355 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:29.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:29.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-05T12:18:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-05T12:18:29.362 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000301521 s, 1.7 MB/s 2024-09-05T12:18:29.365 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T12:18:29.402 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-09-05T12:18:29.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:29.477 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T12:18:29.477 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 657 Links: 1 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-05 12:17:47.157204681 +0000 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-05 12:13:16.735172673 +0000 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-05 12:13:16.735172673 +0000 2024-09-05T12:18:29.478 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-05 12:13:16.735172673 +0000 2024-09-05T12:18:29.478 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T12:18:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:29.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-05T12:18:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-05T12:18:29.571 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.00027841 s, 1.8 MB/s 2024-09-05T12:18:29.574 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T12:18:29.613 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-09-05T12:18:29.689 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-05 12:17:47.220203583 +0000 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-05 12:13:17.207164645 +0000 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-05 12:13:17.207164645 +0000 2024-09-05T12:18:29.690 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-05 12:13:17.207164645 +0000 2024-09-05T12:18:29.690 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T12:18:29.788 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-05T12:18:29.788 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-05T12:18:29.789 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000299271 s, 1.7 MB/s 2024-09-05T12:18:29.791 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T12:18:29.829 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-09-05T12:18:29.905 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T12:18:29.905 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 697 Links: 1 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-05 12:17:47.270202712 +0000 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-05 12:13:17.668156803 +0000 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-05 12:13:17.668156803 +0000 2024-09-05T12:18:29.906 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-05 12:13:17.668156803 +0000 2024-09-05T12:18:29.906 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T12:18:29.999 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-09-05T12:18:30.000 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-09-05T12:18:30.000 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000282227 s, 1.8 MB/s 2024-09-05T12:18:30.002 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T12:18:30.039 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-05T12:18:30.039 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T12:18:30.058 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T12:18:30.058 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-05T12:18:30.117 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-05 12:18:18.619747974 +0000 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-05 12:12:45.749336025 +0000 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-05 12:12:45.749336025 +0000 2024-09-05T12:18:30.118 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-05 12:12:45.749336025 +0000 2024-09-05T12:18:30.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T12:18:30.184 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-05T12:18:30.185 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-05T12:18:30.185 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000216267 s, 2.4 MB/s 2024-09-05T12:18:30.186 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T12:18:30.245 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-05T12:18:30.245 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:30 smithi071 ceph-mon[29845]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:30.246 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:30 smithi071 ceph-mon[29845]: Saving service mgr spec with placement smithi071=x;smithi123=y;smithi174=z;count:3 2024-09-05T12:18:30.246 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:30 smithi071 ceph-mon[29845]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T12:18:30.246 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:30 smithi071 ceph-mon[29845]: Reconfiguring daemon mgr.x on smithi071 2024-09-05T12:18:30.246 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:30 smithi071 ceph-mon[29845]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:30.302 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T12:18:30.302 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:30.302 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-05T12:18:30.302 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:30.302 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:30.303 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-05 12:18:18.619747974 +0000 2024-09-05T12:18:30.303 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-05 12:12:46.076327831 +0000 2024-09-05T12:18:30.303 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-05 12:12:46.076327831 +0000 2024-09-05T12:18:30.303 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-05 12:12:46.076327831 +0000 2024-09-05T12:18:30.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T12:18:30.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:30 smithi123 ceph-mon[34321]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:30.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:30 smithi123 ceph-mon[34321]: Saving service mgr spec with placement smithi071=x;smithi123=y;smithi174=z;count:3 2024-09-05T12:18:30.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:30 smithi123 ceph-mon[34321]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T12:18:30.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:30 smithi123 ceph-mon[34321]: Reconfiguring daemon mgr.x on smithi071 2024-09-05T12:18:30.366 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:30 smithi123 ceph-mon[34321]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:30.368 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-05T12:18:30.368 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-05T12:18:30.369 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000213279 s, 2.4 MB/s 2024-09-05T12:18:30.370 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T12:18:30.427 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-05T12:18:30.483 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T12:18:30.483 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:30.483 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-05T12:18:30.483 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:30.484 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:30.484 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-05 12:18:18.620747949 +0000 2024-09-05T12:18:30.484 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-05 12:12:46.406319562 +0000 2024-09-05T12:18:30.484 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-05 12:12:46.406319562 +0000 2024-09-05T12:18:30.484 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-05 12:12:46.406319562 +0000 2024-09-05T12:18:30.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T12:18:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:30 smithi174 ceph-mon[34313]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi071=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:30 smithi174 ceph-mon[34313]: Saving service mgr spec with placement smithi071=x;smithi123=y;smithi174=z;count:3 2024-09-05T12:18:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:30 smithi174 ceph-mon[34313]: Reconfiguring mgr.x (unknown last config time)... 2024-09-05T12:18:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:30 smithi174 ceph-mon[34313]: Reconfiguring daemon mgr.x on smithi071 2024-09-05T12:18:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:30 smithi174 ceph-mon[34313]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-05T12:18:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-05T12:18:30.547 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000221752 s, 2.3 MB/s 2024-09-05T12:18:30.549 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T12:18:30.606 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-05 12:18:18.620747949 +0000 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-05 12:12:46.730311443 +0000 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-05 12:12:46.730311443 +0000 2024-09-05T12:18:30.662 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-05 12:12:46.730311443 +0000 2024-09-05T12:18:30.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T12:18:30.725 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-05T12:18:30.725 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-05T12:18:30.725 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000176529 s, 2.9 MB/s 2024-09-05T12:18:30.726 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T12:18:30.782 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-05T12:18:30.782 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-09-05T12:18:30.802 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-05T12:18:30.803 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-09-05T12:18:30.861 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-05 12:18:09.350666190 +0000 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-05 12:12:45.743392221 +0000 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-05 12:12:45.743392221 +0000 2024-09-05T12:18:30.862 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-05 12:12:45.743392221 +0000 2024-09-05T12:18:30.862 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-05T12:18:30.929 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-05T12:18:30.929 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-05T12:18:30.929 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000228474 s, 2.2 MB/s 2024-09-05T12:18:30.931 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-05T12:18:30.989 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-05 12:18:09.350666190 +0000 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-05 12:12:46.072388029 +0000 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-05 12:12:46.072388029 +0000 2024-09-05T12:18:31.047 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-05 12:12:46.072388029 +0000 2024-09-05T12:18:31.047 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-05T12:18:31.112 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-05T12:18:31.112 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-05T12:18:31.113 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000276641 s, 1.9 MB/s 2024-09-05T12:18:31.113 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-05T12:18:31.171 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-09-05T12:18:31.227 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-05T12:18:31.227 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:31.227 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-05T12:18:31.227 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:31.227 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:31.227 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-05 12:18:09.350666190 +0000 2024-09-05T12:18:31.228 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-05 12:12:46.400383850 +0000 2024-09-05T12:18:31.228 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-05 12:12:46.400383850 +0000 2024-09-05T12:18:31.228 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-05 12:12:46.400383850 +0000 2024-09-05T12:18:31.228 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-05T12:18:31.291 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-05T12:18:31.291 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-05T12:18:31.291 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000192238 s, 2.7 MB/s 2024-09-05T12:18:31.292 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-05T12:18:31.349 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-05 12:18:09.351666178 +0000 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-05 12:12:46.720379772 +0000 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-05 12:12:46.720379772 +0000 2024-09-05T12:18:31.404 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-05 12:12:46.720379772 +0000 2024-09-05T12:18:31.405 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-05T12:18:31.467 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-05T12:18:31.467 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-05T12:18:31.467 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000191624 s, 2.7 MB/s 2024-09-05T12:18:31.468 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-05T12:18:31.525 INFO:tasks.cephadm:Deploying osd.0 on smithi071 with /dev/vg_nvme/lv_4... 2024-09-05T12:18:31.525 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T12:18:31.899 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:18:31.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: Reconfiguring mon.b (monmap changed)... 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: Reconfiguring daemon mon.b on smithi123 2024-09-05T12:18:31.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:31 smithi071 ceph-mon[29845]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:31.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:31.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:31.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: Reconfiguring mon.b (monmap changed)... 2024-09-05T12:18:31.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:31.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:31.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:31.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: Reconfiguring daemon mon.b on smithi123 2024-09-05T12:18:31.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:31 smithi123 ceph-mon[34321]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:32.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:32.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:32.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: Reconfiguring mon.b (monmap changed)... 2024-09-05T12:18:32.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:32.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:32.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:32.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: Reconfiguring daemon mon.b on smithi123 2024-09-05T12:18:32.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:31 smithi174 ceph-mon[34313]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:33.053 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:33.064 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:33.064 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: Reconfiguring mon.c (monmap changed)... 2024-09-05T12:18:33.064 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:33.064 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:33.064 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:33.064 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:32 smithi071 ceph-mon[29845]: Reconfiguring daemon mon.c on smithi174 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: Reconfiguring mon.c (monmap changed)... 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:33.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:32 smithi123 ceph-mon[34321]: Reconfiguring daemon mon.c on smithi174 2024-09-05T12:18:33.256 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:33.256 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:33.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: Reconfiguring mon.c (monmap changed)... 2024-09-05T12:18:33.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-05T12:18:33.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-05T12:18:33.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:33.257 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:32 smithi174 ceph-mon[34313]: Reconfiguring daemon mon.c on smithi174 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:34.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T12:18:34.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:34.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:34.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:34 smithi071 ceph-mon[29845]: Deploying daemon mgr.z on smithi174 2024-09-05T12:18:34.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:34.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:34 smithi123 ceph-mon[34321]: Deploying daemon mgr.z on smithi174 2024-09-05T12:18:34.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:34.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:34 smithi174 ceph-mon[34313]: Deploying daemon mgr.z on smithi174 2024-09-05T12:18:34.871 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:18:34.903 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-09-05T12:18:35.260 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:18:35.468 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:35.406+0000 7f89f7d62040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:18:35.468 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:35.467+0000 7f89f7d62040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:18:35.760 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:35.613+0000 7f89f7d62040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:18:35.760 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:35.677+0000 7f89f7d62040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:18:35.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 -- 172.21.15.71:0/4020768025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4105810 msgr2=0x7f29b410c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:35.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 --2- 172.21.15.71:0/4020768025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4105810 0x7f29b410c0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009ca0 tx=0x7f29ac02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:18:35.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 -- 172.21.15.71:0/4020768025 shutdown_connections 2024-09-05T12:18:35.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 --2- 172.21.15.71:0/4020768025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4105810 0x7f29b410c0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 --2- 172.21.15.71:0/4020768025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b41052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 --2- 172.21.15.71:0/4020768025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b4103c50 0x7f29b4104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 -- 172.21.15.71:0/4020768025 >> 172.21.15.71:0/4020768025 conn(0x7f29b40ff3e0 msgr2=0x7f29b4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 -- 172.21.15.71:0/4020768025 shutdown_connections 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.827+0000 7f29ba400640 1 -- 172.21.15.71:0/4020768025 wait complete. 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 Processor -- start 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 -- start start 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4103c50 0x7f29b4199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b4105810 0x7f29b41a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:35.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410f8b0 con 0x7f29b4103c50 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410f730 con 0x7f29b4104e50 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.828+0000 7f29ba400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29b410fa30 con 0x7f29b4105810 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4103c50 0x7f29b4199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b9a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b4105810 0x7f29b41a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b419a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37568/0 (socket says 172.21.15.71:37568) 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 -- 172.21.15.71:0/1958348882 learned_addr learned my addr 172.21.15.71:0/1958348882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 -- 172.21.15.71:0/1958348882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b4105810 msgr2=0x7f29b41a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b4105810 0x7f29b41a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 -- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4103c50 msgr2=0x7f29b4199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4103c50 0x7f29b4199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 -- 172.21.15.71:0/1958348882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b41a1a20 con 0x7f29b4104e50 2024-09-05T12:18:35.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b9000640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4103c50 0x7f29b4199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b3e00640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b419a330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29a000c920 tx=0x7f29a000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0007c70 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.829+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29a0007e10 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.830+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b41a1d10 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.830+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0018890 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.830+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b41a2240 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.831+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f29a000f450 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.831+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b410af20 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.831+0000 7f29b1600640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f299c0392c0 0x7f299c03b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.831+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f29a004c470 con 0x7f29b4104e50 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.831+0000 7f29b9000640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f299c0392c0 0x7f299c03b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:18:35.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.832+0000 7f29b9000640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f299c0392c0 0x7f299c03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29b4104cb0 tx=0x7f29a8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:18:35.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:35.836+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29a0014030 con 0x7f29b4104e50 2024-09-05T12:18:36.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:36.018+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f29b4062630 con 0x7f299c0392c0 2024-09-05T12:18:36.158 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:35.824+0000 7f89f7d62040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:18:36.159 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:35.961+0000 7f89f7d62040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:18:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:36 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:18:36.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:36 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:36.453 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:36.275+0000 7f89f7d62040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:18:36.453 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:36.334+0000 7f89f7d62040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:18:36.454 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:36.703 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:36.521+0000 7f89f7d62040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:18:37.359 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-mon[34321]: Deploying daemon mgr.y on smithi123 2024-09-05T12:18:37.359 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-mon[34321]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:37.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:37 smithi174 ceph-mon[34313]: Deploying daemon mgr.y on smithi123 2024-09-05T12:18:37.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:37 smithi174 ceph-mon[34313]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:37.612 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:37 smithi071 ceph-mon[29845]: Deploying daemon mgr.y on smithi123 2024-09-05T12:18:37.612 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:37 smithi071 ceph-mon[29845]: from='client.24107 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:18:37.679 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-mgr[35689]: -- 172.21.15.123:0/4134139696 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56145b2fd0e0 con 0x56145b2da800 2024-09-05T12:18:37.701 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:37.439+0000 7f89f7d62040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:18:37.701 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:37.508+0000 7f89f7d62040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:18:37.995 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:37.698+0000 7fc861909040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:18:37.996 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 bash[35670]: c9167a856f555ca8fd87540539f411373a4bb9fd44308f22671aabd6507b2720 2024-09-05T12:18:37.996 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 systemd[1]: Started Ceph mgr.y for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:18:37.996 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:37.769+0000 7fc861909040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:18:37.996 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:37.924+0000 7fc861909040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:18:37.996 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:37.994+0000 7fc861909040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:18:38.309 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:38.148+0000 7fc861909040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:38.310 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.510 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.258+0000 7f89f7d62040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:18:38.510 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.383+0000 7f89f7d62040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:18:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:38.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:38.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.612 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:38.308+0000 7fc861909040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:18:38.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:38.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.616 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:38.616 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:38.820 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.528+0000 7f89f7d62040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:18:38.820 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.594+0000 7f89f7d62040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:18:38.821 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.751+0000 7f89f7d62040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:18:38.821 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.819+0000 7f89f7d62040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:18:38.884 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:38.635+0000 7fc861909040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:18:38.884 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:38.696+0000 7fc861909040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:18:39.112 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:38.879+0000 7f89f7d62040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:18:39.113 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:39.046+0000 7f89f7d62040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:18:39.113 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:39.111+0000 7f89f7d62040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:18:39.179 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:38.882+0000 7fc861909040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:18:39.387 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:39.181+0000 7f89f7d62040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:18:39.387 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:39.265+0000 7f89f7d62040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:18:39.387 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:39.327+0000 7f89f7d62040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:18:39.387 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:39.385+0000 7f89f7d62040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:18:40.179 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:39.782+0000 7fc861909040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:39.849+0000 7fc861909040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2904816864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]: dispatch 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]: dispatch 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]': finished 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:39 smithi123 ceph-mon[34321]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1243246547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:18:40.260 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:18:40.174+0000 7f89f7d62040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:18:40.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2904816864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]: dispatch 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]: dispatch 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]': finished 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:40.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:40.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:40.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1243246547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:18:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2904816864' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43d56d11-d83e-4c6b-be5b-80824118d6a2"}]': finished 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: osdmap e5: 1 total, 0 up, 1 in 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:40.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1243246547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:18:40.813 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:40.558+0000 7fc861909040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:18:40.813 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:40.671+0000 7fc861909040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:18:40.813 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:40.811+0000 7fc861909040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:18:41.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:41.049+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f29a0018a30 con 0x7f29b4104e50 2024-09-05T12:18:41.089 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-mon[34321]: Standby manager daemon z started 2024-09-05T12:18:41.089 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T12:18:41.089 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:18:41.089 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T12:18:41.089 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:18:41.089 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:40.871+0000 7fc861909040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:18:41.090 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.018+0000 7fc861909040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:18:41.090 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.087+0000 7fc861909040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:18:41.370 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.147+0000 7fc861909040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:18:41.370 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.304+0000 7fc861909040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:18:41.370 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.369+0000 7fc861909040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:18:41.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:41 smithi071 ceph-mon[29845]: Standby manager daemon z started 2024-09-05T12:18:41.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:41 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T12:18:41.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:41 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:18:41.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:41 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T12:18:41.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:41 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:18:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:41 smithi174 ceph-mon[34313]: Standby manager daemon z started 2024-09-05T12:18:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:41 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T12:18:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:41 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:18:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:41 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T12:18:41.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:41 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/3518840445' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:18:41.634 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.435+0000 7fc861909040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:18:41.634 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.517+0000 7fc861909040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:18:41.635 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.575+0000 7fc861909040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:18:41.635 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:41.633+0000 7fc861909040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:18:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:42 smithi071 ceph-mon[29845]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-05T12:18:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:42 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T12:18:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:42 smithi071 ceph-mon[29845]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:42.414 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:42 smithi123 ceph-mon[34321]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-05T12:18:42.414 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:42 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T12:18:42.414 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:42 smithi123 ceph-mon[34321]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:42 smithi174 ceph-mon[34313]: mgrmap e14: x(active, since 68s), standbys: z 2024-09-05T12:18:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:42 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T12:18:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:42 smithi174 ceph-mon[34313]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:42.679 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:18:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:18:42.412+0000 7fc861909040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:18:43.064 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:43.063+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mon.2 v2:172.21.15.123:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f29a0004f40 con 0x7f29b4104e50 2024-09-05T12:18:43.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:43 smithi071 ceph-mon[29845]: Standby manager daemon y started 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:43 smithi174 ceph-mon[34313]: Standby manager daemon y started 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:18:43.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T12:18:43.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:18:43.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:43 smithi123 ceph-mon[34321]: Standby manager daemon y started 2024-09-05T12:18:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T12:18:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:18:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T12:18:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3567564956' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:18:44.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:44 smithi071 ceph-mon[29845]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:44.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:44 smithi071 ceph-mon[29845]: mgrmap e15: x(active, since 70s), standbys: z, y 2024-09-05T12:18:44.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T12:18:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:44 smithi123 ceph-mon[34321]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:44.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:44 smithi123 ceph-mon[34321]: mgrmap e15: x(active, since 70s), standbys: z, y 2024-09-05T12:18:44.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T12:18:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:44 smithi174 ceph-mon[34313]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:44 smithi174 ceph-mon[34313]: mgrmap e15: x(active, since 70s), standbys: z, y 2024-09-05T12:18:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T12:18:46.380 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:46 smithi071 ceph-mon[29845]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:46.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:46 smithi123 ceph-mon[34321]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:46.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:46 smithi174 ceph-mon[34313]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:48.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:48 smithi071 ceph-mon[29845]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:48.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T12:18:48.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:48.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:48 smithi123 ceph-mon[34321]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:48.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T12:18:48.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:48 smithi174 ceph-mon[34313]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-05T12:18:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:49.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:49 smithi071 ceph-mon[29845]: Deploying daemon osd.0 on smithi071 2024-09-05T12:18:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:49 smithi123 ceph-mon[34321]: Deploying daemon osd.0 on smithi071 2024-09-05T12:18:49.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:49 smithi174 ceph-mon[34313]: Deploying daemon osd.0 on smithi071 2024-09-05T12:18:50.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:50 smithi071 ceph-mon[29845]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:50.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:50 smithi123 ceph-mon[34321]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:50 smithi174 ceph-mon[34313]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:52.391 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:52 smithi071 ceph-mon[29845]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:52.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:52 smithi123 ceph-mon[34321]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:52 smithi174 ceph-mon[34313]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:54 smithi071 ceph-mon[29845]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:54 smithi123 ceph-mon[34321]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:54 smithi174 ceph-mon[34313]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:55.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:55.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:55.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:55.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:55.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:55.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:18:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:56.341 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:56 smithi071 ceph-mon[29845]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:56.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:56 smithi123 ceph-mon[34321]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:56 smithi174 ceph-mon[34313]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:58 smithi071 ceph-mon[29845]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:18:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:58 smithi123 ceph-mon[34321]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:58.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:58.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:58.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:18:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:58 smithi174 ceph-mon[34313]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:18:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:18:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:18:58.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:18:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:18:59.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.908+0000 7f29b1600640 1 -- 172.21.15.71:0/1958348882 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f29b4062630 con 0x7f299c0392c0 2024-09-05T12:18:59.911 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 0 on host 'smithi071' 2024-09-05T12:18:59.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f299c0392c0 msgr2=0x7f299c03b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:59.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f299c0392c0 0x7f299c03b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29b4104cb0 tx=0x7f29a8005f90 comp rx=0 tx=0).stop 2024-09-05T12:18:59.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 msgr2=0x7f29b419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:18:59.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b419a330 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29a000c920 tx=0x7f29a000cde0 comp rx=0 tx=0).stop 2024-09-05T12:18:59.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 shutdown_connections 2024-09-05T12:18:59.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f299c0392c0 0x7f299c03b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:59.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b4105810 0x7f29b41a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:59.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b4104e50 0x7f29b419a330 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:59.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 --2- 172.21.15.71:0/1958348882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29b4103c50 0x7f29b4199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:18:59.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.912+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 >> 172.21.15.71:0/1958348882 conn(0x7f29b40ff3e0 msgr2=0x7f29b41080f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:18:59.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.913+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 shutdown_connections 2024-09-05T12:18:59.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:18:59.913+0000 7f29ba400640 1 -- 172.21.15.71:0/1958348882 wait complete. 2024-09-05T12:19:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:00 smithi071 ceph-mon[29845]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:00.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:00 smithi123 ceph-mon[34321]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:00.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:00.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:00.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:00 smithi174 ceph-mon[34313]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:00.691 DEBUG:teuthology.orchestra.run.smithi071:osd.0> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.0.service 2024-09-05T12:19:00.695 INFO:tasks.cephadm:Deploying osd.1 on smithi123 with /dev/vg_nvme/lv_4... 2024-09-05T12:19:00.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T12:19:00.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0[44160]: 2024-09-05T12:19:00.835+0000 7f9af5a4a640 -1 osd.0 0 log_to_monitors true 2024-09-05T12:19:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:19:01.274 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:01 smithi071 ceph-mon[29845]: from='osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T12:19:01.274 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:01 smithi071 ceph-mon[29845]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T12:19:01.400 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:01 smithi123 ceph-mon[34321]: from='osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T12:19:01.400 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:01 smithi123 ceph-mon[34321]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T12:19:01.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:01 smithi174 ceph-mon[34313]: from='osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T12:19:01.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:01 smithi174 ceph-mon[34313]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-05T12:19:02.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:02 smithi123 ceph-mon[34321]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:02.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:02 smithi123 ceph-mon[34321]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T12:19:02.297 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:02 smithi123 ceph-mon[34321]: from='osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:19:02.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:02 smithi123 ceph-mon[34321]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T12:19:02.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:02 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:02.298 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:02 smithi123 ceph-mon[34321]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:19:02.390 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0[44160]: 2024-09-05T12:19:02.126+0000 7f9af0200640 -1 osd.0 0 waiting for initial osdmap 2024-09-05T12:19:02.390 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0[44160]: 2024-09-05T12:19:02.129+0000 7f9aeb200640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:19:02.390 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-mon[29845]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:02.390 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-mon[29845]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T12:19:02.390 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-mon[29845]: from='osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:19:02.390 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-mon[29845]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T12:19:02.390 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:02.390 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:02 smithi071 ceph-mon[29845]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:19:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:02 smithi174 ceph-mon[34313]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:02 smithi174 ceph-mon[34313]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-05T12:19:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:02 smithi174 ceph-mon[34313]: from='osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:19:02.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:02 smithi174 ceph-mon[34313]: osdmap e6: 1 total, 0 up, 1 in 2024-09-05T12:19:02.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:02 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:02.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:02 smithi174 ceph-mon[34313]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:19:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:03 smithi071 ceph-mon[29845]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:19:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:03 smithi071 ceph-mon[29845]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T12:19:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:03 smithi123 ceph-mon[34321]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:19:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:03 smithi123 ceph-mon[34321]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T12:19:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:19:03.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-09-05T12:19:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:03 smithi174 ceph-mon[34313]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:19:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:03 smithi174 ceph-mon[34313]: osdmap e7: 1 total, 0 up, 1 in 2024-09-05T12:19:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:03.742 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:19:04.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 -- 172.21.15.123:0/1814190811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c100ad0 msgr2=0x7f719c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:04.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 --2- 172.21.15.123:0/1814190811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c100ad0 0x7f719c100ed0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f718c009900 tx=0x7f718c02a410 comp rx=0 tx=0).stop 2024-09-05T12:19:04.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 -- 172.21.15.123:0/1814190811 shutdown_connections 2024-09-05T12:19:04.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 --2- 172.21.15.123:0/1814190811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f719c102690 0x7f719c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:04.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 --2- 172.21.15.123:0/1814190811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f719c101cd0 0x7f719c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:04.206 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 --2- 172.21.15.123:0/1814190811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c100ad0 0x7f719c100ed0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 -- 172.21.15.123:0/1814190811 >> 172.21.15.123:0/1814190811 conn(0x7f719c0fc280 msgr2=0x7f719c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 -- 172.21.15.123:0/1814190811 shutdown_connections 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 -- 172.21.15.123:0/1814190811 wait complete. 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 Processor -- start 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.189+0000 7f71a3769640 1 -- start start 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f719c100ad0 0x7f719c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f719c101cd0 0x7f719c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 0x7f719c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10d8c0 con 0x7f719c101cd0 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10d740 con 0x7f719c102690 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f719c10da40 con 0x7f719c100ad0 2024-09-05T12:19:04.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 0x7f719c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 0x7f719c19f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45656/0 (socket says 172.21.15.123:45656) 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 -- 172.21.15.123:0/3678361203 learned_addr learned my addr 172.21.15.123:0/3678361203 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1505640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f719c100ad0 0x7f719c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 -- 172.21.15.123:0/3678361203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f719c100ad0 msgr2=0x7f719c197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f719c100ad0 0x7f719c197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 -- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f719c101cd0 msgr2=0x7f719c198430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f719c101cd0 0x7f719c198430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 -- 172.21.15.123:0/3678361203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719c19fb20 con 0x7f719c102690 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a1d06640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 0x7f719c19f4b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f719400cce0 tx=0x7f719400a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719400f450 con 0x7f719c102690 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f719400fad0 con 0x7f719c102690 2024-09-05T12:19:04.208 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719c19fe10 con 0x7f719c102690 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7194018c10 con 0x7f719c102690 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.190+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f719c1a0340 con 0x7f719c102690 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.192+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f719c107e80 con 0x7f719c102690 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.193+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f719400f5f0 con 0x7f719c102690 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.194+0000 7f71927fc640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71880a2ba0 0x7f71880a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.194+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f71940cdcb0 con 0x7f719c102690 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.194+0000 7f71a1505640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71880a2ba0 0x7f71880a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.195+0000 7f71a1505640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71880a2ba0 0x7f71880a5060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f718c006010 tx=0x7f718c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:19:04.209 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.197+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7194014030 con 0x7f719c102690 2024-09-05T12:19:04.375 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:04.374+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f719c062630 con 0x7f71880a2ba0 2024-09-05T12:19:04.375 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:04 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:19:04.376 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:04 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:19:04.376 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:04 smithi123 ceph-mon[34321]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:04.376 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:04 smithi123 ceph-mon[34321]: osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] boot 2024-09-05T12:19:04.376 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:04 smithi123 ceph-mon[34321]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T12:19:04.376 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:04.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:04 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:19:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:04 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:19:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:04 smithi071 ceph-mon[29845]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:04 smithi071 ceph-mon[29845]: osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] boot 2024-09-05T12:19:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:04 smithi071 ceph-mon[29845]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T12:19:04.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:04 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:19:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:04 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:19:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:04 smithi174 ceph-mon[34313]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-05T12:19:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:04 smithi174 ceph-mon[34313]: osd.0 [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] boot 2024-09-05T12:19:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:04 smithi174 ceph-mon[34313]: osdmap e8: 1 total, 1 up, 1 in 2024-09-05T12:19:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:05.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:05.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: osdmap e9: 1 total, 1 up, 1 in 2024-09-05T12:19:05.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:05.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:05.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:05.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:06.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:06 smithi123 ceph-mon[34321]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:06.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:06 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi071 2024-09-05T12:19:06.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:06 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 17089M 2024-09-05T12:19:06.292 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:06 smithi123 ceph-mon[34321]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:06.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:06 smithi071 ceph-mon[29845]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:06 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi071 2024-09-05T12:19:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:06 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 17089M 2024-09-05T12:19:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:06 smithi071 ceph-mon[29845]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:06 smithi174 ceph-mon[34313]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:06 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi071 2024-09-05T12:19:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:06 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 17089M 2024-09-05T12:19:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:06 smithi174 ceph-mon[34313]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:07.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/2238475625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]: dispatch 2024-09-05T12:19:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:07 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]: dispatch 2024-09-05T12:19:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:07 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]': finished 2024-09-05T12:19:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:07 smithi071 ceph-mon[29845]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T12:19:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:07.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/2238475625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]: dispatch 2024-09-05T12:19:07.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:07 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]: dispatch 2024-09-05T12:19:07.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:07 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]': finished 2024-09-05T12:19:07.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:07 smithi123 ceph-mon[34321]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T12:19:07.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/2238475625' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]: dispatch 2024-09-05T12:19:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:07 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]: dispatch 2024-09-05T12:19:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:07 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4234dbed-6c5c-4abf-892a-d13fc22b44b8"}]': finished 2024-09-05T12:19:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:07 smithi174 ceph-mon[34313]: osdmap e10: 2 total, 1 up, 2 in 2024-09-05T12:19:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:08 smithi071 ceph-mon[29845]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:08.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:08 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/3475158736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:08 smithi123 ceph-mon[34321]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:08 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/3475158736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:08 smithi174 ceph-mon[34313]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:08 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/3475158736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:10 smithi071 ceph-mon[29845]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:10.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:10 smithi123 ceph-mon[34321]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:10 smithi174 ceph-mon[34313]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:12.370 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:12 smithi123 ceph-mon[34321]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:12.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:12 smithi071 ceph-mon[29845]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:12 smithi174 ceph-mon[34313]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:14.330 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:14 smithi123 ceph-mon[34321]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:14.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:14 smithi071 ceph-mon[29845]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:14 smithi174 ceph-mon[34313]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:15.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T12:19:15.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:15.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:15 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T12:19:15.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:15 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-05T12:19:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:16.305 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:16 smithi123 ceph-mon[34321]: Deploying daemon osd.1 on smithi123 2024-09-05T12:19:16.306 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:16 smithi123 ceph-mon[34321]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:16.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:16 smithi071 ceph-mon[29845]: Deploying daemon osd.1 on smithi123 2024-09-05T12:19:16.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:16 smithi071 ceph-mon[29845]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:16 smithi174 ceph-mon[34313]: Deploying daemon osd.1 on smithi123 2024-09-05T12:19:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:16 smithi174 ceph-mon[34313]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:18.239 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:18 smithi123 ceph-mon[34321]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:18.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:18 smithi071 ceph-mon[29845]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:18 smithi174 ceph-mon[34313]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:20.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:20 smithi071 ceph-mon[29845]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:20.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:20 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:20.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:20 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:20.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:20 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:20.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:20 smithi123 ceph-mon[34321]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:20.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:20 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:20.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:20 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:20 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:20 smithi174 ceph-mon[34313]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:20 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:20 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:20 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:22 smithi071 ceph-mon[29845]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:22.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:22 smithi123 ceph-mon[34321]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:22 smithi174 ceph-mon[34313]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:23.497 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.496+0000 7f71927fc640 1 -- 172.21.15.123:0/3678361203 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f719c062630 con 0x7f71880a2ba0 2024-09-05T12:19:23.498 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 1 on host 'smithi123' 2024-09-05T12:19:23.499 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71880a2ba0 msgr2=0x7f71880a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71880a2ba0 0x7f71880a5060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f718c006010 tx=0x7f718c000bc0 comp rx=0 tx=0).stop 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 msgr2=0x7f719c19f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 0x7f719c19f4b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f719400cce0 tx=0x7f719400a620 comp rx=0 tx=0).stop 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 shutdown_connections 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71880a2ba0 0x7f71880a5060 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f719c102690 0x7f719c19f4b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f719c101cd0 0x7f719c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 --2- 172.21.15.123:0/3678361203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f719c100ad0 0x7f719c197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 >> 172.21.15.123:0/3678361203 conn(0x7f719c0fc280 msgr2=0x7f719c105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 shutdown_connections 2024-09-05T12:19:23.500 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:19:23.498+0000 7f71a3769640 1 -- 172.21.15.123:0/3678361203 wait complete. 2024-09-05T12:19:23.860 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:23.861 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:24.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:24.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:24.171 DEBUG:teuthology.orchestra.run.smithi123:osd.1> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.1.service 2024-09-05T12:19:24.174 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-09-05T12:19:24.174 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-05T12:19:24.425 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:19:26.330 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:26 smithi123 ceph-mon[34321]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:26.330 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:26 smithi123 ceph-mon[34321]: from='osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T12:19:26.330 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:26 smithi123 ceph-mon[34321]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T12:19:26.330 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1[38751]: 2024-09-05T12:19:25.958+0000 7f4236f71640 -1 osd.1 0 log_to_monitors true 2024-09-05T12:19:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:26 smithi071 ceph-mon[29845]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:26 smithi071 ceph-mon[29845]: from='osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T12:19:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:26 smithi071 ceph-mon[29845]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T12:19:26.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:26 smithi174 ceph-mon[34313]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:26.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:26 smithi174 ceph-mon[34313]: from='osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T12:19:26.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:26 smithi174 ceph-mon[34313]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-05T12:19:26.911 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:19:26.944 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-09-05T12:19:27.197 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:19:27.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T12:19:27.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T12:19:27.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:27.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:19:27.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:27.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: osdmap e11: 2 total, 1 up, 2 in 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:27.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:27.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:27.677 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2035300911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c01026d0 msgr2=0x7fb1c0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2035300911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c01026d0 0x7fb1c0108f60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009ca0 tx=0x7fb1b802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2035300911 shutdown_connections 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2035300911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c01026d0 0x7fb1c0108f60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2035300911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1c0101d10 0x7fb1c0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2035300911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0100b10 0x7fb1c0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2035300911 >> 172.21.15.174:0/2035300911 conn(0x7fb1c00fc2a0 msgr2=0x7fb1c00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2035300911 shutdown_connections 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.668+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2035300911 wait complete. 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 Processor -- start 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 -- start start 2024-09-05T12:19:27.678 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0100b10 0x7fb1c006b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 0x7fb1c019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1c01026d0 0x7fb1c019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c010c770 con 0x7fb1c0100b10 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c010c5f0 con 0x7fb1c01026d0 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1c53f5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c010c8f0 con 0x7fb1c0101d10 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 0x7fb1c019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 0x7fb1c019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56320/0 (socket says 172.21.15.174:56320) 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 -- 172.21.15.174:0/2130529511 learned_addr learned my addr 172.21.15.174:0/2130529511 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1bf7fe640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1c01026d0 0x7fb1c019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 -- 172.21.15.174:0/2130529511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1c01026d0 msgr2=0x7fb1c019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1c01026d0 0x7fb1c019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 -- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0100b10 msgr2=0x7fb1c006b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.669+0000 7fb1be7fc640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0100b10 0x7fb1c006b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1be7fc640 1 -- 172.21.15.174:0/2130529511 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c019fb20 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1be7fc640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 0x7fb1c019cbf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000d950 tx=0x7fb1b000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b0017070 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1b0009c30 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c019fdb0 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000f6b0 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.670+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c01a0260 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.671+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c0107e60 con 0x7fb1c0101d10 2024-09-05T12:19:27.680 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.672+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fb1b00109d0 con 0x7fb1c0101d10 2024-09-05T12:19:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.673+0000 7fb1b7fff640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb1ac0a2af0 0x7fb1ac0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.673+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7fb1b00cd2d0 con 0x7fb1c0101d10 2024-09-05T12:19:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.673+0000 7fb1beffd640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb1ac0a2af0 0x7fb1ac0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.674+0000 7fb1beffd640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb1ac0a2af0 0x7fb1ac0a4fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0101b70 tx=0x7fb1a8005c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:19:27.681 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.677+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1b0009de0 con 0x7fb1c0101d10 2024-09-05T12:19:27.854 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:27.853+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fb1c0062630 con 0x7fb1ac0a2af0 2024-09-05T12:19:28.139 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1[38751]: 2024-09-05T12:19:27.763+0000 7f4232ef4640 -1 osd.1 0 waiting for initial osdmap 2024-09-05T12:19:28.139 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1[38751]: 2024-09-05T12:19:27.769+0000 7f422ad14640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:19:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi123 2024-09-05T12:19:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-05T12:19:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='osd.1 ' entity='osd.1' 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:28.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi123 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='osd.1 ' entity='osd.1' 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi123 2024-09-05T12:19:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-05T12:19:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: osdmap e12: 2 total, 1 up, 2 in 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='osd.1 ' entity='osd.1' 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:29.191 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:19:29.191 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:19:29.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:29.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:29.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] boot 2024-09-05T12:19:29.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T12:19:29.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] boot 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T12:19:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:29.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:19:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:19:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: osd.1 [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] boot 2024-09-05T12:19:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: osdmap e13: 2 total, 2 up, 2 in 2024-09-05T12:19:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/3310970560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]: dispatch 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]: dispatch 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]': finished 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T12:19:30.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:30 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:30.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T12:19:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/3310970560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]: dispatch 2024-09-05T12:19:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]: dispatch 2024-09-05T12:19:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]': finished 2024-09-05T12:19:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T12:19:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:30 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-05T12:19:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: osdmap e14: 2 total, 2 up, 2 in 2024-09-05T12:19:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/3310970560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]: dispatch 2024-09-05T12:19:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]: dispatch 2024-09-05T12:19:30.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1b0236e-25dc-47d7-b952-edec01683c2e"}]': finished 2024-09-05T12:19:30.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: osdmap e15: 3 total, 2 up, 3 in 2024-09-05T12:19:30.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:30 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:31.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:31 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/255064198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:31.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:31 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/255064198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:31.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:31 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/255064198' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:32.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:32 smithi071 ceph-mon[29845]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:32.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:32 smithi123 ceph-mon[34321]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:32 smithi174 ceph-mon[34313]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:34 smithi071 ceph-mon[29845]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:34.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:34 smithi123 ceph-mon[34321]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:34 smithi174 ceph-mon[34313]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:36.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:36 smithi071 ceph-mon[29845]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:36 smithi123 ceph-mon[34321]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:36.468 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:36 smithi174 ceph-mon[34313]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:38.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:38 smithi174 ceph-mon[34313]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:38.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:38 smithi071 ceph-mon[29845]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:38.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:38 smithi123 ceph-mon[34321]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:39.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T12:19:39.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T12:19:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:39.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-05T12:19:39.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:40.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:40 smithi174 ceph-mon[34313]: Deploying daemon osd.2 on smithi174 2024-09-05T12:19:40.385 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:40 smithi174 ceph-mon[34313]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:40.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:40 smithi123 ceph-mon[34321]: Deploying daemon osd.2 on smithi174 2024-09-05T12:19:40.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:40 smithi123 ceph-mon[34321]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:40 smithi071 ceph-mon[29845]: Deploying daemon osd.2 on smithi174 2024-09-05T12:19:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:40 smithi071 ceph-mon[29845]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:42.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:42 smithi174 ceph-mon[34313]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:42 smithi123 ceph-mon[34321]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:42 smithi071 ceph-mon[29845]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:44 smithi174 ceph-mon[34313]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:44 smithi123 ceph-mon[34321]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:44.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:44.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:44.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:44 smithi071 ceph-mon[29845]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:46.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:46 smithi174 ceph-mon[34313]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:46.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:46 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:46.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:46 smithi071 ceph-mon[29845]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:46.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:46 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:46 smithi123 ceph-mon[34321]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:46 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:47.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:47.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.457 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.457 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:47.457 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:47.457 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:47.759 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-09-05T12:19:47.759 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.758+0000 7fb1b7fff640 1 -- 172.21.15.174:0/2130529511 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb1c0062630 con 0x7fb1ac0a2af0 2024-09-05T12:19:47.761 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb1ac0a2af0 msgr2=0x7fb1ac0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:47.761 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb1ac0a2af0 0x7fb1ac0a4fb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0101b70 tx=0x7fb1a8005c10 comp rx=0 tx=0).stop 2024-09-05T12:19:47.761 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 msgr2=0x7fb1c019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 0x7fb1c019cbf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000d950 tx=0x7fb1b000de10 comp rx=0 tx=0).stop 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 shutdown_connections 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb1ac0a2af0 0x7fb1ac0a4fb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1c01026d0 0x7fb1c019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1c0101d10 0x7fb1c019cbf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 --2- 172.21.15.174:0/2130529511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1c0100b10 0x7fb1c006b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 >> 172.21.15.174:0/2130529511 conn(0x7fb1c00fc2a0 msgr2=0x7fb1c0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 shutdown_connections 2024-09-05T12:19:47.762 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:19:47.760+0000 7fb1c53f5640 1 -- 172.21.15.174:0/2130529511 wait complete. 2024-09-05T12:19:48.312 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:48 smithi174 ceph-mon[34313]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:48.312 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:48.312 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:48.312 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:48.345 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.2.service 2024-09-05T12:19:48.348 INFO:tasks.cephadm:Deploying osd.3 on smithi071 with /dev/vg_nvme/lv_3... 2024-09-05T12:19:48.348 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T12:19:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:48 smithi071 ceph-mon[29845]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:48 smithi123 ceph-mon[34321]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:19:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:48.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:48.706 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:19:50.011 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2[39237]: 2024-09-05T12:19:49.864+0000 7fab1dfb1640 -1 osd.2 0 log_to_monitors true 2024-09-05T12:19:50.316 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:50 smithi071 ceph-mon[29845]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:50.316 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:50 smithi071 ceph-mon[29845]: from='osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T12:19:50.316 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:50 smithi071 ceph-mon[29845]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T12:19:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:50 smithi174 ceph-mon[34313]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:50 smithi174 ceph-mon[34313]: from='osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T12:19:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:50 smithi174 ceph-mon[34313]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T12:19:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:50 smithi123 ceph-mon[34321]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:50 smithi123 ceph-mon[34321]: from='osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T12:19:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:50 smithi123 ceph-mon[34321]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:51.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:51.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T12:19:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T12:19:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: osdmap e16: 3 total, 2 up, 3 in 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:51.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:19:51.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:51.938 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:19:51.969 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-09-05T12:19:52.214 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2[39237]: 2024-09-05T12:19:51.844+0000 7fab19f34640 -1 osd.2 0 waiting for initial osdmap 2024-09-05T12:19:52.214 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2[39237]: 2024-09-05T12:19:51.848+0000 7fab11d54640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:19:52.327 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:19:52.465 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi174 2024-09-05T12:19:52.465 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-05T12:19:52.465 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:52.466 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:19:52.466 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T12:19:52.466 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:52.466 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:52.466 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:52 smithi071 ceph-mon[29845]: from='osd.2 ' entity='osd.2' 2024-09-05T12:19:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi174 2024-09-05T12:19:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-05T12:19:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:19:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T12:19:52.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:52.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:52.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:52 smithi174 ceph-mon[34313]: from='osd.2 ' entity='osd.2' 2024-09-05T12:19:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi174 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: osdmap e17: 3 total, 2 up, 3 in 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:52 smithi123 ceph-mon[34321]: from='osd.2 ' entity='osd.2' 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 -- 172.21.15.71:0/1543481571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254105920 msgr2=0x7f325410c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 --2- 172.21.15.71:0/1543481571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254105920 0x7f325410c240 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f324c009ca0 tx=0x7f324c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 -- 172.21.15.71:0/1543481571 shutdown_connections 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 --2- 172.21.15.71:0/1543481571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254105920 0x7f325410c240 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 --2- 172.21.15.71:0/1543481571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3254104f60 0x7f32541053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 --2- 172.21.15.71:0/1543481571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254103e00 0x7f3254104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 -- 172.21.15.71:0/1543481571 >> 172.21.15.71:0/1543481571 conn(0x7f32540ff5b0 msgr2=0x7f32541019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:19:53.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 -- 172.21.15.71:0/1543481571 shutdown_connections 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.974+0000 7f3259a00640 1 -- 172.21.15.71:0/1543481571 wait complete. 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.975+0000 7f3259a00640 1 Processor -- start 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.975+0000 7f3259a00640 1 -- start start 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3259a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3254103e00 0x7f325419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3259a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254104f60 0x7f325419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3259a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254105920 0x7f32541a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3259a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f325410fab0 con 0x7f3254104f60 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3259a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f325410f930 con 0x7f3254103e00 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3259a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f325410fc30 con 0x7f3254105920 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f324be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254104f60 0x7f325419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3253400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254105920 0x7f32541a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:53.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f324be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254104f60 0x7f325419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34994/0 (socket says 172.21.15.71:34994) 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f324be00640 1 -- 172.21.15.71:0/2532047605 learned_addr learned my addr 172.21.15.71:0/2532047605 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3252a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3254103e00 0x7f325419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3253400640 1 -- 172.21.15.71:0/2532047605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3254103e00 msgr2=0x7f325419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3253400640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3254103e00 0x7f325419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3253400640 1 -- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254104f60 msgr2=0x7f325419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3253400640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254104f60 0x7f325419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.976+0000 7f3253400640 1 -- 172.21.15.71:0/2532047605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32541a1ca0 con 0x7f3254105920 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.977+0000 7f3253400640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254105920 0x7f32541a1630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f324c031ae0 tx=0x7f324c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.977+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c003b70 con 0x7f3254105920 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.977+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f324c003d10 con 0x7f3254105920 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.977+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32541a1f30 con 0x7f3254105920 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.977+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c02fc40 con 0x7f3254105920 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.977+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32541a23e0 con 0x7f3254105920 2024-09-05T12:19:53.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.978+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325410b120 con 0x7f3254105920 2024-09-05T12:19:53.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.980+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f324c004050 con 0x7f3254105920 2024-09-05T12:19:53.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.981+0000 7f324b400640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f32380a2b20 0x7f32380a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:19:53.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.981+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7f324c0eea70 con 0x7f3254105920 2024-09-05T12:19:53.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.981+0000 7f3252a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f32380a2b20 0x7f32380a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:19:53.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.982+0000 7f3252a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f32380a2b20 0x7f32380a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3244009bd0 tx=0x7f3244005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:19:53.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:52.985+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f324c0bcf60 con 0x7f3254105920 2024-09-05T12:19:53.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:53.176+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3254062630 con 0x7f32380a2b20 2024-09-05T12:19:53.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:53 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:19:53.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:53 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:19:53.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:53.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:53 smithi071 ceph-mon[29845]: osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] boot 2024-09-05T12:19:53.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:53 smithi071 ceph-mon[29845]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T12:19:53.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:53 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:19:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:53 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:19:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:53 smithi123 ceph-mon[34321]: osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] boot 2024-09-05T12:19:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:53 smithi123 ceph-mon[34321]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T12:19:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:53 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:19:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:53 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:19:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:53 smithi174 ceph-mon[34313]: osd.2 [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] boot 2024-09-05T12:19:54.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:53 smithi174 ceph-mon[34313]: osdmap e18: 3 total, 3 up, 3 in 2024-09-05T12:19:54.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:19:54.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:54.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:19:54.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:54.926 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:54.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:54.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:54.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T12:19:54.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T12:19:54.927 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:19:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T12:19:54.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:19:55.001 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: osdmap e19: 3 total, 3 up, 3 in 2024-09-05T12:19:55.002 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:19:55.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:19:55.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:55 smithi071 ceph-mon[29845]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T12:19:55.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:55 smithi071 ceph-mon[29845]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:19:56.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:19:56.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:55 smithi123 ceph-mon[34321]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T12:19:56.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:55 smithi123 ceph-mon[34321]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:19:56.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:19:56.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:55 smithi174 ceph-mon[34313]: osdmap e20: 3 total, 3 up, 3 in 2024-09-05T12:19:56.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:55 smithi174 ceph-mon[34313]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:19:56.323 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:55 smithi071 sudo[48534]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T12:19:56.323 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:55 smithi071 sudo[48534]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:56.324 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:55 smithi071 sudo[48534]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:56.324 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:56 smithi071 sudo[48534]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:56.660 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:56 smithi071 sudo[48579]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T12:19:56.661 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:56 smithi071 sudo[48579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:56.661 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:56 smithi071 sudo[48579]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:56.661 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:19:56 smithi071 sudo[48579]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:57.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:56 smithi123 ceph-mon[34321]: osdmap e21: 3 total, 3 up, 3 in 2024-09-05T12:19:57.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:56 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:57.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2973515846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b33ca22-8dc7-41ca-8cd6-44d700c3d3da"}]: dispatch 2024-09-05T12:19:57.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2973515846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b33ca22-8dc7-41ca-8cd6-44d700c3d3da"}]': finished 2024-09-05T12:19:57.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:56 smithi123 ceph-mon[34321]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T12:19:57.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:56 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:19:57.102 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:56 smithi123 sudo[42901]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T12:19:57.102 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:56 smithi123 sudo[42901]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:57.103 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:56 smithi123 sudo[42901]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:57.103 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:56 smithi123 sudo[42901]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:56 smithi071 ceph-mon[29845]: osdmap e21: 3 total, 3 up, 3 in 2024-09-05T12:19:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:56 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2973515846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b33ca22-8dc7-41ca-8cd6-44d700c3d3da"}]: dispatch 2024-09-05T12:19:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2973515846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b33ca22-8dc7-41ca-8cd6-44d700c3d3da"}]': finished 2024-09-05T12:19:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:56 smithi071 ceph-mon[29845]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T12:19:57.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:56 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:19:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:56 smithi174 ceph-mon[34313]: osdmap e21: 3 total, 3 up, 3 in 2024-09-05T12:19:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:56 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2973515846' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b33ca22-8dc7-41ca-8cd6-44d700c3d3da"}]: dispatch 2024-09-05T12:19:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2973515846' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b33ca22-8dc7-41ca-8cd6-44d700c3d3da"}]': finished 2024-09-05T12:19:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:56 smithi174 ceph-mon[34313]: osdmap e22: 4 total, 3 up, 4 in 2024-09-05T12:19:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:56 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:19:57.429 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:57 smithi123 sudo[42906]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T12:19:57.429 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:57 smithi123 sudo[42906]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:57.430 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:57 smithi123 sudo[42906]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:57.430 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:19:57 smithi123 sudo[42906]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:57.636 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T12:19:57.636 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:57.636 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42859]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:57.636 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42859]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:58.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42862]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T12:19:58.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42862]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:58.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42862]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:58.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:19:57 smithi174 sudo[42862]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:58.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[48663]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T12:19:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[48663]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[48663]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[48663]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[49038]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[49038]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[49038]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 sudo[49038]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 ceph-mon[29845]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/269613196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:59.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:58 smithi071 ceph-mon[29845]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:19:59.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:58 smithi123 ceph-mon[34321]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:19:59.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:59.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:58 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/269613196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:59.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:59.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:58 smithi123 ceph-mon[34321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:19:59.276 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42865]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T12:19:59.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42865]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:59.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42865]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:59.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42865]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:59.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:19:59.285+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f324c0b8ed0 con 0x7f3254105920 2024-09-05T12:19:59.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42868]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T12:19:59.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42868]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:59.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42868]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:59.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 sudo[42868]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:19:59.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:19:59.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:59.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/269613196' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:19:59.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:19:59.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:19:59.772 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42909]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-05T12:19:59.772 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42909]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:19:59.772 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42909]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:19:59.772 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42909]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:19:59 smithi071 ceph-mon[29845]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42912]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42912]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42912]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 sudo[42912]: pam_unix(sudo:session): session closed for user root 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:19:59 smithi123 ceph-mon[34321]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:20:00.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:19:59 smithi174 ceph-mon[34313]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-05T12:20:01.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:00 smithi071 ceph-mon[29845]: overall HEALTH_OK 2024-09-05T12:20:01.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:00 smithi123 ceph-mon[34321]: overall HEALTH_OK 2024-09-05T12:20:01.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:00 smithi174 ceph-mon[34313]: overall HEALTH_OK 2024-09-05T12:20:01.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:01 smithi071 ceph-mon[29845]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:02.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:01 smithi123 ceph-mon[34321]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:01 smithi174 ceph-mon[34313]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:04 smithi071 ceph-mon[29845]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:04.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:04 smithi123 ceph-mon[34321]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:04 smithi174 ceph-mon[34313]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:06.372 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:06 smithi071 ceph-mon[29845]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-05T12:20:06.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:06 smithi123 ceph-mon[34321]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-05T12:20:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:06 smithi174 ceph-mon[34313]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-05T12:20:07.331 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T12:20:07.331 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T12:20:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-05T12:20:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:08.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:08 smithi071 ceph-mon[29845]: Deploying daemon osd.3 on smithi071 2024-09-05T12:20:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:08 smithi071 ceph-mon[29845]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-05T12:20:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:08 smithi123 ceph-mon[34321]: Deploying daemon osd.3 on smithi071 2024-09-05T12:20:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:08 smithi123 ceph-mon[34321]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-05T12:20:08.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:08 smithi174 ceph-mon[34313]: Deploying daemon osd.3 on smithi071 2024-09-05T12:20:08.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:08 smithi174 ceph-mon[34313]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-05T12:20:10.247 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:10 smithi071 ceph-mon[29845]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-05T12:20:10.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:10 smithi123 ceph-mon[34321]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-05T12:20:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:10 smithi174 ceph-mon[34313]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-05T12:20:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:12 smithi071 ceph-mon[29845]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-05T12:20:12.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:12 smithi123 ceph-mon[34321]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-05T12:20:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:12 smithi174 ceph-mon[34313]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-05T12:20:14.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:14 smithi071 ceph-mon[29845]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-05T12:20:14.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:14.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:14.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:14.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:14 smithi123 ceph-mon[34321]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-05T12:20:14.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:14.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:14.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:14 smithi174 ceph-mon[34313]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-05T12:20:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:16.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:16 smithi071 ceph-mon[29845]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-05T12:20:16.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:16 smithi123 ceph-mon[34321]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-05T12:20:16.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:16 smithi174 ceph-mon[34313]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-05T12:20:17.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:17.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:17.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.734 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:17 smithi071 ceph-mon[29845]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:17.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:17.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:17.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:17 smithi174 ceph-mon[34313]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:17.770 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.770 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.770 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:17.770 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:17.770 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:17.770 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:17 smithi123 ceph-mon[34321]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:18.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.705+0000 7f324b400640 1 -- 172.21.15.71:0/2532047605 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3254062630 con 0x7f32380a2b20 2024-09-05T12:20:18.707 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 3 on host 'smithi071' 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f32380a2b20 msgr2=0x7f32380a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f32380a2b20 0x7f32380a4fe0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3244009bd0 tx=0x7f3244005f90 comp rx=0 tx=0).stop 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254105920 msgr2=0x7f32541a1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254105920 0x7f32541a1630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f324c031ae0 tx=0x7f324c004ab0 comp rx=0 tx=0).stop 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 shutdown_connections 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f32380a2b20 0x7f32380a4fe0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:18.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254105920 0x7f32541a1630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:18.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3254104f60 0x7f325419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:18.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 --2- 172.21.15.71:0/2532047605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3254103e00 0x7f325419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:18.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 >> 172.21.15.71:0/2532047605 conn(0x7f32540ff5b0 msgr2=0x7f32541082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:20:18.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 shutdown_connections 2024-09-05T12:20:18.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:20:18.707+0000 7f3259a00640 1 -- 172.21.15.71:0/2532047605 wait complete. 2024-09-05T12:20:18.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:18 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:18.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:18 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:18.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:18 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:18 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:18 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:18 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:19.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:18 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:19.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:18 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:19.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:18 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:19.525 DEBUG:teuthology.orchestra.run.smithi071:osd.3> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.3.service 2024-09-05T12:20:19.529 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-09-05T12:20:19.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T12:20:19.782 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:20:19.902 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:19 smithi123 ceph-mon[34321]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:19.902 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:19 smithi123 ceph-mon[34321]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T12:20:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:19 smithi174 ceph-mon[34313]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:19 smithi174 ceph-mon[34313]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T12:20:20.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:19 smithi071 ceph-mon[29845]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:20.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:19 smithi071 ceph-mon[29845]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-05T12:20:20.989 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:20 smithi071 ceph-mon[29845]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T12:20:20.989 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:20 smithi071 ceph-mon[29845]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T12:20:20.989 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:20 smithi071 ceph-mon[29845]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:20:20.989 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:20 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:20.990 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:20:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:20:20.780+0000 7fdd40a00640 -1 osd.3 0 waiting for initial osdmap 2024-09-05T12:20:20.990 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:20:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:20:20.787+0000 7fdd3ba00640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:20:21.058 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:20 smithi123 ceph-mon[34321]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T12:20:21.058 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:20 smithi123 ceph-mon[34321]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T12:20:21.058 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:20 smithi123 ceph-mon[34321]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:20:21.059 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:20 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:21.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:20 smithi174 ceph-mon[34313]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-05T12:20:21.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:20 smithi174 ceph-mon[34313]: osdmap e23: 4 total, 3 up, 4 in 2024-09-05T12:20:21.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:20 smithi174 ceph-mon[34313]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:20:21.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:20 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:21.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:21 smithi071 ceph-mon[29845]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:20:21.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:21 smithi071 ceph-mon[29845]: osdmap e24: 4 total, 3 up, 4 in 2024-09-05T12:20:21.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:21 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:21.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:21 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:21.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:21 smithi071 ceph-mon[29845]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:21.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:21 smithi123 ceph-mon[34321]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:20:21.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:21 smithi123 ceph-mon[34321]: osdmap e24: 4 total, 3 up, 4 in 2024-09-05T12:20:21.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:21 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:21.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:21 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:21.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:21 smithi123 ceph-mon[34321]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:22.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:21 smithi174 ceph-mon[34313]: from='osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:20:22.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:21 smithi174 ceph-mon[34313]: osdmap e24: 4 total, 3 up, 4 in 2024-09-05T12:20:22.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:21 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:22.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:21 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:22.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:21 smithi174 ceph-mon[34313]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-09-05T12:20:22.305 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:20:22.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-09-05T12:20:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:20:23.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:22 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:20:23.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:22 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:20:23.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:23.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:22 smithi123 ceph-mon[34321]: osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] boot 2024-09-05T12:20:23.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:22 smithi123 ceph-mon[34321]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T12:20:23.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 -- 172.21.15.123:0/1838073474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c101d10 msgr2=0x7fbb0c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/1838073474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c101d10 0x7fbb0c102190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009900 tx=0x7fbafc02a410 comp rx=0 tx=0).stop 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 -- 172.21.15.123:0/1838073474 shutdown_connections 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/1838073474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 0x7fbb0c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/1838073474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c101d10 0x7fbb0c102190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/1838073474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb0c100b10 0x7fbb0c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 -- 172.21.15.123:0/1838073474 >> 172.21.15.123:0/1838073474 conn(0x7fbb0c0fc2a0 msgr2=0x7fbb0c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 -- 172.21.15.123:0/1838073474 shutdown_connections 2024-09-05T12:20:23.013 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.011+0000 7fbb13a1d640 1 -- 172.21.15.123:0/1838073474 wait complete. 2024-09-05T12:20:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 Processor -- start 2024-09-05T12:20:23.014 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 -- start start 2024-09-05T12:20:23.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 0x7fbb0c196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:23.015 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb0c101d10 0x7fbb0c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 0x7fbb0c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10c890 con 0x7fbb0c1026d0 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10c710 con 0x7fbb0c100b10 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb13a1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c10ca10 con 0x7fbb0c101d10 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 0x7fbb0c196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 0x7fbb0c196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:55650/0 (socket says 172.21.15.123:55650) 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 -- 172.21.15.123:0/2112824998 learned_addr learned my addr 172.21.15.123:0/2112824998 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb11fba640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 0x7fbb0c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb10fb8640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb0c101d10 0x7fbb0c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 -- 172.21.15.123:0/2112824998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb0c101d10 msgr2=0x7fbb0c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb0c101d10 0x7fbb0c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:23.016 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 -- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 msgr2=0x7fbb0c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.012+0000 7fbb117b9640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 0x7fbb0c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb117b9640 1 -- 172.21.15.123:0/2112824998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c19ebd0 con 0x7fbb0c100b10 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb11fba640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 0x7fbb0c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb117b9640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 0x7fbb0c196fa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d920 tx=0x7fbb0800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0801b070 con 0x7fbb0c100b10 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c06a7f0 con 0x7fbb0c100b10 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb0800edc0 con 0x7fbb0c100b10 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08014800 con 0x7fbb0c100b10 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.013+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c06ad80 con 0x7fbb0c100b10 2024-09-05T12:20:23.017 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.014+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb0c1079f0 con 0x7fbb0c100b10 2024-09-05T12:20:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.015+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbb0801e070 con 0x7fbb0c100b10 2024-09-05T12:20:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.016+0000 7fbb027fc640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbaf80a2bb0 0x7fbaf80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.016+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3698+0+0 (secure 0 0 0) 0x7fbb080cf5c0 con 0x7fbb0c100b10 2024-09-05T12:20:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.016+0000 7fbb10fb8640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbaf80a2bb0 0x7fbaf80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:23.018 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.017+0000 7fbb10fb8640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbaf80a2bb0 0x7fbaf80a5070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbafc000c00 tx=0x7fbafc031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:20:23.022 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.020+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb0809d720 con 0x7fbb0c100b10 2024-09-05T12:20:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:22 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:20:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:22 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:20:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:22 smithi071 ceph-mon[29845]: osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] boot 2024-09-05T12:20:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:22 smithi071 ceph-mon[29845]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T12:20:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:23.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:23.196+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbb0c06b040 con 0x7fbaf80a2bb0 2024-09-05T12:20:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:22 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:20:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:22 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:20:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:22 smithi174 ceph-mon[34313]: osd.3 [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] boot 2024-09-05T12:20:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:22 smithi174 ceph-mon[34313]: osdmap e25: 4 total, 4 up, 4 in 2024-09-05T12:20:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi071 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 8544M 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:24.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T12:20:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi071 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 8544M 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:24.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: osdmap e26: 4 total, 4 up, 4 in 2024-09-05T12:20:24.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 118 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:24.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:20:24.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:20:24.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi071 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 8544M 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:24.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:24.888 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:24 smithi123 ceph-mon[34321]: osdmap e27: 4 total, 4 up, 4 in 2024-09-05T12:20:25.055 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:24 smithi174 ceph-mon[34313]: osdmap e27: 4 total, 4 up, 4 in 2024-09-05T12:20:25.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:24 smithi071 ceph-mon[29845]: osdmap e27: 4 total, 4 up, 4 in 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: osdmap e28: 4 total, 4 up, 4 in 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/1493972582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]: dispatch 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]: dispatch 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]': finished 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T12:20:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: osdmap e28: 4 total, 4 up, 4 in 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/1493972582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]: dispatch 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]: dispatch 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]': finished 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T12:20:26.118 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: osdmap e28: 4 total, 4 up, 4 in 2024-09-05T12:20:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:20:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/1493972582' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]: dispatch 2024-09-05T12:20:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]: dispatch 2024-09-05T12:20:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f983652a-0773-4d6d-9cac-5a0f4602a729"}]': finished 2024-09-05T12:20:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: osdmap e29: 5 total, 4 up, 5 in 2024-09-05T12:20:26.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:27.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:26 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/1809883127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:20:27.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:26 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/1809883127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:20:27.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:26 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/1809883127' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:20:28.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:27 smithi071 ceph-mon[29845]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-09-05T12:20:28.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:27 smithi123 ceph-mon[34321]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-09-05T12:20:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:27 smithi174 ceph-mon[34313]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 134 KiB/s, 0 objects/s recovering 2024-09-05T12:20:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:30 smithi071 ceph-mon[29845]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T12:20:30.424 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:30 smithi123 ceph-mon[34321]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T12:20:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:30 smithi174 ceph-mon[34313]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-05T12:20:32.407 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:32 smithi123 ceph-mon[34321]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T12:20:32.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:32 smithi071 ceph-mon[29845]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T12:20:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:32 smithi174 ceph-mon[34313]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-05T12:20:34.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:34 smithi071 ceph-mon[29845]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:34.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:34 smithi123 ceph-mon[34321]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:34 smithi174 ceph-mon[34313]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:35.290 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T12:20:35.290 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:35.290 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:35 smithi123 ceph-mon[34321]: Deploying daemon osd.4 on smithi123 2024-09-05T12:20:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T12:20:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:35 smithi071 ceph-mon[29845]: Deploying daemon osd.4 on smithi123 2024-09-05T12:20:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-05T12:20:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:35 smithi174 ceph-mon[34313]: Deploying daemon osd.4 on smithi123 2024-09-05T12:20:36.387 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:36 smithi123 ceph-mon[34321]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:36.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:36 smithi071 ceph-mon[29845]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:36.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:36 smithi174 ceph-mon[34313]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:38.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:38 smithi071 ceph-mon[29845]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:38.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:38 smithi123 ceph-mon[34321]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:38 smithi174 ceph-mon[34313]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:40.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:40 smithi123 ceph-mon[34321]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:40.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:40.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:40 smithi071 ceph-mon[29845]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:40 smithi174 ceph-mon[34313]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:42 smithi123 ceph-mon[34321]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:42 smithi174 ceph-mon[34313]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:42 smithi071 ceph-mon[29845]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.511 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.509+0000 7fbb027fc640 1 -- 172.21.15.123:0/2112824998 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbb0c06b040 con 0x7fbaf80a2bb0 2024-09-05T12:20:43.511 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-09-05T12:20:43.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbaf80a2bb0 msgr2=0x7fbaf80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:43.513 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbaf80a2bb0 0x7fbaf80a5070 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbafc000c00 tx=0x7fbafc031400 comp rx=0 tx=0).stop 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 msgr2=0x7fbb0c196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 0x7fbb0c196fa0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d920 tx=0x7fbb0800dde0 comp rx=0 tx=0).stop 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 shutdown_connections 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbaf80a2bb0 0x7fbaf80a5070 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb0c1026d0 0x7fbb0c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb0c101d10 0x7fbb0c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 --2- 172.21.15.123:0/2112824998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb0c100b10 0x7fbb0c196fa0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 >> 172.21.15.123:0/2112824998 conn(0x7fbb0c0fc2a0 msgr2=0x7fbb0c105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 shutdown_connections 2024-09-05T12:20:43.514 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:20:43.512+0000 7fbb13a1d640 1 -- 172.21.15.123:0/2112824998 wait complete. 2024-09-05T12:20:43.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:44.172 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.4.service 2024-09-05T12:20:44.175 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-09-05T12:20:44.175 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-05T12:20:44.428 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:20:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:44 smithi123 ceph-mon[34321]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:44 smithi174 ceph-mon[34313]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:44 smithi071 ceph-mon[29845]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:20:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:45.667 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:20:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4[45623]: 2024-09-05T12:20:45.332+0000 7f90129c5640 -1 osd.4 0 log_to_monitors true 2024-09-05T12:20:46.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:46 smithi071 ceph-mon[29845]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:46.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:46 smithi071 ceph-mon[29845]: from='osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T12:20:46.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:46 smithi071 ceph-mon[29845]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T12:20:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:46 smithi174 ceph-mon[34313]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:46 smithi174 ceph-mon[34313]: from='osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T12:20:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:46 smithi174 ceph-mon[34313]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T12:20:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:46 smithi123 ceph-mon[34321]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:46 smithi123 ceph-mon[34321]: from='osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T12:20:46.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:46 smithi123 ceph-mon[34321]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-05T12:20:47.063 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:20:47.089 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-09-05T12:20:47.335 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: osdmap e31: 5 total, 4 up, 5 in 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.429 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4[45623]: 2024-09-05T12:20:47.198+0000 7f900e948640 -1 osd.4 0 waiting for initial osdmap 2024-09-05T12:20:47.429 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4[45623]: 2024-09-05T12:20:47.203+0000 7f900a770640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: osdmap e31: 5 total, 4 up, 5 in 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.455 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-05T12:20:47.455 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:20:47.455 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: osdmap e30: 5 total, 4 up, 5 in 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: osdmap e31: 5 total, 4 up, 5 in 2024-09-05T12:20:47.456 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.457 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 -- 172.21.15.174:0/501079761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58101cd0 msgr2=0x7f4a58102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 --2- 172.21.15.174:0/501079761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58101cd0 0x7f4a58102150 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009900 tx=0x7f4a5402a410 comp rx=0 tx=0).stop 2024-09-05T12:20:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 -- 172.21.15.174:0/501079761 shutdown_connections 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 --2- 172.21.15.174:0/501079761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a58102690 0x7f4a58108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 --2- 172.21.15.174:0/501079761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58101cd0 0x7f4a58102150 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 --2- 172.21.15.174:0/501079761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a58100ad0 0x7f4a58100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 -- 172.21.15.174:0/501079761 >> 172.21.15.174:0/501079761 conn(0x7f4a580fc280 msgr2=0x7f4a580fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 -- 172.21.15.174:0/501079761 shutdown_connections 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.778+0000 7f4a603ee640 1 -- 172.21.15.174:0/501079761 wait complete. 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 Processor -- start 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 -- start start 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 0x7f4a58197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a58101cd0 0x7f4a58198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a58102690 0x7f4a5819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:47.797 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5810d880 con 0x7f4a58102690 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5810d700 con 0x7f4a58101cd0 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a603ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5810da00 con 0x7f4a58100ad0 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 0x7f4a58197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 0x7f4a58197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:45458/0 (socket says 172.21.15.174:45458) 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 -- 172.21.15.174:0/3602061808 learned_addr learned my addr 172.21.15.174:0/3602061808 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5d989640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a58101cd0 0x7f4a58198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e98b640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a58102690 0x7f4a5819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 -- 172.21.15.174:0/3602061808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a58101cd0 msgr2=0x7f4a58198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a58101cd0 0x7f4a58198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 -- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a58102690 msgr2=0x7f4a5819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a58102690 0x7f4a5819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 -- 172.21.15.174:0/3602061808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5819faf0 con 0x7f4a58100ad0 2024-09-05T12:20:47.798 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.779+0000 7f4a5e18a640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 0x7f4a58197ec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800d770 tx=0x7f4a4800dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.780+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48017070 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.780+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a48009c30 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.780+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5819fde0 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.780+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48003dd0 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.780+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a0230 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.781+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a58107ea0 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.782+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4a48004000 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.782+0000 7f4a4f7fe640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4a440a2bb0 0x7f4a440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.783+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f4a480cd960 con 0x7f4a58100ad0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.783+0000 7f4a5d989640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4a440a2bb0 0x7f4a440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.783+0000 7f4a5d989640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4a440a2bb0 0x7f4a440a5070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000c00 tx=0x7f4a540315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:20:47.799 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.787+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a4809b9b0 con 0x7f4a58100ad0 2024-09-05T12:20:47.961 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:20:47.960+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4a58062630 con 0x7f4a440a2bb0 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi123 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:48.446 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi123 2024-09-05T12:20:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-05T12:20:48.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:48.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:20:48.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:20:48.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:48.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi123 2024-09-05T12:20:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-05T12:20:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:20:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:20:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:49.302 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:49 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:20:49.302 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:49 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:20:49.302 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:49 smithi174 ceph-mon[34313]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:20:49.303 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:49 smithi174 ceph-mon[34313]: osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] boot 2024-09-05T12:20:49.303 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:49 smithi174 ceph-mon[34313]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T12:20:49.303 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:49.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:49 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:20:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:49 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:20:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:49 smithi071 ceph-mon[29845]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:20:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:49 smithi071 ceph-mon[29845]: osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] boot 2024-09-05T12:20:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:49 smithi071 ceph-mon[29845]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T12:20:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:49.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:49 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:20:49.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:49 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:20:49.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:49 smithi123 ceph-mon[34321]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:20:49.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:49 smithi123 ceph-mon[34321]: osd.4 [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] boot 2024-09-05T12:20:49.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:49 smithi123 ceph-mon[34321]: osdmap e32: 5 total, 5 up, 5 in 2024-09-05T12:20:49.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:20:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: osdmap e33: 5 total, 5 up, 5 in 2024-09-05T12:20:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/3902730528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]: dispatch 2024-09-05T12:20:50.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]: dispatch 2024-09-05T12:20:50.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]': finished 2024-09-05T12:20:50.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T12:20:50.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:50 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: osdmap e33: 5 total, 5 up, 5 in 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/3902730528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]: dispatch 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]: dispatch 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]': finished 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T12:20:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:50 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:20:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-05T12:20:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: osdmap e33: 5 total, 5 up, 5 in 2024-09-05T12:20:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/3902730528' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]: dispatch 2024-09-05T12:20:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]: dispatch 2024-09-05T12:20:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "379b7d31-7108-49f2-a223-93c1214c9460"}]': finished 2024-09-05T12:20:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: osdmap e34: 6 total, 5 up, 6 in 2024-09-05T12:20:50.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:50 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:20:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:51 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/1263054587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:20:51.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:51 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/1263054587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:20:51.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:51 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/1263054587' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:20:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:52 smithi174 ceph-mon[34313]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:52 smithi071 ceph-mon[29845]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:52 smithi123 ceph-mon[34321]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:54 smithi174 ceph-mon[34313]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:54.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:54 smithi071 ceph-mon[29845]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:54 smithi123 ceph-mon[34321]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:56.282 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:56 smithi174 ceph-mon[34313]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:56.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:56 smithi071 ceph-mon[29845]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:56 smithi123 ceph-mon[34321]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:58.348 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:58 smithi174 ceph-mon[34313]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:58.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:58 smithi071 ceph-mon[29845]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:58 smithi123 ceph-mon[34321]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:20:59.443 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T12:20:59.443 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:20:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T12:20:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:20:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:20:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-05T12:20:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:20:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:00.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:00 smithi174 ceph-mon[34313]: Deploying daemon osd.5 on smithi174 2024-09-05T12:21:00.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:00 smithi174 ceph-mon[34313]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:00 smithi071 ceph-mon[29845]: Deploying daemon osd.5 on smithi174 2024-09-05T12:21:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:00 smithi071 ceph-mon[29845]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:00 smithi123 ceph-mon[34321]: Deploying daemon osd.5 on smithi174 2024-09-05T12:21:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:00 smithi123 ceph-mon[34321]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:02.291 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:02 smithi174 ceph-mon[34313]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:02 smithi071 ceph-mon[29845]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:02 smithi123 ceph-mon[34321]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:04 smithi174 ceph-mon[34313]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:04.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:04 smithi071 ceph-mon[29845]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:04 smithi123 ceph-mon[34321]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:06.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:06 smithi174 ceph-mon[34313]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:06 smithi071 ceph-mon[29845]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:06 smithi123 ceph-mon[34321]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:08.073 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.073 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:08.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:08.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.074 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:07 smithi174 ceph-mon[34313]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:08.116 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.115+0000 7f4a4f7fe640 1 -- 172.21.15.174:0/3602061808 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4a58062630 con 0x7f4a440a2bb0 2024-09-05T12:21:08.116 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-09-05T12:21:08.118 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4a440a2bb0 msgr2=0x7f4a440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:08.118 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4a440a2bb0 0x7f4a440a5070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4a54000c00 tx=0x7f4a540315d0 comp rx=0 tx=0).stop 2024-09-05T12:21:08.118 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 msgr2=0x7f4a58197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 0x7f4a58197ec0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800d770 tx=0x7f4a4800dc30 comp rx=0 tx=0).stop 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 shutdown_connections 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4a440a2bb0 0x7f4a440a5070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a58102690 0x7f4a5819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a58101cd0 0x7f4a58198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 --2- 172.21.15.174:0/3602061808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a58100ad0 0x7f4a58197ec0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 >> 172.21.15.174:0/3602061808 conn(0x7f4a580fc280 msgr2=0x7f4a58105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 shutdown_connections 2024-09-05T12:21:08.119 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:21:08.117+0000 7f4a603ee640 1 -- 172.21.15.174:0/3602061808 wait complete. 2024-09-05T12:21:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:07 smithi071 ceph-mon[29845]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:08.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:08.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:08.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:07 smithi123 ceph-mon[34321]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:08.704 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.5.service 2024-09-05T12:21:08.707 INFO:tasks.cephadm:Deploying osd.6 on smithi071 with /dev/vg_nvme/lv_2... 2024-09-05T12:21:08.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T12:21:08.960 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:08.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:08.961 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:09.076 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:21:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:09.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:09.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:09.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:09.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:09 smithi071 ceph-mon[29845]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:09.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:09 smithi071 ceph-mon[29845]: from='osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T12:21:09.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:09 smithi071 ceph-mon[29845]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T12:21:09.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:09 smithi174 ceph-mon[34313]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:09.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:09 smithi174 ceph-mon[34313]: from='osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T12:21:09.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:09 smithi174 ceph-mon[34313]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T12:21:09.943 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:21:09 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:21:09.598+0000 7f5643144640 -1 osd.5 0 log_to_monitors true 2024-09-05T12:21:10.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:09 smithi123 ceph-mon[34321]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:10.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:09 smithi123 ceph-mon[34321]: from='osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T12:21:10.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:09 smithi123 ceph-mon[34321]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-05T12:21:10.872 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:10 smithi071 ceph-mon[29845]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T12:21:10.872 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:10 smithi071 ceph-mon[29845]: from='osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:21:10.872 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:10 smithi071 ceph-mon[29845]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T12:21:10.872 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:10.872 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:10 smithi071 ceph-mon[29845]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:21:11.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:10 smithi123 ceph-mon[34321]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T12:21:11.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:10 smithi123 ceph-mon[34321]: from='osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:21:11.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:10 smithi123 ceph-mon[34321]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T12:21:11.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:11.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:10 smithi123 ceph-mon[34321]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:21:11.198 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-mon[34313]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-05T12:21:11.198 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-mon[34313]: from='osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:21:11.198 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-mon[34313]: osdmap e35: 6 total, 5 up, 6 in 2024-09-05T12:21:11.199 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:11.199 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-mon[34313]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:21:11.199 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:21:10.831+0000 7f563f0c7640 -1 osd.5 0 waiting for initial osdmap 2024-09-05T12:21:11.199 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:21:10 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:21:10.838+0000 7f563aeef640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: osdmap e36: 6 total, 5 up, 6 in 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] boot 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T12:21:12.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:21:12.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: osdmap e36: 6 total, 5 up, 6 in 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] boot 2024-09-05T12:21:12.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T12:21:12.181 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:21:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: osdmap e36: 6 total, 5 up, 6 in 2024-09-05T12:21:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: osd.5 [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] boot 2024-09-05T12:21:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: osdmap e37: 6 total, 6 up, 6 in 2024-09-05T12:21:12.262 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:21:12.579 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:21:12.611 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-09-05T12:21:12.976 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:21:13.115 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:12 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:21:13.115 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:12 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:21:13.115 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:12 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi174 2024-09-05T12:21:13.116 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:12 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-05T12:21:13.116 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:12 smithi071 ceph-mon[29845]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T12:21:13.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:12 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:21:13.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:12 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:21:13.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:12 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi174 2024-09-05T12:21:13.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:12 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-05T12:21:13.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:12 smithi123 ceph-mon[34321]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T12:21:13.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:12 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:21:13.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:12 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:21:13.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:12 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi174 2024-09-05T12:21:13.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:12 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-05T12:21:13.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:12 smithi174 ceph-mon[34313]: osdmap e38: 6 total, 6 up, 6 in 2024-09-05T12:21:13.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 -- 172.21.15.71:0/307442426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110104f60 msgr2=0x7fa1101053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:13.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 --2- 172.21.15.71:0/307442426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110104f60 0x7fa1101053e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009900 tx=0x7fa0f802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:21:13.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 -- 172.21.15.71:0/307442426 shutdown_connections 2024-09-05T12:21:13.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 --2- 172.21.15.71:0/307442426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa110105920 0x7fa11010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 --2- 172.21.15.71:0/307442426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110104f60 0x7fa1101053e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 --2- 172.21.15.71:0/307442426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110103e00 0x7fa110104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 -- 172.21.15.71:0/307442426 >> 172.21.15.71:0/307442426 conn(0x7fa1100ff5b0 msgr2=0x7fa1101019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 -- 172.21.15.71:0/307442426 shutdown_connections 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.579+0000 7fa116c00640 1 -- 172.21.15.71:0/307442426 wait complete. 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.580+0000 7fa116c00640 1 Processor -- start 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.580+0000 7fa116c00640 1 -- start start 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa116c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 0x7fa11019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa116c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa110104f60 0x7fa11019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa116c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110105920 0x7fa1101a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa116c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010fb00 con 0x7fa110103e00 2024-09-05T12:21:13.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa116c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010f980 con 0x7fa110104f60 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa116c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa11010fc80 con 0x7fa110105920 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 0x7fa11019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 0x7fa11019a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50414/0 (socket says 172.21.15.71:50414) 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 -- 172.21.15.71:0/3288406217 learned_addr learned my addr 172.21.15.71:0/3288406217 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa114c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110105920 0x7fa1101a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa107e00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa110104f60 0x7fa11019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 -- 172.21.15.71:0/3288406217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110105920 msgr2=0x7fa1101a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110105920 0x7fa1101a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 -- 172.21.15.71:0/3288406217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa110104f60 msgr2=0x7fa11019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.581+0000 7fa10fe00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa110104f60 0x7fa11019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa10fe00640 1 -- 172.21.15.71:0/3288406217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1101a1d30 con 0x7fa110103e00 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa10fe00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 0x7fa11019a100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e9f0 tx=0x7fa10000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:21:13.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000cd80 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa100005500 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1101a1fc0 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100010690 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.582+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1101a2470 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.583+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11010b1c0 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.585+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa1000046c0 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.586+0000 7fa10d600640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa0f40a2c30 0x7fa0f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.586+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4562+0+0 (secure 0 0 0) 0x7fa1000ce580 con 0x7fa110103e00 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.586+0000 7fa107e00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa0f40a2c30 0x7fa0f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.587+0000 7fa107e00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa0f40a2c30 0x7fa0f40a50f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802a9e0 tx=0x7fa0f80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:21:13.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.591+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa10009c2e0 con 0x7fa110103e00 2024-09-05T12:21:13.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:13.775+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fa110062630 con 0x7fa0f40a2c30 2024-09-05T12:21:14.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:13 smithi071 ceph-mon[29845]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:14.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:13 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:21:14.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:13 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:21:14.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:13 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:14.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:13 smithi071 ceph-mon[29845]: osdmap e39: 6 total, 6 up, 6 in 2024-09-05T12:21:14.161 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:13 smithi174 ceph-mon[34313]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:14.162 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:13 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:21:14.162 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:13 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:21:14.162 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:13 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:14.162 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:13 smithi174 ceph-mon[34313]: osdmap e39: 6 total, 6 up, 6 in 2024-09-05T12:21:14.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:13 smithi123 ceph-mon[34321]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:13 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:21:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:13 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:21:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:13 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:13 smithi123 ceph-mon[34321]: osdmap e39: 6 total, 6 up, 6 in 2024-09-05T12:21:14.897 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:14 smithi071 ceph-mon[29845]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:21:15.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:14 smithi123 ceph-mon[34321]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:21:15.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:14 smithi174 ceph-mon[34313]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:21:15.944 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:15 smithi071 ceph-mon[29845]: osdmap e40: 6 total, 6 up, 6 in 2024-09-05T12:21:15.944 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:15 smithi071 ceph-mon[29845]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:21:16.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:15 smithi123 ceph-mon[34321]: osdmap e40: 6 total, 6 up, 6 in 2024-09-05T12:21:16.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:15 smithi123 ceph-mon[34321]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:21:16.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:15 smithi174 ceph-mon[34313]: osdmap e40: 6 total, 6 up, 6 in 2024-09-05T12:21:16.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:15 smithi174 ceph-mon[34313]: pgmap v141: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:21:17.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:16 smithi071 ceph-mon[29845]: osdmap e41: 6 total, 6 up, 6 in 2024-09-05T12:21:17.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:16 smithi123 ceph-mon[34321]: osdmap e41: 6 total, 6 up, 6 in 2024-09-05T12:21:17.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:16 smithi174 ceph-mon[34313]: osdmap e41: 6 total, 6 up, 6 in 2024-09-05T12:21:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:17 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3455885792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ac35b8a-e077-44a6-91ea-1e5efd56d8b8"}]: dispatch 2024-09-05T12:21:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:17 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3455885792' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ac35b8a-e077-44a6-91ea-1e5efd56d8b8"}]': finished 2024-09-05T12:21:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:17 smithi071 ceph-mon[29845]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T12:21:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:17 smithi071 ceph-mon[29845]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:21:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:17 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/379815747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:21:18.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:17 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3455885792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ac35b8a-e077-44a6-91ea-1e5efd56d8b8"}]: dispatch 2024-09-05T12:21:18.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:17 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3455885792' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ac35b8a-e077-44a6-91ea-1e5efd56d8b8"}]': finished 2024-09-05T12:21:18.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:17 smithi123 ceph-mon[34321]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T12:21:18.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:18.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:17 smithi123 ceph-mon[34321]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:21:18.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:17 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/379815747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:21:18.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:17 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3455885792' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6ac35b8a-e077-44a6-91ea-1e5efd56d8b8"}]: dispatch 2024-09-05T12:21:18.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:17 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3455885792' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6ac35b8a-e077-44a6-91ea-1e5efd56d8b8"}]': finished 2024-09-05T12:21:18.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:17 smithi174 ceph-mon[34313]: osdmap e42: 7 total, 6 up, 7 in 2024-09-05T12:21:18.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:18.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:17 smithi174 ceph-mon[34313]: pgmap v144: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-05T12:21:18.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:17 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/379815747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:21:20.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:20 smithi071 ceph-mon[29845]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-05T12:21:20.433 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:20 smithi123 ceph-mon[34321]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-05T12:21:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:20 smithi174 ceph-mon[34313]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-05T12:21:22.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:22 smithi071 ceph-mon[29845]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:22.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:22 smithi123 ceph-mon[34321]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:22 smithi174 ceph-mon[34313]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:24.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:24 smithi071 ceph-mon[29845]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:24.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:24 smithi123 ceph-mon[34321]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:24 smithi174 ceph-mon[34313]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:26 smithi071 ceph-mon[29845]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:26.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:26 smithi123 ceph-mon[34321]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:26 smithi174 ceph-mon[34313]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:27.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T12:21:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:27.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T12:21:27.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:27.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-05T12:21:27.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:28 smithi123 ceph-mon[34321]: Deploying daemon osd.6 on smithi071 2024-09-05T12:21:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:28 smithi123 ceph-mon[34321]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:28.468 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:28 smithi071 ceph-mon[29845]: Deploying daemon osd.6 on smithi071 2024-09-05T12:21:28.468 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:28 smithi071 ceph-mon[29845]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:28 smithi174 ceph-mon[34313]: Deploying daemon osd.6 on smithi071 2024-09-05T12:21:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:28 smithi174 ceph-mon[34313]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:30 smithi071 ceph-mon[29845]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:30 smithi123 ceph-mon[34321]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:30 smithi174 ceph-mon[34313]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:32.323 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:32 smithi071 ceph-mon[29845]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:32.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:32 smithi123 ceph-mon[34321]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:32 smithi174 ceph-mon[34313]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:34.361 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:34 smithi071 ceph-mon[29845]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:34.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:34 smithi123 ceph-mon[34321]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:34 smithi174 ceph-mon[34313]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:35.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:36.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:36 smithi071 ceph-mon[29845]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:36.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:36 smithi174 ceph-mon[34313]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:36 smithi123 ceph-mon[34321]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:38.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:38 smithi071 ceph-mon[29845]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:38.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:38.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:38.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:38 smithi174 ceph-mon[34313]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:38 smithi123 ceph-mon[34321]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:39.440 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 6 on host 'smithi071' 2024-09-05T12:21:39.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.439+0000 7fa10d600640 1 -- 172.21.15.71:0/3288406217 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa110062630 con 0x7fa0f40a2c30 2024-09-05T12:21:39.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa0f40a2c30 msgr2=0x7fa0f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:39.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa0f40a2c30 0x7fa0f40a50f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa0f802a9e0 tx=0x7fa0f80315b0 comp rx=0 tx=0).stop 2024-09-05T12:21:39.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 msgr2=0x7fa11019a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:39.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 0x7fa11019a100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e9f0 tx=0x7fa10000edf0 comp rx=0 tx=0).stop 2024-09-05T12:21:39.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 shutdown_connections 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa0f40a2c30 0x7fa0f40a50f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa110105920 0x7fa1101a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa110104f60 0x7fa11019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 --2- 172.21.15.71:0/3288406217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa110103e00 0x7fa11019a100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 >> 172.21.15.71:0/3288406217 conn(0x7fa1100ff5b0 msgr2=0x7fa110108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 shutdown_connections 2024-09-05T12:21:39.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:21:39.442+0000 7fa116c00640 1 -- 172.21.15.71:0/3288406217 wait complete. 2024-09-05T12:21:40.154 DEBUG:teuthology.orchestra.run.smithi071:osd.6> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.6.service 2024-09-05T12:21:40.159 INFO:tasks.cephadm:Deploying osd.7 on smithi123 with /dev/vg_nvme/lv_2... 2024-09-05T12:21:40.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T12:21:40.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:40 smithi071 ceph-mon[29845]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:40.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:40.411 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:21:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:40 smithi174 ceph-mon[34313]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:40.524 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:40 smithi123 ceph-mon[34321]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:40.524 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:21:40.524 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:40.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:40.730 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:21:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:21:40.527+0000 7f2b51611640 -1 osd.6 0 log_to_monitors true 2024-09-05T12:21:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:41 smithi174 ceph-mon[34313]: from='osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T12:21:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:41 smithi174 ceph-mon[34313]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T12:21:41.527 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:41 smithi123 ceph-mon[34321]: from='osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T12:21:41.527 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:41 smithi123 ceph-mon[34321]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T12:21:41.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:41 smithi071 ceph-mon[29845]: from='osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T12:21:41.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:41 smithi071 ceph-mon[29845]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-05T12:21:42.215 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-mon[29845]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:42.216 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-mon[29845]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T12:21:42.216 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-mon[29845]: osdmap e43: 7 total, 6 up, 7 in 2024-09-05T12:21:42.216 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:42.216 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-mon[29845]: from='osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:21:42.216 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-mon[29845]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:21:42.346 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:42 smithi123 ceph-mon[34321]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:42.346 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:42 smithi123 ceph-mon[34321]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T12:21:42.346 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:42 smithi123 ceph-mon[34321]: osdmap e43: 7 total, 6 up, 7 in 2024-09-05T12:21:42.347 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:42 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:42.347 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:42 smithi123 ceph-mon[34321]: from='osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:21:42.347 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:42 smithi123 ceph-mon[34321]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:21:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:42 smithi174 ceph-mon[34313]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:42 smithi174 ceph-mon[34313]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-05T12:21:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:42 smithi174 ceph-mon[34313]: osdmap e43: 7 total, 6 up, 7 in 2024-09-05T12:21:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:42 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:42 smithi174 ceph-mon[34313]: from='osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:21:42.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:42 smithi174 ceph-mon[34313]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:21:42.661 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:21:42.214+0000 7f2b4c200640 -1 osd.6 0 waiting for initial osdmap 2024-09-05T12:21:42.661 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:21:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:21:42.225+0000 7f2b47200640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:21:42.974 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:21:43.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-09-05T12:21:43.240 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:21:43.361 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:43 smithi123 ceph-mon[34321]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:21:43.361 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:43 smithi123 ceph-mon[34321]: osdmap e44: 7 total, 6 up, 7 in 2024-09-05T12:21:43.361 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:43.361 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:43 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:43 smithi174 ceph-mon[34313]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:21:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:43 smithi174 ceph-mon[34313]: osdmap e44: 7 total, 6 up, 7 in 2024-09-05T12:21:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:43 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:43 smithi071 ceph-mon[29845]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:21:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:43 smithi071 ceph-mon[29845]: osdmap e44: 7 total, 6 up, 7 in 2024-09-05T12:21:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:43.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:43 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:43.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 -- 172.21.15.123:0/772377132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100ad0 msgr2=0x7f2204100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:43.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 --2- 172.21.15.123:0/772377132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100ad0 0x7f2204100ed0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f21f4009900 tx=0x7f21f402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:21:43.699 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 -- 172.21.15.123:0/772377132 shutdown_connections 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 --2- 172.21.15.123:0/772377132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2204102690 0x7f2204108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 --2- 172.21.15.123:0/772377132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 0x7f2204102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 --2- 172.21.15.123:0/772377132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204100ad0 0x7f2204100ed0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 -- 172.21.15.123:0/772377132 >> 172.21.15.123:0/772377132 conn(0x7f22040fc280 msgr2=0x7f22040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 -- 172.21.15.123:0/772377132 shutdown_connections 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.681+0000 7f220a8e0640 1 -- 172.21.15.123:0/772377132 wait complete. 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 Processor -- start 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 -- start start 2024-09-05T12:21:43.700 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2204100ad0 0x7f220406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 0x7f220419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 0x7f220419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f220410c730 con 0x7f2204101cd0 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f220410c5b0 con 0x7f2204102690 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f220a8e0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f220410c8b0 con 0x7f2204100ad0 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f2208e7d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 0x7f220419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f2208e7d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 0x7f220419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:44916/0 (socket says 172.21.15.123:44916) 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.682+0000 7f2208e7d640 1 -- 172.21.15.123:0/1842683834 learned_addr learned my addr 172.21.15.123:0/1842683834 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2203fff640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2204100ad0 0x7f220406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f22037fe640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 0x7f220419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:43.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2208e7d640 1 -- 172.21.15.123:0/1842683834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2204100ad0 msgr2=0x7f220406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2208e7d640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2204100ad0 0x7f220406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2208e7d640 1 -- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 msgr2=0x7f220419cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2208e7d640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 0x7f220419cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2208e7d640 1 -- 172.21.15.123:0/1842683834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220419fa80 con 0x7f2204102690 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f22037fe640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 0x7f220419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f2208e7d640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 0x7f220419f170 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00cce0 tx=0x7f21fc00a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00f450 con 0x7f2204102690 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc00fad0 con 0x7f2204102690 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc018c10 con 0x7f2204102690 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220419fd10 con 0x7f2204102690 2024-09-05T12:21:43.702 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.683+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041a0250 con 0x7f2204102690 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.684+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2204107e00 con 0x7f2204102690 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.685+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f21fc00f5f0 con 0x7f2204102690 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.686+0000 7f22017fa640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21ec0a2c00 0x7f21ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.686+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7f21fc0ce9a0 con 0x7f2204102690 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.686+0000 7f2203fff640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21ec0a2c00 0x7f21ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.686+0000 7f2203fff640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21ec0a2c00 0x7f21ec0a50c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21f4005ab0 tx=0x7f21f4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:21:43.703 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.690+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21fc014030 con 0x7f2204102690 2024-09-05T12:21:43.864 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:21:43.862+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2204062630 con 0x7f21ec0a2c00 2024-09-05T12:21:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:21:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:21:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] boot 2024-09-05T12:21:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: osdmap e45: 7 total, 7 up, 7 in 2024-09-05T12:21:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:44.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:21:44.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:21:44.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:44.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:21:44.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:21:44.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:44.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] boot 2024-09-05T12:21:44.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: osdmap e45: 7 total, 7 up, 7 in 2024-09-05T12:21:44.536 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:44.537 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:21:44.537 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:21:44.537 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: osd.6 [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] boot 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: osdmap e45: 7 total, 7 up, 7 in 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:21:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:45.323 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: osdmap e46: 7 total, 7 up, 7 in 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:45.324 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:21:45.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: osdmap e46: 7 total, 7 up, 7 in 2024-09-05T12:21:45.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='client.24287 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: osdmap e46: 7 total, 7 up, 7 in 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:21:45.665 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:21:46.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:46 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi071 2024-09-05T12:21:46.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:46 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 5696M 2024-09-05T12:21:46.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:46 smithi123 ceph-mon[34321]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:46 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi071 2024-09-05T12:21:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:46 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 5696M 2024-09-05T12:21:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:46 smithi174 ceph-mon[34313]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:46.562 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:46 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi071 2024-09-05T12:21:46.562 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:46 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 5696M 2024-09-05T12:21:46.563 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:46 smithi071 ceph-mon[29845]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:47.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:47 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/1728532825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]: dispatch 2024-09-05T12:21:47.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:47 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]: dispatch 2024-09-05T12:21:47.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:47 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]': finished 2024-09-05T12:21:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:47 smithi123 ceph-mon[34321]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T12:21:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:21:47.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:47 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/1579372511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:21:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:47 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/1728532825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]: dispatch 2024-09-05T12:21:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:47 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]: dispatch 2024-09-05T12:21:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:47 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]': finished 2024-09-05T12:21:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:47 smithi174 ceph-mon[34313]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T12:21:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:21:47.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:47 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/1579372511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:21:47.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:47 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/1728532825' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]: dispatch 2024-09-05T12:21:47.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:47 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]: dispatch 2024-09-05T12:21:47.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:47 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43640d60-2b1a-44c8-8be5-e1540265c057"}]': finished 2024-09-05T12:21:47.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:47 smithi071 ceph-mon[29845]: osdmap e47: 8 total, 7 up, 8 in 2024-09-05T12:21:47.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:21:47.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:47 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/1579372511' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:21:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:48 smithi174 ceph-mon[34313]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:48 smithi071 ceph-mon[29845]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:48 smithi123 ceph-mon[34321]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:50 smithi174 ceph-mon[34313]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:50 smithi071 ceph-mon[29845]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:50 smithi123 ceph-mon[34321]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:52.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:52 smithi123 ceph-mon[34321]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:52 smithi174 ceph-mon[34313]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:52 smithi071 ceph-mon[29845]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:54 smithi123 ceph-mon[34321]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:54 smithi174 ceph-mon[34313]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:54.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:54 smithi071 ceph-mon[29845]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T12:21:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:55.525 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T12:21:55.526 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:55.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-05T12:21:55.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:21:56.312 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:56 smithi123 ceph-mon[34321]: Deploying daemon osd.7 on smithi123 2024-09-05T12:21:56.312 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:56 smithi123 ceph-mon[34321]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:56 smithi174 ceph-mon[34313]: Deploying daemon osd.7 on smithi123 2024-09-05T12:21:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:56 smithi174 ceph-mon[34313]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:56.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:56 smithi071 ceph-mon[29845]: Deploying daemon osd.7 on smithi123 2024-09-05T12:21:56.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:56 smithi071 ceph-mon[29845]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:58.386 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:21:58 smithi123 ceph-mon[34321]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:21:58 smithi174 ceph-mon[34313]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:21:58.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:21:58 smithi071 ceph-mon[29845]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:00 smithi174 ceph-mon[34313]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:00 smithi071 ceph-mon[29845]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:00 smithi123 ceph-mon[34321]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:01.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:01 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:01.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:01 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:01.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:01 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:01.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:01 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:01.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:01 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:01.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:01 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:01.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:01 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:01.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:01 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:01.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:01 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:02 smithi174 ceph-mon[34313]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:02.560 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:02 smithi123 ceph-mon[34321]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:02 smithi071 ceph-mon[29845]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:04.267 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:04 smithi123 ceph-mon[34321]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:04.267 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.267 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.267 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:04.267 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:04.267 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.308 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 7 on host 'smithi123' 2024-09-05T12:22:04.308 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.307+0000 7f22017fa640 1 -- 172.21.15.123:0/1842683834 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2204062630 con 0x7f21ec0a2c00 2024-09-05T12:22:04.310 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21ec0a2c00 msgr2=0x7f21ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21ec0a2c00 0x7f21ec0a50c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f21f4005ab0 tx=0x7f21f4000bc0 comp rx=0 tx=0).stop 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 msgr2=0x7f220419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 0x7f220419f170 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00cce0 tx=0x7f21fc00a620 comp rx=0 tx=0).stop 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 shutdown_connections 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21ec0a2c00 0x7f21ec0a50c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2204102690 0x7f220419f170 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2204101cd0 0x7f220419cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 --2- 172.21.15.123:0/1842683834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2204100ad0 0x7f220406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 >> 172.21.15.123:0/1842683834 conn(0x7f22040fc280 msgr2=0x7f2204104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 shutdown_connections 2024-09-05T12:22:04.311 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:22:04.309+0000 7f220a8e0640 1 -- 172.21.15.123:0/1842683834 wait complete. 2024-09-05T12:22:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:04 smithi174 ceph-mon[34313]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:04.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:04 smithi071 ceph-mon[29845]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:04.947 DEBUG:teuthology.orchestra.run.smithi123:osd.7> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.7.service 2024-09-05T12:22:04.950 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-09-05T12:22:04.950 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-05T12:22:05.204 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:22:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:05 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:05.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:05 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:05.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:05.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:05.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:05 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:06.422 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:06 smithi123 ceph-mon[34321]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:06.453 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:06 smithi174 ceph-mon[34313]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:06.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:06 smithi071 ceph-mon[29845]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:07.163 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:22:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:22:06.706+0000 7fe2ad61f640 -1 osd.7 0 log_to_monitors true 2024-09-05T12:22:07.280 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:07 smithi174 ceph-mon[34313]: from='osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T12:22:07.280 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:07 smithi174 ceph-mon[34313]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T12:22:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:07 smithi123 ceph-mon[34321]: from='osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T12:22:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:07 smithi123 ceph-mon[34321]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T12:22:07.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:07 smithi071 ceph-mon[29845]: from='osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T12:22:07.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:07 smithi071 ceph-mon[29845]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-05T12:22:07.896 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:22:07.922 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-09-05T12:22:08.171 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: osdmap e48: 8 total, 7 up, 8 in 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.479 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:08.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:08.480 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:08 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: osdmap e48: 8 total, 7 up, 8 in 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:08.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:08.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:08 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: osdmap e48: 8 total, 7 up, 8 in 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:08.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:08.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:08.689 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/2457627438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44102690 msgr2=0x7f1a44108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/2457627438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44102690 0x7f1a44108f20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009ca0 tx=0x7f1a3c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/2457627438 shutdown_connections 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/2457627438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44102690 0x7f1a44108f20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/2457627438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 0x7f1a44102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/2457627438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a44100ad0 0x7f1a44100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/2457627438 >> 172.21.15.174:0/2457627438 conn(0x7f1a440fc280 msgr2=0x7f1a440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/2457627438 shutdown_connections 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.664+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/2457627438 wait complete. 2024-09-05T12:22:08.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 Processor -- start 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 -- start start 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 0x7f1a4406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 0x7f1a4419cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a44102690 0x7f1a4419f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410c7d0 con 0x7f1a44101cd0 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410c650 con 0x7f1a44102690 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4c7c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410c950 con 0x7f1a44100ad0 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 0x7f1a4406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 0x7f1a4406aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:56962/0 (socket says 172.21.15.174:56962) 2024-09-05T12:22:08.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 -- 172.21.15.174:0/3991158674 learned_addr learned my addr 172.21.15.174:0/3991158674 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4ad63640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a44102690 0x7f1a4419f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a49d61640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 0x7f1a4419cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 -- 172.21.15.174:0/3991158674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a44102690 msgr2=0x7f1a4419f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a44102690 0x7f1a4419f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 -- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 msgr2=0x7f1a4419cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 0x7f1a4419cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.665+0000 7f1a4a562640 1 -- 172.21.15.174:0/3991158674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4419fb30 con 0x7f1a44100ad0 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a49d61640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 0x7f1a4419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a4a562640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 0x7f1a4406aaf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e9c0 tx=0x7f1a4000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:22:08.692 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000cd80 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4419fe20 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a400035f0 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40010690 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.666+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a441a0350 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.667+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44107ed0 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.668+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1a400037a0 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.669+0000 7f1a3b7fe640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1a300a2b80 0x7f1a300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.669+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5117+0+0 (secure 0 0 0) 0x7f1a400ce7c0 con 0x7f1a44100ad0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.669+0000 7f1a49d61640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1a300a2b80 0x7f1a300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.670+0000 7f1a49d61640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1a300a2b80 0x7f1a300a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1a34006fd0 tx=0x7f1a34008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:22:08.693 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.673+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a4009c3a0 con 0x7f1a44100ad0 2024-09-05T12:22:08.848 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:08.847+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1a44062630 con 0x7f1a300a2b80 2024-09-05T12:22:09.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:22:08.867+0000 7fe2a95a2640 -1 osd.7 0 waiting for initial osdmap 2024-09-05T12:22:09.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:22:08 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:22:08.878+0000 7fe2a13c2640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:22:09.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi123 2024-09-05T12:22:09.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-05T12:22:09.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:22:09.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: osdmap e49: 8 total, 7 up, 8 in 2024-09-05T12:22:09.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='osd.7 ' entity='osd.7' 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] boot 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T12:22:09.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:09 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi123 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: osdmap e49: 8 total, 7 up, 8 in 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:22:09.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:22:09.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:09.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='osd.7 ' entity='osd.7' 2024-09-05T12:22:09.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] boot 2024-09-05T12:22:09.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T12:22:09.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:09 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi123 2024-09-05T12:22:09.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-05T12:22:09.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: osdmap e49: 8 total, 7 up, 8 in 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='osd.7 ' entity='osd.7' 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: osd.7 [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] boot 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: osdmap e50: 8 total, 8 up, 8 in 2024-09-05T12:22:09.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:09 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:22:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:10 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:22:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:10 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:22:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:10 smithi174 ceph-mon[34313]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:22:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:10 smithi174 ceph-mon[34313]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:10 smithi174 ceph-mon[34313]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T12:22:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:10 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:22:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:10 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:22:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:10 smithi071 ceph-mon[29845]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:22:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:10 smithi071 ceph-mon[29845]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:10 smithi071 ceph-mon[29845]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T12:22:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:10 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:22:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:10 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:22:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:10 smithi123 ceph-mon[34321]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:22:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:10 smithi123 ceph-mon[34321]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-05T12:22:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:10 smithi123 ceph-mon[34321]: osdmap e51: 8 total, 8 up, 8 in 2024-09-05T12:22:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: osdmap e52: 8 total, 8 up, 8 in 2024-09-05T12:22:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/725887731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]: dispatch 2024-09-05T12:22:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]: dispatch 2024-09-05T12:22:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]': finished 2024-09-05T12:22:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T12:22:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:12.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:11 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/2303525335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: osdmap e52: 8 total, 8 up, 8 in 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/725887731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]: dispatch 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]: dispatch 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]': finished 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:11 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/2303525335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: osdmap e52: 8 total, 8 up, 8 in 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/725887731' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]: dispatch 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]: dispatch 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8a331c8c-f658-4b3c-b1a8-25199643e7d1"}]': finished 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: osdmap e53: 9 total, 8 up, 9 in 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:12.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:11 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/2303525335' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:22:13.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:13 smithi174 ceph-mon[34313]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T12:22:13.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:13 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:13.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:13 smithi071 ceph-mon[29845]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T12:22:13.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:13 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:13.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:13 smithi123 ceph-mon[34321]: osdmap e54: 9 total, 8 up, 9 in 2024-09-05T12:22:13.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:13 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:14 smithi174 ceph-mon[34313]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:14 smithi174 ceph-mon[34313]: osdmap e55: 9 total, 8 up, 9 in 2024-09-05T12:22:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:14 smithi071 ceph-mon[29845]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:14 smithi071 ceph-mon[29845]: osdmap e55: 9 total, 8 up, 9 in 2024-09-05T12:22:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:14 smithi123 ceph-mon[34321]: pgmap v184: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:14 smithi123 ceph-mon[34321]: osdmap e55: 9 total, 8 up, 9 in 2024-09-05T12:22:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:16.349 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:16 smithi174 ceph-mon[34313]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:16.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:16 smithi123 ceph-mon[34321]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:16.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:16 smithi071 ceph-mon[29845]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:18.462 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:18 smithi174 ceph-mon[34313]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:18.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:18 smithi071 ceph-mon[29845]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:18 smithi123 ceph-mon[34321]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:20 smithi174 ceph-mon[34313]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:20 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T12:22:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:20 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:20 smithi071 ceph-mon[29845]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:20 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T12:22:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:20 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:20 smithi123 ceph-mon[34321]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:20 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-05T12:22:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:20 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:21.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:21 smithi174 ceph-mon[34313]: Deploying daemon osd.8 on smithi174 2024-09-05T12:22:21.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:21 smithi071 ceph-mon[29845]: Deploying daemon osd.8 on smithi174 2024-09-05T12:22:21.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:21 smithi123 ceph-mon[34321]: Deploying daemon osd.8 on smithi174 2024-09-05T12:22:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:22 smithi174 ceph-mon[34313]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:22.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:22 smithi071 ceph-mon[29845]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:22 smithi123 ceph-mon[34321]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:24.377 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:24 smithi174 ceph-mon[34313]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:24 smithi071 ceph-mon[29845]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:24 smithi123 ceph-mon[34321]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:26.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:26 smithi174 ceph-mon[34313]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:26.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:26.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:26.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:26 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:26.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:26 smithi071 ceph-mon[29845]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:26.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:26.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:26.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:26 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:26 smithi123 ceph-mon[34321]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:26 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.506 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:28 smithi174 ceph-mon[34313]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:28.506 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.506 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.506 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:28.507 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:28.507 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:28 smithi071 ceph-mon[29845]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:28 smithi123 ceph-mon[34321]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:28.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:28.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:28.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:28.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.126 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.125+0000 7f1a3b7fe640 1 -- 172.21.15.174:0/3991158674 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1a44062630 con 0x7f1a300a2b80 2024-09-05T12:22:29.126 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-09-05T12:22:29.128 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1a300a2b80 msgr2=0x7f1a300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1a300a2b80 0x7f1a300a5040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1a34006fd0 tx=0x7f1a34008040 comp rx=0 tx=0).stop 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 msgr2=0x7f1a4406aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 0x7f1a4406aaf0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000e9c0 tx=0x7f1a4000ee80 comp rx=0 tx=0).stop 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 shutdown_connections 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1a300a2b80 0x7f1a300a5040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a44102690 0x7f1a4419f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a44101cd0 0x7f1a4419cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.127+0000 7f1a4c7c6640 1 --2- 172.21.15.174:0/3991158674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a44100ad0 0x7f1a4406aaf0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.128+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 >> 172.21.15.174:0/3991158674 conn(0x7f1a440fc280 msgr2=0x7f1a441050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.128+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 shutdown_connections 2024-09-05T12:22:29.129 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:22:29.128+0000 7f1a4c7c6640 1 -- 172.21.15.174:0/3991158674 wait complete. 2024-09-05T12:22:29.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:29.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.493 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:29.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:29.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:29.829 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.8.service 2024-09-05T12:22:29.832 INFO:tasks.cephadm:Deploying osd.9 on smithi071 with /dev/vg_nvme/lv_1... 2024-09-05T12:22:29.832 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T12:22:30.203 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:22:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:30 smithi174 ceph-mon[34313]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:30.516 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:30 smithi071 ceph-mon[29845]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:30 smithi123 ceph-mon[34321]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:31.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:31 smithi174 ceph-mon[34313]: from='osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T12:22:31.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:31 smithi174 ceph-mon[34313]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T12:22:31.261 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:22:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:22:31.093+0000 7f2525c16640 -1 osd.8 0 log_to_monitors true 2024-09-05T12:22:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:31 smithi071 ceph-mon[29845]: from='osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T12:22:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:31 smithi071 ceph-mon[29845]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T12:22:31.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:31 smithi123 ceph-mon[34321]: from='osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T12:22:31.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:31 smithi123 ceph-mon[34321]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-05T12:22:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-mon[34313]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-mon[34313]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T12:22:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-mon[34313]: osdmap e56: 9 total, 8 up, 9 in 2024-09-05T12:22:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-mon[34313]: from='osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:22:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:32.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-mon[34313]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:22:32.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:32 smithi071 ceph-mon[29845]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:32.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:32 smithi071 ceph-mon[29845]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T12:22:32.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:32 smithi071 ceph-mon[29845]: osdmap e56: 9 total, 8 up, 9 in 2024-09-05T12:22:32.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:32 smithi071 ceph-mon[29845]: from='osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:22:32.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:32.633 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:32 smithi071 ceph-mon[29845]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:22:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:32 smithi123 ceph-mon[34321]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:32.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:32 smithi123 ceph-mon[34321]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-05T12:22:32.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:32 smithi123 ceph-mon[34321]: osdmap e56: 9 total, 8 up, 9 in 2024-09-05T12:22:32.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:32 smithi123 ceph-mon[34321]: from='osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:22:32.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:32.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:32 smithi123 ceph-mon[34321]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:22:33.260 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:22:32.955+0000 7f2521b99640 -1 osd.8 0 waiting for initial osdmap 2024-09-05T12:22:33.261 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:22:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:22:32.966+0000 7f25199b9640 -1 osd.8 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:22:33.581 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:22:33.581 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: osdmap e57: 9 total, 8 up, 9 in 2024-09-05T12:22:33.581 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:33.581 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:33.581 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='osd.8 ' entity='osd.8' 2024-09-05T12:22:33.581 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:33.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.587 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:22:33.619 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: osdmap e57: 9 total, 8 up, 9 in 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='osd.8 ' entity='osd.8' 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:33.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:33.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:22:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: osdmap e57: 9 total, 8 up, 9 in 2024-09-05T12:22:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='osd.8 ' entity='osd.8' 2024-09-05T12:22:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:33.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:33.983 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:22:34.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi174 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] boot 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T12:22:34.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 -- 172.21.15.71:0/819352462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0074c50 msgr2=0x7f6ba00750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/819352462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0074c50 0x7f6ba00750b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009900 tx=0x7f6b8802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 -- 172.21.15.71:0/819352462 shutdown_connections 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/819352462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0109a80 0x7f6ba010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/819352462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0074c50 0x7f6ba00750b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/819352462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0076220 0x7f6ba0074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.557+0000 7f6ba6c00640 1 -- 172.21.15.71:0/819352462 >> 172.21.15.71:0/819352462 conn(0x7f6ba00fd390 msgr2=0x7f6ba00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:22:34.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.558+0000 7f6ba6c00640 1 -- 172.21.15.71:0/819352462 shutdown_connections 2024-09-05T12:22:34.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.558+0000 7f6ba6c00640 1 -- 172.21.15.71:0/819352462 wait complete. 2024-09-05T12:22:34.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 Processor -- start 2024-09-05T12:22:34.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 -- start start 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 0x7f6ba019a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0076220 0x7f6ba019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0109a80 0x7f6ba01a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010faf0 con 0x7f6ba0109a80 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010f970 con 0x7f6ba0076220 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba6c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010fc70 con 0x7f6ba0074c50 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6ba4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0109a80 0x7f6ba01a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 0x7f6ba019a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6b97e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0076220 0x7f6ba019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6b9fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 0x7f6ba019a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47326/0 (socket says 172.21.15.71:47326) 2024-09-05T12:22:34.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.559+0000 7f6b9fe00640 1 -- 172.21.15.71:0/3395311000 learned_addr learned my addr 172.21.15.71:0/3395311000 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9fe00640 1 -- 172.21.15.71:0/3395311000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0076220 msgr2=0x7f6ba019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9fe00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0076220 0x7f6ba019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9fe00640 1 -- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0109a80 msgr2=0x7f6ba01a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9fe00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0109a80 0x7f6ba01a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9fe00640 1 -- 172.21.15.71:0/3395311000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba01a1d70 con 0x7f6ba0074c50 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b97e00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0076220 0x7f6ba019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9fe00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 0x7f6ba019a140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e9f0 tx=0x7f6b9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9000ccc0 con 0x7f6ba0074c50 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba01a2060 con 0x7f6ba0074c50 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b90005500 con 0x7f6ba0074c50 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90010690 con 0x7f6ba0074c50 2024-09-05T12:22:34.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.560+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba01a2570 con 0x7f6ba0074c50 2024-09-05T12:22:34.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.561+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba0108800 con 0x7f6ba0074c50 2024-09-05T12:22:34.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.563+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6b900046c0 con 0x7f6ba0074c50 2024-09-05T12:22:34.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.565+0000 7f6b9d600640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b840a2bb0 0x7f6b840a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:22:34.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.565+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5858+0+0 (secure 0 0 0) 0x7f6b900cea70 con 0x7f6ba0074c50 2024-09-05T12:22:34.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.565+0000 7f6b97e00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b840a2bb0 0x7f6b840a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:22:34.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.565+0000 7f6b97e00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b840a2bb0 0x7f6b840a5070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b88031440 tx=0x7f6b880315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:22:34.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.567+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b9009c2c0 con 0x7f6ba0074c50 2024-09-05T12:22:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi174 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] boot 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T12:22:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:34.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:34.749+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6ba0062630 con 0x7f6b840a2bb0 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi174 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: osd.8 [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] boot 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: osdmap e58: 9 total, 9 up, 9 in 2024-09-05T12:22:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:22:35.629 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:35 smithi071 ceph-mon[29845]: osdmap e59: 9 total, 9 up, 9 in 2024-09-05T12:22:35.629 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:22:35.629 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:22:35.629 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:35.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:35 smithi123 ceph-mon[34321]: osdmap e59: 9 total, 9 up, 9 in 2024-09-05T12:22:35.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:22:35.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:22:35.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:35.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:35 smithi174 ceph-mon[34313]: osdmap e59: 9 total, 9 up, 9 in 2024-09-05T12:22:35.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:22:35.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:22:35.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:36.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:36 smithi071 ceph-mon[29845]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:22:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:36 smithi071 ceph-mon[29845]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:36 smithi071 ceph-mon[29845]: osdmap e60: 9 total, 9 up, 9 in 2024-09-05T12:22:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:36 smithi123 ceph-mon[34321]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:22:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:36 smithi123 ceph-mon[34321]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:36 smithi123 ceph-mon[34321]: osdmap e60: 9 total, 9 up, 9 in 2024-09-05T12:22:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:36 smithi174 ceph-mon[34313]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:22:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:36 smithi174 ceph-mon[34313]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:36 smithi174 ceph-mon[34313]: osdmap e60: 9 total, 9 up, 9 in 2024-09-05T12:22:37.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:37 smithi071 ceph-mon[29845]: osdmap e61: 9 total, 9 up, 9 in 2024-09-05T12:22:37.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:37 smithi123 ceph-mon[34321]: osdmap e61: 9 total, 9 up, 9 in 2024-09-05T12:22:37.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:37 smithi174 ceph-mon[34313]: osdmap e61: 9 total, 9 up, 9 in 2024-09-05T12:22:38.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:38 smithi071 ceph-mon[29845]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:38.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:38 smithi071 ceph-mon[29845]: osdmap e62: 9 total, 9 up, 9 in 2024-09-05T12:22:38.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:38 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/120421400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fcc4a9e-318d-482f-babb-6ad8e5ff5790"}]: dispatch 2024-09-05T12:22:38.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:38 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/120421400' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fcc4a9e-318d-482f-babb-6ad8e5ff5790"}]': finished 2024-09-05T12:22:38.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:38 smithi071 ceph-mon[29845]: osdmap e63: 10 total, 9 up, 10 in 2024-09-05T12:22:38.590 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:22:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:38 smithi123 ceph-mon[34321]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:38 smithi123 ceph-mon[34321]: osdmap e62: 9 total, 9 up, 9 in 2024-09-05T12:22:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:38 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/120421400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fcc4a9e-318d-482f-babb-6ad8e5ff5790"}]: dispatch 2024-09-05T12:22:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:38 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/120421400' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fcc4a9e-318d-482f-babb-6ad8e5ff5790"}]': finished 2024-09-05T12:22:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:38 smithi123 ceph-mon[34321]: osdmap e63: 10 total, 9 up, 10 in 2024-09-05T12:22:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:22:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:38 smithi174 ceph-mon[34313]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:38 smithi174 ceph-mon[34313]: osdmap e62: 9 total, 9 up, 9 in 2024-09-05T12:22:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:38 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/120421400' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9fcc4a9e-318d-482f-babb-6ad8e5ff5790"}]: dispatch 2024-09-05T12:22:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:38 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/120421400' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9fcc4a9e-318d-482f-babb-6ad8e5ff5790"}]': finished 2024-09-05T12:22:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:38 smithi174 ceph-mon[34313]: osdmap e63: 10 total, 9 up, 10 in 2024-09-05T12:22:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:22:39.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:39 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2284681774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:22:39.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:39 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2284681774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:22:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:39 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2284681774' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:22:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:40 smithi071 ceph-mon[29845]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:40 smithi123 ceph-mon[34321]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:40 smithi174 ceph-mon[34313]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:41.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:41 smithi071 ceph-mon[29845]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-05T12:22:41.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:41 smithi123 ceph-mon[34321]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-05T12:22:41.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:41 smithi174 ceph-mon[34313]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-05T12:22:44.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:44 smithi123 ceph-mon[34321]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-05T12:22:44.463 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:44 smithi174 ceph-mon[34313]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-05T12:22:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:44 smithi071 ceph-mon[29845]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-05T12:22:46.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:46 smithi123 ceph-mon[34321]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T12:22:46.434 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:46 smithi071 ceph-mon[29845]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T12:22:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:46 smithi174 ceph-mon[34313]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T12:22:47.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T12:22:47.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:47 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:47.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T12:22:47.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:47 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-05T12:22:47.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:47 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:48.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:48 smithi123 ceph-mon[34321]: Deploying daemon osd.9 on smithi071 2024-09-05T12:22:48.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:48 smithi123 ceph-mon[34321]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T12:22:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:48 smithi174 ceph-mon[34313]: Deploying daemon osd.9 on smithi071 2024-09-05T12:22:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:48 smithi174 ceph-mon[34313]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T12:22:48.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:48 smithi071 ceph-mon[29845]: Deploying daemon osd.9 on smithi071 2024-09-05T12:22:48.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:48 smithi071 ceph-mon[29845]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-05T12:22:50.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:50 smithi071 ceph-mon[29845]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-05T12:22:50.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:50 smithi123 ceph-mon[34321]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-05T12:22:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:50 smithi174 ceph-mon[34313]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-09-05T12:22:52.217 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:52 smithi071 ceph-mon[29845]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T12:22:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:52 smithi174 ceph-mon[34313]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T12:22:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:52 smithi123 ceph-mon[34321]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T12:22:54.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:54 smithi071 ceph-mon[29845]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:54 smithi174 ceph-mon[34313]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:54 smithi123 ceph-mon[34321]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:55.242 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:55.242 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:55.242 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:55 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:55 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:55.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:55 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:56.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:56 smithi071 ceph-mon[29845]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:56 smithi174 ceph-mon[34313]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:56 smithi123 ceph-mon[34321]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:58.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:58 smithi071 ceph-mon[29845]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:58.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:58 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:58 smithi174 ceph-mon[34313]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:58 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:58 smithi123 ceph-mon[34321]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:22:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:22:58.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:22:58.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:58 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:58.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.893+0000 7f6b9d600640 1 -- 172.21.15.71:0/3395311000 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ba0062630 con 0x7f6b840a2bb0 2024-09-05T12:22:58.896 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 9 on host 'smithi071' 2024-09-05T12:22:58.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b840a2bb0 msgr2=0x7f6b840a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b840a2bb0 0x7f6b840a5070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6b88031440 tx=0x7f6b880315b0 comp rx=0 tx=0).stop 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 msgr2=0x7f6ba019a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 0x7f6ba019a140 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b9000e9f0 tx=0x7f6b9000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 shutdown_connections 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b840a2bb0 0x7f6b840a5070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ba0109a80 0x7f6ba01a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ba0076220 0x7f6ba019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 --2- 172.21.15.71:0/3395311000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ba0074c50 0x7f6ba019a140 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 >> 172.21.15.71:0/3395311000 conn(0x7f6ba00fd390 msgr2=0x7f6ba01059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 shutdown_connections 2024-09-05T12:22:58.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:22:58.896+0000 7f6ba6c00640 1 -- 172.21.15.71:0/3395311000 wait complete. 2024-09-05T12:22:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:22:59 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:22:59 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:22:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:22:59 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:22:59.866 DEBUG:teuthology.orchestra.run.smithi071:osd.9> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.9.service 2024-09-05T12:22:59.870 INFO:tasks.cephadm:Deploying osd.10 on smithi123 with /dev/vg_nvme/lv_1... 2024-09-05T12:22:59.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T12:23:00.123 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:23:00.127 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:22:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:22:59.980+0000 7f346cb24640 -1 osd.9 0 log_to_monitors true 2024-09-05T12:23:00.388 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:00 smithi071 ceph-mon[29845]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:00.388 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:00 smithi071 ceph-mon[29845]: from='osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T12:23:00.388 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:00 smithi071 ceph-mon[29845]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T12:23:00.486 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:00 smithi123 ceph-mon[34321]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:00.487 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:00 smithi123 ceph-mon[34321]: from='osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T12:23:00.487 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:00 smithi123 ceph-mon[34321]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T12:23:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:00 smithi174 ceph-mon[34313]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:00 smithi174 ceph-mon[34313]: from='osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T12:23:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:00 smithi174 ceph-mon[34313]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-05T12:23:02.101 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:01 smithi123 ceph-mon[34321]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T12:23:02.101 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:01 smithi123 ceph-mon[34321]: from='osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:23:02.101 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:01 smithi123 ceph-mon[34321]: osdmap e64: 10 total, 9 up, 10 in 2024-09-05T12:23:02.101 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:01 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:02.101 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:01 smithi123 ceph-mon[34321]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:23:02.102 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:01 smithi123 ceph-mon[34321]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:01 smithi071 ceph-mon[29845]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T12:23:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:01 smithi071 ceph-mon[29845]: from='osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:23:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:01 smithi071 ceph-mon[29845]: osdmap e64: 10 total, 9 up, 10 in 2024-09-05T12:23:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:01 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:01 smithi071 ceph-mon[29845]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:23:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:01 smithi071 ceph-mon[29845]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:01 smithi174 ceph-mon[34313]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-05T12:23:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:01 smithi174 ceph-mon[34313]: from='osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:23:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:01 smithi174 ceph-mon[34313]: osdmap e64: 10 total, 9 up, 10 in 2024-09-05T12:23:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:01 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:01 smithi174 ceph-mon[34313]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-09-05T12:23:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:01 smithi174 ceph-mon[34313]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:02.754 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-05T12:23:02.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-09-05T12:23:02.909 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:23:02.630+0000 7f3466c00640 -1 osd.9 0 waiting for initial osdmap 2024-09-05T12:23:02.910 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:23:02.652+0000 7f345cc00640 -1 osd.9 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:23:03.023 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:23:03.144 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:02 smithi123 ceph-mon[34321]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:23:03.145 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:02 smithi123 ceph-mon[34321]: osdmap e65: 10 total, 9 up, 10 in 2024-09-05T12:23:03.145 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:02 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:03.145 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:02 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:03.145 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:02 smithi123 ceph-mon[34321]: from='osd.9 ' entity='osd.9' 2024-09-05T12:23:03.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-mon[29845]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:23:03.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-mon[29845]: osdmap e65: 10 total, 9 up, 10 in 2024-09-05T12:23:03.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:03.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:03.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:02 smithi071 ceph-mon[29845]: from='osd.9 ' entity='osd.9' 2024-09-05T12:23:03.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:02 smithi174 ceph-mon[34313]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-09-05T12:23:03.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:02 smithi174 ceph-mon[34313]: osdmap e65: 10 total, 9 up, 10 in 2024-09-05T12:23:03.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:02 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:03.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:02 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:03.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:02 smithi174 ceph-mon[34313]: from='osd.9 ' entity='osd.9' 2024-09-05T12:23:03.542 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 -- 172.21.15.123:0/996915976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 msgr2=0x7fa2181073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 --2- 172.21.15.123:0/996915976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa2181073e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa214009900 tx=0x7fa21402a410 comp rx=0 tx=0).stop 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 -- 172.21.15.123:0/996915976 shutdown_connections 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 --2- 172.21.15.123:0/996915976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa218107920 0x7fa218109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 --2- 172.21.15.123:0/996915976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa2181073e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 --2- 172.21.15.123:0/996915976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2181026c0 0x7fa218104ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 -- 172.21.15.123:0/996915976 >> 172.21.15.123:0/996915976 conn(0x7fa2180fc280 msgr2=0x7fa2180fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 -- 172.21.15.123:0/996915976 shutdown_connections 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.532+0000 7fa21fe69640 1 -- 172.21.15.123:0/996915976 wait complete. 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 Processor -- start 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 -- start start 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2181026c0 0x7fa218197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:03.543 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa218198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa218107920 0x7fa21819f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa21810d830 con 0x7fa2181026c0 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa21810d6b0 con 0x7fa218104ff0 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21fe69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa21810d9b0 con 0x7fa218107920 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa218198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa218198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:39042/0 (socket says 172.21.15.123:39042) 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 -- 172.21.15.123:0/4107655568 learned_addr learned my addr 172.21.15.123:0/4107655568 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21dc05640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2181026c0 0x7fa218197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21e406640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa218107920 0x7fa21819f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 -- 172.21.15.123:0/4107655568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa218107920 msgr2=0x7fa21819f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:03.544 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa218107920 0x7fa21819f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 -- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2181026c0 msgr2=0x7fa218197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2181026c0 0x7fa218197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 -- 172.21.15.123:0/4107655568 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21819fb50 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21e406640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa218107920 0x7fa21819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.533+0000 7fa21d404640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa218198460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa214031610 tx=0x7fa2140043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.534+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21403f030 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.534+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa214003ce0 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.534+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21819fde0 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.534+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214035070 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.534+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2181a0260 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.535+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2180fdd70 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.536+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa214042030 con 0x7fa218104ff0 2024-09-05T12:23:03.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.536+0000 7fa20affd640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2040a2bb0 0x7fa2040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:03.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.536+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5981+0+0 (secure 0 0 0) 0x7fa2140eed20 con 0x7fa218104ff0 2024-09-05T12:23:03.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.536+0000 7fa21dc05640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2040a2bb0 0x7fa2040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:03.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.537+0000 7fa21dc05640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2040a2bb0 0x7fa2040a5070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005fd0 tx=0x7fa20c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:03.546 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.540+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2140bc5a0 con 0x7fa218104ff0 2024-09-05T12:23:03.718 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:03.716+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa218062630 con 0x7fa2040a2bb0 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] boot 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: osdmap e66: 10 total, 10 up, 10 in 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:23:04.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:23:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:23:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] boot 2024-09-05T12:23:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: osdmap e66: 10 total, 10 up, 10 in 2024-09-05T12:23:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:23:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:23:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:04.352 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: osd.9 [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] boot 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: osdmap e66: 10 total, 10 up, 10 in 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:23:04.353 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi071 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 4272M 2024-09-05T12:23:05.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:05.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:05.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:04 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi071 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.355 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 4272M 2024-09-05T12:23:05.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:05.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:05.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:04 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi071 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 4272M 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:05.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:04 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:06.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:06 smithi174 ceph-mon[34313]: osdmap e67: 10 total, 10 up, 10 in 2024-09-05T12:23:06.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:06 smithi174 ceph-mon[34313]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:06.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:06 smithi071 ceph-mon[29845]: osdmap e67: 10 total, 10 up, 10 in 2024-09-05T12:23:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:06 smithi071 ceph-mon[29845]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:06.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:06 smithi123 ceph-mon[34321]: osdmap e67: 10 total, 10 up, 10 in 2024-09-05T12:23:06.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:06 smithi123 ceph-mon[34321]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:07.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/1550177411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]: dispatch 2024-09-05T12:23:07.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:07 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]: dispatch 2024-09-05T12:23:07.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:07 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]': finished 2024-09-05T12:23:07.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:07 smithi123 ceph-mon[34321]: osdmap e68: 11 total, 10 up, 11 in 2024-09-05T12:23:07.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:07.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.123:0/3308798510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:23:07.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/1550177411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]: dispatch 2024-09-05T12:23:07.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:07 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]: dispatch 2024-09-05T12:23:07.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:07 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]': finished 2024-09-05T12:23:07.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:07 smithi174 ceph-mon[34313]: osdmap e68: 11 total, 10 up, 11 in 2024-09-05T12:23:07.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:07.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.123:0/3308798510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:23:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/1550177411' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]: dispatch 2024-09-05T12:23:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:07 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]: dispatch 2024-09-05T12:23:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:07 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbae3521-9c6b-45e7-9912-2afebeecdf3b"}]': finished 2024-09-05T12:23:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:07 smithi071 ceph-mon[29845]: osdmap e68: 11 total, 10 up, 11 in 2024-09-05T12:23:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.123:0/3308798510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:23:08.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:08 smithi174 ceph-mon[34313]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:08 smithi071 ceph-mon[29845]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:08 smithi123 ceph-mon[34321]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:10.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:10 smithi123 ceph-mon[34321]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:10 smithi174 ceph-mon[34313]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:10.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:10 smithi071 ceph-mon[29845]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:12.375 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:12 smithi123 ceph-mon[34321]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:12 smithi174 ceph-mon[34313]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:12.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:12 smithi071 ceph-mon[29845]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:14.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:14 smithi123 ceph-mon[34321]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:14.440 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:14 smithi071 ceph-mon[29845]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:14 smithi174 ceph-mon[34313]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:15.374 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:15 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T12:23:15.374 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:15 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T12:23:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:15.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-05T12:23:15.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:16.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:16 smithi123 ceph-mon[34321]: Deploying daemon osd.10 on smithi123 2024-09-05T12:23:16.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:16 smithi123 ceph-mon[34321]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:16 smithi174 ceph-mon[34313]: Deploying daemon osd.10 on smithi123 2024-09-05T12:23:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:16 smithi174 ceph-mon[34313]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:16.562 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:16 smithi071 ceph-mon[29845]: Deploying daemon osd.10 on smithi123 2024-09-05T12:23:16.563 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:16 smithi071 ceph-mon[29845]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:18.329 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:18 smithi123 ceph-mon[34321]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:18.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:18 smithi071 ceph-mon[29845]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:18 smithi174 ceph-mon[34313]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:20.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:20 smithi123 ceph-mon[34321]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:20 smithi174 ceph-mon[34313]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:20 smithi071 ceph-mon[29845]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:21.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:21 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:21.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:21 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:21.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:21 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:21.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:21 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:21.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:21 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:21.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:21 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:21.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:21 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:21.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:21 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:21.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:21 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:22.319 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:22 smithi123 ceph-mon[34321]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:22 smithi174 ceph-mon[34313]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:22.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:22 smithi071 ceph-mon[29845]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:23.904 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.902+0000 7fa20affd640 1 -- 172.21.15.123:0/4107655568 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa218062630 con 0x7fa2040a2bb0 2024-09-05T12:23:23.905 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 10 on host 'smithi123' 2024-09-05T12:23:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2040a2bb0 msgr2=0x7fa2040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2040a2bb0 0x7fa2040a5070 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa20c005fd0 tx=0x7fa20c005850 comp rx=0 tx=0).stop 2024-09-05T12:23:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 msgr2=0x7fa218198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa218198460 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa214031610 tx=0x7fa2140043b0 comp rx=0 tx=0).stop 2024-09-05T12:23:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 shutdown_connections 2024-09-05T12:23:23.906 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2040a2bb0 0x7fa2040a5070 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa218107920 0x7fa21819f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa218104ff0 0x7fa218198460 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 --2- 172.21.15.123:0/4107655568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2181026c0 0x7fa218197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 >> 172.21.15.123:0/4107655568 conn(0x7fa2180fc280 msgr2=0x7fa218103000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 shutdown_connections 2024-09-05T12:23:23.907 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-05T12:23:23.905+0000 7fa21fe69640 1 -- 172.21.15.123:0/4107655568 wait complete. 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:24 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:24 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:24 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:24.576 DEBUG:teuthology.orchestra.run.smithi123:osd.10> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.10.service 2024-09-05T12:23:24.578 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-09-05T12:23:24.578 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-05T12:23:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:23:26.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:26 smithi174 ceph-mon[34313]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:26.247 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:26 smithi174 ceph-mon[34313]: from='osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T12:23:26.247 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:26 smithi174 ceph-mon[34313]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T12:23:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:26 smithi071 ceph-mon[29845]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:26 smithi071 ceph-mon[29845]: from='osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T12:23:26.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:26 smithi071 ceph-mon[29845]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T12:23:26.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:26 smithi123 ceph-mon[34321]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:26.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:26 smithi123 ceph-mon[34321]: from='osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T12:23:26.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:26 smithi123 ceph-mon[34321]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-05T12:23:26.430 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:23:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:23:26.101+0000 7f9d913bc640 -1 osd.10 0 log_to_monitors true 2024-09-05T12:23:27.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-mon[34321]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T12:23:27.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-mon[34321]: from='osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:23:27.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-mon[34321]: osdmap e69: 11 total, 10 up, 11 in 2024-09-05T12:23:27.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:27.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-mon[34321]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:23:27.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-mon[34321]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:23:27.357 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:23:27.133+0000 7f9d8d33f640 -1 osd.10 0 waiting for initial osdmap 2024-09-05T12:23:27.357 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:23:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:23:27.152+0000 7f9d89167640 -1 osd.10 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:23:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:27 smithi071 ceph-mon[29845]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T12:23:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:27 smithi071 ceph-mon[29845]: from='osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:23:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:27 smithi071 ceph-mon[29845]: osdmap e69: 11 total, 10 up, 11 in 2024-09-05T12:23:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:27 smithi071 ceph-mon[29845]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:23:27.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:27 smithi071 ceph-mon[29845]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:23:27.497 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:27 smithi174 ceph-mon[34313]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-05T12:23:27.497 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:27 smithi174 ceph-mon[34313]: from='osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:23:27.497 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:27 smithi174 ceph-mon[34313]: osdmap e69: 11 total, 10 up, 11 in 2024-09-05T12:23:27.497 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:27.497 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:27 smithi174 ceph-mon[34313]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-09-05T12:23:27.497 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:27 smithi174 ceph-mon[34313]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-05T12:23:27.500 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-05T12:23:27.526 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-09-05T12:23:27.776 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:23:28.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 -- 172.21.15.174:0/692579918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 msgr2=0x7f2b1c108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 --2- 172.21.15.174:0/692579918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c108f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009ca0 tx=0x7f2b1402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 -- 172.21.15.174:0/692579918 shutdown_connections 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 --2- 172.21.15.174:0/692579918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c108f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 --2- 172.21.15.174:0/692579918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 0x7f2b1c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 --2- 172.21.15.174:0/692579918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b1c100ad0 0x7f2b1c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.285+0000 7f2b22050640 1 -- 172.21.15.174:0/692579918 >> 172.21.15.174:0/692579918 conn(0x7f2b1c0fc280 msgr2=0x7f2b1c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 -- 172.21.15.174:0/692579918 shutdown_connections 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 -- 172.21.15.174:0/692579918 wait complete. 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 Processor -- start 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 -- start start 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b1c100ad0 0x7f2b1c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 0x7f2b1c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c10c750 con 0x7f2b1c100ad0 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c10c5d0 con 0x7f2b1c101cd0 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.286+0000 7f2b22050640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c10c8d0 con 0x7f2b1c102690 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c19f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:37558/0 (socket says 172.21.15.174:37558) 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 -- 172.21.15.174:0/3285964539 learned_addr learned my addr 172.21.15.174:0/3285964539 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1affd640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 0x7f2b1c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:28.303 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1b7fe640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b1c100ad0 0x7f2b1c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 -- 172.21.15.174:0/3285964539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 msgr2=0x7f2b1c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 0x7f2b1c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 -- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b1c100ad0 msgr2=0x7f2b1c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b1c100ad0 0x7f2b1c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 -- 172.21.15.174:0/3285964539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c19faa0 con 0x7f2b1c102690 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1affd640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 0x7f2b1c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b1bfff640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c19f430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b14031ae0 tx=0x7f2b14031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14003b70 con 0x7f2b1c102690 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b14003d10 con 0x7f2b1c102690 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c19fd30 con 0x7f2b1c102690 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b1402fc40 con 0x7f2b1c102690 2024-09-05T12:23:28.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.287+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c1a0260 con 0x7f2b1c102690 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.288+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1c1a0490 con 0x7f2b1c102690 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.289+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2b14004050 con 0x7f2b1c102690 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.290+0000 7f2b18ff9640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b040a2c30 0x7f2b040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.290+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6727+0+0 (secure 0 0 0) 0x7f2b140efab0 con 0x7f2b1c102690 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.290+0000 7f2b1b7fe640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b040a2c30 0x7f2b040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.291+0000 7f2b1b7fe640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b040a2c30 0x7f2b040a50f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c101b30 tx=0x7f2b0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:28.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.294+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2b140bd0c0 con 0x7f2b1c102690 2024-09-05T12:23:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: osdmap e70: 11 total, 10 up, 11 in 2024-09-05T12:23:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:28 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: osdmap e70: 11 total, 10 up, 11 in 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:28.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:28.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:28 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.475 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:28.474+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2b1c0fe4b0 con 0x7f2b040a2c30 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: osdmap e70: 11 total, 10 up, 11 in 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.476 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:28.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:28.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:28.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:28.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:28 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi123 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] boot 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: osdmap e71: 11 total, 11 up, 11 in 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:23:29.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:29 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:29.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi123 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] boot 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: osdmap e71: 11 total, 11 up, 11 in 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:23:29.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:29 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi123 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: osd.10 [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] boot 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: osdmap e71: 11 total, 11 up, 11 in 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:23:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-05T12:23:29.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-05T12:23:29.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:29 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:30 smithi123 ceph-mon[34321]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:23:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:30 smithi123 ceph-mon[34321]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:30 smithi123 ceph-mon[34321]: osdmap e72: 11 total, 11 up, 11 in 2024-09-05T12:23:30.437 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:30 smithi174 ceph-mon[34313]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:23:30.437 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:30 smithi174 ceph-mon[34313]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:30.437 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:30 smithi174 ceph-mon[34313]: osdmap e72: 11 total, 11 up, 11 in 2024-09-05T12:23:30.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:30 smithi071 ceph-mon[29845]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:23:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:30 smithi071 ceph-mon[29845]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-05T12:23:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:30 smithi071 ceph-mon[29845]: osdmap e72: 11 total, 11 up, 11 in 2024-09-05T12:23:31.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:31 smithi123 ceph-mon[34321]: osdmap e73: 11 total, 11 up, 11 in 2024-09-05T12:23:31.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:31 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/417931723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]: dispatch 2024-09-05T12:23:31.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:31 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]: dispatch 2024-09-05T12:23:31.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:31 smithi123 ceph-mon[34321]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]': finished 2024-09-05T12:23:31.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:31 smithi123 ceph-mon[34321]: osdmap e74: 12 total, 11 up, 12 in 2024-09-05T12:23:31.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:31.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:31 smithi174 ceph-mon[34313]: osdmap e73: 11 total, 11 up, 11 in 2024-09-05T12:23:31.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:31 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/417931723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]: dispatch 2024-09-05T12:23:31.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:31 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]: dispatch 2024-09-05T12:23:31.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:31 smithi174 ceph-mon[34313]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]': finished 2024-09-05T12:23:31.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:31 smithi174 ceph-mon[34313]: osdmap e74: 12 total, 11 up, 12 in 2024-09-05T12:23:31.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:31.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:31 smithi071 ceph-mon[29845]: osdmap e73: 11 total, 11 up, 11 in 2024-09-05T12:23:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:31 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/417931723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]: dispatch 2024-09-05T12:23:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:31 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]: dispatch 2024-09-05T12:23:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:31 smithi071 ceph-mon[29845]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3749498e-3656-4d1e-aa2c-40b565b46488"}]': finished 2024-09-05T12:23:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:31 smithi071 ceph-mon[29845]: osdmap e74: 12 total, 11 up, 12 in 2024-09-05T12:23:31.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:32.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:32 smithi123 ceph-mon[34321]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:32.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:32 smithi123 ceph-mon[34321]: from='client.? 172.21.15.174:0/3687123537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:23:32.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:32 smithi123 ceph-mon[34321]: osdmap e75: 12 total, 11 up, 12 in 2024-09-05T12:23:32.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:32 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:32 smithi174 ceph-mon[34313]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:32 smithi174 ceph-mon[34313]: from='client.? 172.21.15.174:0/3687123537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:23:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:32 smithi174 ceph-mon[34313]: osdmap e75: 12 total, 11 up, 12 in 2024-09-05T12:23:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:32 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:32.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:32 smithi071 ceph-mon[29845]: pgmap v242: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:32.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:32 smithi071 ceph-mon[29845]: from='client.? 172.21.15.174:0/3687123537' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-05T12:23:32.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:32 smithi071 ceph-mon[29845]: osdmap e75: 12 total, 11 up, 12 in 2024-09-05T12:23:32.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:32 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:34 smithi174 ceph-mon[34313]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:34.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:34 smithi071 ceph-mon[29845]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:34 smithi123 ceph-mon[34321]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:36.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:36 smithi174 ceph-mon[34313]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T12:23:36.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:36 smithi071 ceph-mon[29845]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T12:23:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:36 smithi123 ceph-mon[34321]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-05T12:23:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:38 smithi174 ceph-mon[34313]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T12:23:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:38 smithi071 ceph-mon[29845]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T12:23:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:38 smithi123 ceph-mon[34321]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-05T12:23:40.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:40 smithi174 ceph-mon[34313]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T12:23:40.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T12:23:40.508 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:40 smithi071 ceph-mon[29845]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T12:23:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T12:23:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:40 smithi123 ceph-mon[34321]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-05T12:23:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-05T12:23:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:41.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:41 smithi174 ceph-mon[34313]: Deploying daemon osd.11 on smithi174 2024-09-05T12:23:41.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:41 smithi071 ceph-mon[29845]: Deploying daemon osd.11 on smithi174 2024-09-05T12:23:41.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:41 smithi123 ceph-mon[34321]: Deploying daemon osd.11 on smithi174 2024-09-05T12:23:42.426 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:42 smithi174 ceph-mon[34313]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T12:23:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:42 smithi071 ceph-mon[29845]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T12:23:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:42 smithi123 ceph-mon[34321]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-05T12:23:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:44 smithi174 ceph-mon[34313]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-05T12:23:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:44 smithi071 ceph-mon[29845]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-05T12:23:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:44 smithi123 ceph-mon[34321]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-05T12:23:45.502 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:45.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:45.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:45 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:45.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:45.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:45.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:45 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:45.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:45.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:45.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:45 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:46 smithi174 ceph-mon[34313]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T12:23:46.562 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:46 smithi071 ceph-mon[29845]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T12:23:46.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:46 smithi123 ceph-mon[34321]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-05T12:23:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:48 smithi174 ceph-mon[34313]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:48 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.622+0000 7f2b18ff9640 1 -- 172.21.15.174:0/3285964539 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f2b1c0fe4b0 con 0x7f2b040a2c30 2024-09-05T12:23:48.857 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-09-05T12:23:48.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b040a2c30 msgr2=0x7f2b040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:48.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b040a2c30 0x7f2b040a50f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c101b30 tx=0x7f2b0c005f90 comp rx=0 tx=0).stop 2024-09-05T12:23:48.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 msgr2=0x7f2b1c19f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:48.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c19f430 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b14031ae0 tx=0x7f2b14031ee0 comp rx=0 tx=0).stop 2024-09-05T12:23:48.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 shutdown_connections 2024-09-05T12:23:48.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b040a2c30 0x7f2b040a50f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:48.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c102690 0x7f2b1c19f430 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:48.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b1c101cd0 0x7f2b1c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:48.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 --2- 172.21.15.174:0/3285964539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b1c100ad0 0x7f2b1c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:48.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 >> 172.21.15.174:0/3285964539 conn(0x7f2b1c0fc280 msgr2=0x7f2b1c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:48.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 shutdown_connections 2024-09-05T12:23:48.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:48.625+0000 7f2b22050640 1 -- 172.21.15.174:0/3285964539 wait complete. 2024-09-05T12:23:48.859 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:48 smithi123 ceph-mon[34321]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:48 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:48 smithi071 ceph-mon[29845]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:48.860 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:48 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:49.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.362 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.11.service 2024-09-05T12:23:49.364 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-05T12:23:49.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T12:23:49.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:23:49.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:49.736 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:23:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 -- 172.21.15.71:0/3625640678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 msgr2=0x7eff481052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3625640678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff481052d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7eff34009900 tx=0x7eff3402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 -- 172.21.15.71:0/3625640678 shutdown_connections 2024-09-05T12:23:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3625640678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 0x7eff4810c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3625640678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff481052d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3625640678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff48103c50 0x7eff48104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 -- 172.21.15.71:0/3625640678 >> 172.21.15.71:0/3625640678 conn(0x7eff480ff3e0 msgr2=0x7eff48101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 -- 172.21.15.71:0/3625640678 shutdown_connections 2024-09-05T12:23:50.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.371+0000 7eff4ec00640 1 -- 172.21.15.71:0/3625640678 wait complete. 2024-09-05T12:23:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 Processor -- start 2024-09-05T12:23:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 -- start start 2024-09-05T12:23:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff48103c50 0x7eff48199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff4819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 0x7eff481a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:50.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7eff4810f920 con 0x7eff48105810 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff4810f7a0 con 0x7eff48103c50 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff4810faa0 con 0x7eff48104e50 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff4819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.372+0000 7eff4e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 0x7eff481a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff4819a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:43348/0 (socket says 172.21.15.71:43348) 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 -- 172.21.15.71:0/3728407915 learned_addr learned my addr 172.21.15.71:0/3728407915 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4d800640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff48103c50 0x7eff48199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 -- 172.21.15.71:0/3728407915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff48103c50 msgr2=0x7eff48199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff48103c50 0x7eff48199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 -- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 msgr2=0x7eff481a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 0x7eff481a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 -- 172.21.15.71:0/3728407915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff481a1b20 con 0x7eff48104e50 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4e200640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 0x7eff481a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.373+0000 7eff4ce00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff4819a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff34031440 tx=0x7eff34032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.374+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3403f030 con 0x7eff48104e50 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.374+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff34003ce0 con 0x7eff48104e50 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.374+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff481a1db0 con 0x7eff48104e50 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.374+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34035070 con 0x7eff48104e50 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.374+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff481a2250 con 0x7eff48104e50 2024-09-05T12:23:50.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.375+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4810b000 con 0x7eff48104e50 2024-09-05T12:23:50.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.377+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7eff34042030 con 0x7eff48104e50 2024-09-05T12:23:50.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.378+0000 7eff46000640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7eff300a2c30 0x7eff300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:50.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.378+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6809+0+0 (secure 0 0 0) 0x7eff340eeec0 con 0x7eff48104e50 2024-09-05T12:23:50.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.378+0000 7eff4d800640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7eff300a2c30 0x7eff300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:50.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.379+0000 7eff4d800640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7eff300a2c30 0x7eff300a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff48104cb0 tx=0x7eff3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:50.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.382+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff340bbd10 con 0x7eff48104e50 2024-09-05T12:23:50.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:50 smithi071 ceph-mon[29845]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:50.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:50 smithi174 ceph-mon[34313]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:50.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.558+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7eff48062e90 con 0x7eff48104e50 2024-09-05T12:23:50.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.558+0000 7eff46000640 1 -- 172.21.15.71:0/3728407915 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7eff340c0010 con 0x7eff48104e50 2024-09-05T12:23:50.561 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7eff300a2c30 msgr2=0x7eff300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7eff300a2c30 0x7eff300a50f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff48104cb0 tx=0x7eff3c005f90 comp rx=0 tx=0).stop 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 msgr2=0x7eff4819a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff4819a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7eff34031440 tx=0x7eff34032ce0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 shutdown_connections 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7eff300a2c30 0x7eff300a50f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff48105810 0x7eff481a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff48104e50 0x7eff4819a430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 --2- 172.21.15.71:0/3728407915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff48103c50 0x7eff48199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:50.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 >> 172.21.15.71:0/3728407915 conn(0x7eff480ff3e0 msgr2=0x7eff481081d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:50.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 shutdown_connections 2024-09-05T12:23:50.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:50.561+0000 7eff4ec00640 1 -- 172.21.15.71:0/3728407915 wait complete. 2024-09-05T12:23:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:50 smithi123 ceph-mon[34321]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:50.989 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:23:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:23:50.726+0000 7fafefb44640 -1 osd.11 0 log_to_monitors true 2024-09-05T12:23:51.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:51 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3728407915' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T12:23:51.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:51 smithi174 ceph-mon[34313]: from='osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T12:23:51.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:51 smithi174 ceph-mon[34313]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T12:23:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:51 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3728407915' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T12:23:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:51 smithi071 ceph-mon[29845]: from='osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T12:23:51.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:51 smithi071 ceph-mon[29845]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T12:23:51.495 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1725539008,"num_in_osds":12,"osd_in_since":1725539011,"num_remapped_pgs":0} 2024-09-05T12:23:51.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:51 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3728407915' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T12:23:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:51 smithi123 ceph-mon[34321]: from='osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T12:23:51.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:51 smithi123 ceph-mon[34321]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-05T12:23:52.496 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd stat -f json 2024-09-05T12:23:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-mon[34313]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-mon[34313]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T12:23:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-mon[34313]: from='osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:23:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-mon[34313]: osdmap e76: 12 total, 11 up, 12 in 2024-09-05T12:23:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:52.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-mon[34313]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:23:52.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:23:52.226+0000 7fafebac7640 -1 osd.11 0 waiting for initial osdmap 2024-09-05T12:23:52.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:23:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:23:52.237+0000 7fafe78ef640 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-05T12:23:52.545 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:52 smithi071 ceph-mon[29845]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:52.545 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:52 smithi071 ceph-mon[29845]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T12:23:52.546 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:52 smithi071 ceph-mon[29845]: from='osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:23:52.546 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:52 smithi071 ceph-mon[29845]: osdmap e76: 12 total, 11 up, 12 in 2024-09-05T12:23:52.546 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:52 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:52.546 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:52 smithi071 ceph-mon[29845]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:23:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:52 smithi123 ceph-mon[34321]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:52 smithi123 ceph-mon[34321]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-05T12:23:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:52 smithi123 ceph-mon[34321]: from='osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:23:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:52 smithi123 ceph-mon[34321]: osdmap e76: 12 total, 11 up, 12 in 2024-09-05T12:23:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:52 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:52 smithi123 ceph-mon[34321]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-09-05T12:23:52.874 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:23:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:23:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: osdmap e77: 12 total, 11 up, 12 in 2024-09-05T12:23:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: Detected new or changed devices on smithi174 2024-09-05T12:23:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:53.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: osdmap e77: 12 total, 11 up, 12 in 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: Detected new or changed devices on smithi174 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-05T12:23:53.515 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.515 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:53.515 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:53.515 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 -- 172.21.15.71:0/2959571968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac102c40 msgr2=0x7ff8ac1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 --2- 172.21.15.71:0/2959571968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac102c40 0x7ff8ac1030c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff898009900 tx=0x7ff89802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 -- 172.21.15.71:0/2959571968 shutdown_connections 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 --2- 172.21.15.71:0/2959571968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8ac103600 0x7ff8ac109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 --2- 172.21.15.71:0/2959571968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac102c40 0x7ff8ac1030c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 --2- 172.21.15.71:0/2959571968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 -- 172.21.15.71:0/2959571968 >> 172.21.15.71:0/2959571968 conn(0x7ff8ac0fd1f0 msgr2=0x7ff8ac0ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:53.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 -- 172.21.15.71:0/2959571968 shutdown_connections 2024-09-05T12:23:53.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.517+0000 7ff8b3000640 1 -- 172.21.15.71:0/2959571968 wait complete. 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.518+0000 7ff8b3000640 1 Processor -- start 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 -- start start 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8ac102c40 0x7ff8ac0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac103600 0x7ff8ac074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10d6f0 con 0x7ff8ac103600 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10d570 con 0x7ff8ac101a40 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b3000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10d870 con 0x7ff8ac102c40 2024-09-05T12:23:53.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:59902/0 (socket says 172.21.15.71:59902) 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b1c00640 1 -- 172.21.15.71:0/752356755 learned_addr learned my addr 172.21.15.71:0/752356755 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b2600640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac103600 0x7ff8ac074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.519+0000 7ff8b1200640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8ac102c40 0x7ff8ac0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b1c00640 1 -- 172.21.15.71:0/752356755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8ac102c40 msgr2=0x7ff8ac0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b1c00640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8ac102c40 0x7ff8ac0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b1c00640 1 -- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac103600 msgr2=0x7ff8ac074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b1c00640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac103600 0x7ff8ac074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b1c00640 1 -- 172.21.15.71:0/752356755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac0753e0 con 0x7ff8ac101a40 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b1c00640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac075f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000c970 tx=0x7ff8a000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:53.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0007c10 con 0x7ff8ac101a40 2024-09-05T12:23:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.520+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac1a1de0 con 0x7ff8ac101a40 2024-09-05T12:23:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.521+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0007db0 con 0x7ff8ac101a40 2024-09-05T12:23:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.521+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000f8a0 con 0x7ff8ac101a40 2024-09-05T12:23:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.521+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1a2230 con 0x7ff8ac101a40 2024-09-05T12:23:53.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.522+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac108d10 con 0x7ff8ac101a40 2024-09-05T12:23:53.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.524+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8a0043ae0 con 0x7ff8ac101a40 2024-09-05T12:23:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.525+0000 7ff8aa000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff8940a2b80 0x7ff8940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.525+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff8a00cefc0 con 0x7ff8ac101a40 2024-09-05T12:23:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.525+0000 7ff8b1200640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff8940a2b80 0x7ff8940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:53.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.526+0000 7ff8b1200640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff8940a2b80 0x7ff8940a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff898031440 tx=0x7ff8980315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:53.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.527+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8a009c3b0 con 0x7ff8ac101a40 2024-09-05T12:23:53.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-05T12:23:53.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: osdmap e77: 12 total, 11 up, 12 in 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: Detected new or changed devices on smithi174 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:23:53.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:23:53.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.708+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff8ac02d7c0 con 0x7ff8ac101a40 2024-09-05T12:23:53.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.709+0000 7ff8aa000640 1 -- 172.21.15.71:0/752356755 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7ff8a009bcb0 con 0x7ff8ac101a40 2024-09-05T12:23:53.711 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:23:53.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff8940a2b80 msgr2=0x7ff8940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:53.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff8940a2b80 0x7ff8940a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff898031440 tx=0x7ff8980315b0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 msgr2=0x7ff8ac075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:53.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac075f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000c970 tx=0x7ff8a000ce30 comp rx=0 tx=0).stop 2024-09-05T12:23:53.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 shutdown_connections 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff8940a2b80 0x7ff8940a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8ac103600 0x7ff8ac074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8ac102c40 0x7ff8ac0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 --2- 172.21.15.71:0/752356755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8ac101a40 0x7ff8ac075f90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 >> 172.21.15.71:0/752356755 conn(0x7ff8ac0fd1f0 msgr2=0x7ff8ac105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 shutdown_connections 2024-09-05T12:23:53.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:53.711+0000 7ff8b3000640 1 -- 172.21.15.71:0/752356755 wait complete. 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: purged_snaps scrub starts 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: purged_snaps scrub ok 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] boot 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:54 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/752356755' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T12:23:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: purged_snaps scrub starts 2024-09-05T12:23:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: purged_snaps scrub ok 2024-09-05T12:23:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] boot 2024-09-05T12:23:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T12:23:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:54.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:54 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/752356755' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T12:23:54.645 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1725539033,"num_in_osds":12,"osd_in_since":1725539011,"num_remapped_pgs":0} 2024-09-05T12:23:54.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:23:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: purged_snaps scrub starts 2024-09-05T12:23:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: purged_snaps scrub ok 2024-09-05T12:23:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-09-05T12:23:54.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: osd.11 [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] boot 2024-09-05T12:23:54.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: osdmap e78: 12 total, 12 up, 12 in 2024-09-05T12:23:54.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:23:54.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:54 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/752356755' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-05T12:23:55.009 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:23:55.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:55 smithi071 ceph-mon[29845]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T12:23:55.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:55 smithi071 ceph-mon[29845]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.612+0000 7feb17000640 1 -- 172.21.15.71:0/192803478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 msgr2=0x7feb10101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.612+0000 7feb17000640 1 --2- 172.21.15.71:0/192803478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10101e20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7feafc009930 tx=0x7feafc02a420 comp rx=0 tx=0).stop 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 -- 172.21.15.71:0/192803478 shutdown_connections 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 --2- 172.21.15.71:0/192803478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb10102360 0x7feb1010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 --2- 172.21.15.71:0/192803478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10101e20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 --2- 172.21.15.71:0/192803478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb101079c0 0x7feb10107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 -- 172.21.15.71:0/192803478 >> 172.21.15.71:0/192803478 conn(0x7feb100fd740 msgr2=0x7feb100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 -- 172.21.15.71:0/192803478 shutdown_connections 2024-09-05T12:23:55.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.613+0000 7feb17000640 1 -- 172.21.15.71:0/192803478 wait complete. 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 Processor -- start 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 -- start start 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb10102360 0x7feb10198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb101079c0 0x7feb1019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111e10 con 0x7feb101019c0 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111c90 con 0x7feb101079c0 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb17000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb10111f90 con 0x7feb10102360 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb15c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb15c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42380/0 (socket says 172.21.15.71:42380) 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb15c00640 1 -- 172.21.15.71:0/1728189000 learned_addr learned my addr 172.21.15.71:0/1728189000 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.614+0000 7feb15200640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb10102360 0x7feb10198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:55.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb16600640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb101079c0 0x7feb1019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb15c00640 1 -- 172.21.15.71:0/1728189000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb10102360 msgr2=0x7feb10198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb15c00640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb10102360 0x7feb10198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb15c00640 1 -- 172.21.15.71:0/1728189000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb101079c0 msgr2=0x7feb1019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb15c00640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb101079c0 0x7feb1019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb15c00640 1 -- 172.21.15.71:0/1728189000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1019d300 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb15c00640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10198350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d770 tx=0x7feb0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04017070 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb04005500 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb04004e00 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.615+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1019d5f0 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.616+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb101a4ed0 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.617+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb10101e20 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.619+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feb04004fa0 con 0x7feb101019c0 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.620+0000 7feb0e000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feaf80a2c00 0x7feaf80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:55.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.620+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7feb040ce4f0 con 0x7feb101019c0 2024-09-05T12:23:55.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.620+0000 7feb15200640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feaf80a2c00 0x7feaf80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:55.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.621+0000 7feb15200640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feaf80a2c00 0x7feaf80a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feafc006010 tx=0x7feafc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:55.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.625+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb0409b200 con 0x7feb101019c0 2024-09-05T12:23:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:55 smithi123 ceph-mon[34321]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T12:23:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:55 smithi123 ceph-mon[34321]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:55 smithi174 ceph-mon[34313]: osdmap e79: 12 total, 12 up, 12 in 2024-09-05T12:23:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:55 smithi174 ceph-mon[34313]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:55.805 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.803+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb10109f30 con 0x7feb101019c0 2024-09-05T12:23:55.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.804+0000 7feb0e000640 1 -- 172.21.15.71:0/1728189000 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+18470 (secure 0 0 0) 0x7feb0409f500 con 0x7feb101019c0 2024-09-05T12:23:55.806 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:23:55.807 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":79,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:23:54.316708+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feaf80a2c00 msgr2=0x7feaf80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feaf80a2c00 0x7feaf80a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feafc006010 tx=0x7feafc000bc0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 msgr2=0x7feb10198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10198350 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d770 tx=0x7feb0400dc30 comp rx=0 tx=0).stop 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 shutdown_connections 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feaf80a2c00 0x7feaf80a50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb101079c0 0x7feb1019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb10102360 0x7feb10198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.806+0000 7feb17000640 1 --2- 172.21.15.71:0/1728189000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb101019c0 0x7feb10198350 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.807+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 >> 172.21.15.71:0/1728189000 conn(0x7feb100fd740 msgr2=0x7feb100fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:55.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.807+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 shutdown_connections 2024-09-05T12:23:55.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:55.807+0000 7feb17000640 1 -- 172.21.15.71:0/1728189000 wait complete. 2024-09-05T12:23:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1728189000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:23:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1728189000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:23:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1728189000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:23:56.826 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T12:19:53.639902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-05T12:23:56.827 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-05T12:23:57.187 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:23:57.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:57 smithi071 ceph-mon[29845]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:57.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:57 smithi123 ceph-mon[34321]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:57.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:57 smithi174 ceph-mon[34313]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 -- 172.21.15.71:0/3922064882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c103ff0 msgr2=0x7f124c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 --2- 172.21.15.71:0/3922064882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c103ff0 0x7f124c1043d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1240009900 tx=0x7f124002a410 comp rx=0 tx=0).stop 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 -- 172.21.15.71:0/3922064882 shutdown_connections 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 --2- 172.21.15.71:0/3922064882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c109120 0x7f124c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 --2- 172.21.15.71:0/3922064882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c1049a0 0x7f124c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 --2- 172.21.15.71:0/3922064882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c103ff0 0x7f124c1043d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 -- 172.21.15.71:0/3922064882 >> 172.21.15.71:0/3922064882 conn(0x7f124c0ffa90 msgr2=0x7f124c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 -- 172.21.15.71:0/3922064882 shutdown_connections 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.783+0000 7f1253400640 1 -- 172.21.15.71:0/3922064882 wait complete. 2024-09-05T12:23:57.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 Processor -- start 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 -- start start 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c103ff0 0x7f124c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 0x7f124c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c109120 0x7f124c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f124c114140 con 0x7f124c1049a0 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f124c113fc0 con 0x7f124c103ff0 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.784+0000 7f1253400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f124c1142c0 con 0x7f124c109120 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 0x7f124c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 0x7f124c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42408/0 (socket says 172.21.15.71:42408) 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 -- 172.21.15.71:0/1597190586 learned_addr learned my addr 172.21.15.71:0/1597190586 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f1251400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c109120 0x7f124c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f1250a00640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c103ff0 0x7f124c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 -- 172.21.15.71:0/1597190586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c109120 msgr2=0x7f124c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:57.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c109120 0x7f124c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 -- 172.21.15.71:0/1597190586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c103ff0 msgr2=0x7f124c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c103ff0 0x7f124c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 -- 172.21.15.71:0/1597190586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f124c19f5b0 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f1251400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c109120 0x7f124c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.785+0000 7f124be00640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 0x7f124c19ab40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f123800e9f0 tx=0x7f123800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.786+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f123800cd80 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.786+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1238005500 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.786+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f124c19f8a0 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.786+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238010690 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.786+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f124c1a7180 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.787+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f124c1043d0 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.789+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f12380046c0 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.790+0000 7f1249600640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f12340a2b80 0x7f12340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.790+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f12380cefe0 con 0x7f124c1049a0 2024-09-05T12:23:57.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.790+0000 7f1250a00640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f12340a2b80 0x7f12340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.791+0000 7f1250a00640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f12340a2b80 0x7f12340a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1240009230 tx=0x7f12400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:57.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.794+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f123809c380 con 0x7f124c1049a0 2024-09-05T12:23:57.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.968+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f124c02d7c0 con 0x7f124c1049a0 2024-09-05T12:23:57.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.968+0000 7f1249600640 1 -- 172.21.15.71:0/1597190586 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f123809bc80 con 0x7f124c1049a0 2024-09-05T12:23:57.970 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-05T12:23:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f12340a2b80 msgr2=0x7f12340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f12340a2b80 0x7f12340a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1240009230 tx=0x7f12400058d0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 msgr2=0x7f124c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 0x7f124c19ab40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f123800e9f0 tx=0x7f123800edf0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 shutdown_connections 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f12340a2b80 0x7f12340a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f124c109120 0x7f124c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f124c1049a0 0x7f124c19ab40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 --2- 172.21.15.71:0/1597190586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f124c103ff0 0x7f124c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.971+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 >> 172.21.15.71:0/1597190586 conn(0x7f124c0ffa90 msgr2=0x7f124c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.972+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 shutdown_connections 2024-09-05T12:23:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:23:57.972+0000 7f1253400640 1 -- 172.21.15.71:0/1597190586 wait complete. 2024-09-05T12:23:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:58 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1597190586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:23:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:58 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1597190586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:23:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:58 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1597190586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:23:58.764 INFO:tasks.cephadm:Adding mds.a on smithi071 2024-09-05T12:23:58.764 INFO:tasks.cephadm:Adding mds.d on smithi071 2024-09-05T12:23:58.764 INFO:tasks.cephadm:Adding mds.g on smithi071 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.j on smithi071 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.b on smithi123 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.e on smithi123 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.h on smithi123 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.k on smithi123 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-09-05T12:23:58.765 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-09-05T12:23:58.765 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph orch apply mds all '12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-09-05T12:23:59.024 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:23:59.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:23:59 smithi174 ceph-mon[34313]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 -- 172.21.15.174:0/3557535367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c101c00 msgr2=0x7f021c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 --2- 172.21.15.174:0/3557535367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c101c00 0x7f021c102000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f020c009900 tx=0x7f020c02a410 comp rx=0 tx=0).stop 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 -- 172.21.15.174:0/3557535367 shutdown_connections 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 --2- 172.21.15.174:0/3557535367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f021c1037c0 0x7f021c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 --2- 172.21.15.174:0/3557535367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f021c102e00 0x7f021c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 --2- 172.21.15.174:0/3557535367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c101c00 0x7f021c102000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 -- 172.21.15.174:0/3557535367 >> 172.21.15.174:0/3557535367 conn(0x7f021c0fd390 msgr2=0x7f021c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 -- 172.21.15.174:0/3557535367 shutdown_connections 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 -- 172.21.15.174:0/3557535367 wait complete. 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.500+0000 7f0223bbe640 1 Processor -- start 2024-09-05T12:23:59.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 -- start start 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f021c101c00 0x7f021c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f021c102e00 0x7f021c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 0x7f021c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10d960 con 0x7f021c102e00 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10d7e0 con 0x7f021c101c00 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f021c10dae0 con 0x7f021c1037c0 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 0x7f021c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 0x7f021c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:60310/0 (socket says 172.21.15.174:60310) 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 -- 172.21.15.174:0/1960703098 learned_addr learned my addr 172.21.15.174:0/1960703098 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022195a640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f021c101c00 0x7f021c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0221159640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f021c102e00 0x7f021c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 -- 172.21.15.174:0/1960703098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f021c101c00 msgr2=0x7f021c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f021c101c00 0x7f021c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 -- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f021c102e00 msgr2=0x7f021c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f021c102e00 0x7f021c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 -- 172.21.15.174:0/1960703098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f021c0753e0 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f022215b640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 0x7f021c074b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f021400b970 tx=0x7f021400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0214019070 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.501+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f021c1a2230 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.502+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f021c1a27c0 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.502+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f021400cb40 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.502+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021401d5f0 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.503+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02000052c0 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.504+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f021401c070 con 0x7f021c1037c0 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.505+0000 7f0212ffd640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02080a2c00 0x7f02080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:23:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.505+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f02140ce5a0 con 0x7f021c1037c0 2024-09-05T12:23:59.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.505+0000 7f022195a640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02080a2c00 0x7f02080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:23:59.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.506+0000 7f022195a640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02080a2c00 0x7f02080a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f020c006010 tx=0x7f020c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:23:59.517 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.508+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f021409b900 con 0x7f021c1037c0 2024-09-05T12:23:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:23:59 smithi071 ceph-mon[29845]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:23:59 smithi123 ceph-mon[34321]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:23:59.694 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.693+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7f0200000cf0 con 0x7f02080a2c00 2024-09-05T12:23:59.703 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.702+0000 7f0212ffd640 1 -- 172.21.15.174:0/1960703098 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0200000cf0 con 0x7f02080a2c00 2024-09-05T12:23:59.703 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-09-05T12:23:59.705 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02080a2c00 msgr2=0x7f02080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02080a2c00 0x7f02080a50c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f020c006010 tx=0x7f020c000bc0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 msgr2=0x7f021c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 0x7f021c074b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f021400b970 tx=0x7f021400be30 comp rx=0 tx=0).stop 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 shutdown_connections 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02080a2c00 0x7f02080a50c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f021c1037c0 0x7f021c074b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f021c102e00 0x7f021c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 --2- 172.21.15.174:0/1960703098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f021c101c00 0x7f021c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 >> 172.21.15.174:0/1960703098 conn(0x7f021c0fd390 msgr2=0x7f021c106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 shutdown_connections 2024-09-05T12:23:59.706 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-05T12:23:59.704+0000 7f0223bbe640 1 -- 172.21.15.174:0/1960703098 wait complete. 2024-09-05T12:24:00.460 DEBUG:teuthology.orchestra.run.smithi071:mds.a> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.a.service 2024-09-05T12:24:00.465 DEBUG:teuthology.orchestra.run.smithi071:mds.d> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.d.service 2024-09-05T12:24:00.509 DEBUG:teuthology.orchestra.run.smithi071:mds.g> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.g.service 2024-09-05T12:24:00.555 DEBUG:teuthology.orchestra.run.smithi071:mds.j> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.j.service 2024-09-05T12:24:00.600 DEBUG:teuthology.orchestra.run.smithi123:mds.b> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.b.service 2024-09-05T12:24:00.603 DEBUG:teuthology.orchestra.run.smithi123:mds.e> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.e.service 2024-09-05T12:24:00.646 DEBUG:teuthology.orchestra.run.smithi123:mds.h> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.h.service 2024-09-05T12:24:00.689 DEBUG:teuthology.orchestra.run.smithi123:mds.k> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.k.service 2024-09-05T12:24:00.732 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.c.service 2024-09-05T12:24:00.735 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.f.service 2024-09-05T12:24:00.778 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.i.service 2024-09-05T12:24:00.820 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.l.service 2024-09-05T12:24:00.863 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-05T12:24:00.864 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: Saving service mds.all spec with placement smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:00.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:00.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:00 smithi071 ceph-mon[29845]: Deploying daemon mds.l on smithi174 2024-09-05T12:24:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:24:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: Saving service mds.all spec with placement smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-05T12:24:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:00 smithi123 ceph-mon[34321]: Deploying daemon mds.l on smithi174 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: Saving service mds.all spec with placement smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:00.998 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:00.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:00.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:00.999 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:00 smithi174 ceph-mon[34313]: Deploying daemon mds.l on smithi174 2024-09-05T12:24:01.219 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:01.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 -- 172.21.15.71:0/2479220878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81019c0 msgr2=0x7fc6d8101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:01.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2479220878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81019c0 0x7fc6d8101e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009890 tx=0x7fc6c002a360 comp rx=0 tx=0).stop 2024-09-05T12:24:01.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 -- 172.21.15.71:0/2479220878 shutdown_connections 2024-09-05T12:24:01.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2479220878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6d8102360 0x7fc6d810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2479220878 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81019c0 0x7fc6d8101e20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2479220878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81079c0 0x7fc6d8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 -- 172.21.15.71:0/2479220878 >> 172.21.15.71:0/2479220878 conn(0x7fc6d80fd740 msgr2=0x7fc6d80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 -- 172.21.15.71:0/2479220878 shutdown_connections 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.772+0000 7fc6dd600640 1 -- 172.21.15.71:0/2479220878 wait complete. 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 Processor -- start 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 -- start start 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81019c0 0x7fc6d819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6d8102360 0x7fc6d819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 0x7fc6d819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8111dc0 con 0x7fc6d81079c0 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8111c40 con 0x7fc6d81019c0 2024-09-05T12:24:01.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.773+0000 7fc6dd600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d8111f40 con 0x7fc6d8102360 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 0x7fc6d819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 0x7fc6d819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42424/0 (socket says 172.21.15.71:42424) 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6cfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6d8102360 0x7fc6d819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 -- 172.21.15.71:0/2200324446 learned_addr learned my addr 172.21.15.71:0/2200324446 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6d7e00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81019c0 0x7fc6d819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 -- 172.21.15.71:0/2200324446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6d8102360 msgr2=0x7fc6d819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6d8102360 0x7fc6d819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 -- 172.21.15.71:0/2200324446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81019c0 msgr2=0x7fc6d819a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81019c0 0x7fc6d819a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 -- 172.21.15.71:0/2200324446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d819f4d0 con 0x7fc6d81079c0 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6d7e00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81019c0 0x7fc6d819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:24:01.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dcc00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 0x7fc6d819edf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000efc0 tx=0x7fc6d000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0003970 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6d0004020 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d819f7c0 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.774+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d000b9f0 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.775+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d81a70a0 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.776+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d8101e20 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.777+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc6d001d020 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.778+0000 7fc6d5600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc6bc0a2b80 0x7fc6bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.779+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fc6d00ceb60 con 0x7fc6d81079c0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.779+0000 7fc6d7e00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc6bc0a2b80 0x7fc6bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.779+0000 7fc6d7e00640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc6bc0a2b80 0x7fc6bc0a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009ba0 tx=0x7fc6c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:01.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.783+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6d009bec0 con 0x7fc6d81079c0 2024-09-05T12:24:01.956 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:01 smithi174 ceph-mon[34313]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:01.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.992+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6d81a7270 con 0x7fc6d81079c0 2024-09-05T12:24:01.994 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:01 smithi071 ceph-mon[29845]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.993+0000 7fc6d5600640 1 -- 172.21.15.71:0/2200324446 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fc6d009b7c0 con 0x7fc6d81079c0 2024-09-05T12:24:01.995 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:01.995 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"btime":"2024-09-05T12:16:33:931477+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-05T12:24:01.995 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 1 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc6bc0a2b80 msgr2=0x7fc6bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc6bc0a2b80 0x7fc6bc0a5040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009ba0 tx=0x7fc6c8005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 msgr2=0x7fc6d819edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 0x7fc6d819edf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000efc0 tx=0x7fc6d000c800 comp rx=0 tx=0).stop 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 shutdown_connections 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc6bc0a2b80 0x7fc6bc0a5040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc6d81079c0 0x7fc6d819edf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6d8102360 0x7fc6d819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 --2- 172.21.15.71:0/2200324446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d81019c0 0x7fc6d819a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:01.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 >> 172.21.15.71:0/2200324446 conn(0x7fc6d80fd740 msgr2=0x7fc6d80fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:01.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 shutdown_connections 2024-09-05T12:24:01.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:01.995+0000 7fc6dd600640 1 -- 172.21.15.71:0/2200324446 wait complete. 2024-09-05T12:24:02.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:01 smithi123 ceph-mon[34321]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:02.260 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 systemd[1]: Starting Ceph mds.l for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:02.621 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 podman[63465]: 2024-09-05 12:24:02.247737327 +0000 UTC m=+0.015591798 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:02.621 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 podman[63465]: 2024-09-05 12:24:02.368752141 +0000 UTC m=+0.136606597 container create 4708f31af45f58a5cd8bfd0151df6544c3994c0df8847ea1335b29d192ae7db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-05T12:24:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:02 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2200324446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:02 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6832/679053656,v1:172.21.15.174:6833/679053656] up:boot 2024-09-05T12:24:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:02 smithi071 ceph-mon[29845]: fsmap 1 up:standby 2024-09-05T12:24:02.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:02 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:24:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:02 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2200324446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:02 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6832/679053656,v1:172.21.15.174:6833/679053656] up:boot 2024-09-05T12:24:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:02 smithi174 ceph-mon[34313]: fsmap 1 up:standby 2024-09-05T12:24:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:02 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:24:03.011 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 podman[63465]: 2024-09-05 12:24:02.621336462 +0000 UTC m=+0.389190919 container init 4708f31af45f58a5cd8bfd0151df6544c3994c0df8847ea1335b29d192ae7db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS) 2024-09-05T12:24:03.011 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 podman[63465]: 2024-09-05 12:24:02.624277433 +0000 UTC m=+0.392131897 container start 4708f31af45f58a5cd8bfd0151df6544c3994c0df8847ea1335b29d192ae7db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:24:03.011 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 ceph-mds[63485]: -- 172.21.15.174:0/4050671167 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad7f9db860 con 0x55ad7f9b8800 2024-09-05T12:24:03.011 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l[63481]: starting mds.l at 2024-09-05T12:24:03.011 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 bash[63465]: 4708f31af45f58a5cd8bfd0151df6544c3994c0df8847ea1335b29d192ae7db9 2024-09-05T12:24:03.011 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:24:02 smithi174 systemd[1]: Started Ceph mds.l for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:03.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:02 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2200324446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:03.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:02 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6832/679053656,v1:172.21.15.174:6833/679053656] up:boot 2024-09-05T12:24:03.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:02 smithi123 ceph-mon[34321]: fsmap 1 up:standby 2024-09-05T12:24:03.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:02 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:24:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: Deploying daemon mds.k on smithi123 2024-09-05T12:24:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:03 smithi123 ceph-mon[34321]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: Deploying daemon mds.k on smithi123 2024-09-05T12:24:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:03 smithi071 ceph-mon[29845]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: Deploying daemon mds.k on smithi123 2024-09-05T12:24:04.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:03 smithi174 ceph-mon[34313]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:05.065 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:05.161 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:04 smithi123 systemd[1]: Starting Ceph mds.k for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:05.420 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 podman[63883]: 2024-09-05 12:24:05.062639392 +0000 UTC m=+0.015440410 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:05.420 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 podman[63883]: 2024-09-05 12:24:05.200478909 +0000 UTC m=+0.153279915 container create 3da75bb26ba549f87cb37fb7c684b3a1d4b133fc34826313a001afa87c8919c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-05T12:24:05.420 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 podman[63883]: 2024-09-05 12:24:05.419868713 +0000 UTC m=+0.372669717 container init 3da75bb26ba549f87cb37fb7c684b3a1d4b133fc34826313a001afa87c8919c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:24:05.441 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:05.679 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 podman[63883]: 2024-09-05 12:24:05.422801101 +0000 UTC m=+0.375602110 container start 3da75bb26ba549f87cb37fb7c684b3a1d4b133fc34826313a001afa87c8919c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:24:05.679 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 ceph-mds[63902]: -- 172.21.15.123:0/1669884420 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae6ac6f860 con 0x55ae6ac4d000 2024-09-05T12:24:05.680 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k[63898]: starting mds.k at 2024-09-05T12:24:05.680 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 bash[63883]: 3da75bb26ba549f87cb37fb7c684b3a1d4b133fc34826313a001afa87c8919c4 2024-09-05T12:24:05.680 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:24:05 smithi123 systemd[1]: Started Ceph mds.k for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:06.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 -- 172.21.15.71:0/1593893027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0112b90 msgr2=0x7f8ee0114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 --2- 172.21.15.71:0/1593893027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0112b90 0x7f8ee0114f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8009ca0 tx=0x7f8ed802a950 comp rx=0 tx=0).stop 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 -- 172.21.15.71:0/1593893027 shutdown_connections 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 --2- 172.21.15.71:0/1593893027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0112b90 0x7f8ee0114f80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 --2- 172.21.15.71:0/1593893027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee0074eb0 0x7f8ee0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 --2- 172.21.15.71:0/1593893027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074590 0x7f8ee0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 -- 172.21.15.71:0/1593893027 >> 172.21.15.71:0/1593893027 conn(0x7f8ee00fd8c0 msgr2=0x7f8ee00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 -- 172.21.15.71:0/1593893027 shutdown_connections 2024-09-05T12:24:06.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.015+0000 7f8ee7000640 1 -- 172.21.15.71:0/1593893027 wait complete. 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.016+0000 7f8ee7000640 1 Processor -- start 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.016+0000 7f8ee7000640 1 -- start start 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0074590 0x7f8ee019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074eb0 0x7f8ee019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee7000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee0112b90 0x7f8ee01a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee01185c0 con 0x7f8ee0074590 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0118440 con 0x7f8ee0074eb0 2024-09-05T12:24:06.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee7000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee0118740 con 0x7f8ee0112b90 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0074590 0x7f8ee019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0074590 0x7f8ee019eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39948/0 (socket says 172.21.15.71:39948) 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edfe00640 1 -- 172.21.15.71:0/881120907 learned_addr learned my addr 172.21.15.71:0/881120907 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edf400640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074eb0 0x7f8ee019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8ee5000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee0112b90 0x7f8ee01a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edf400640 1 -- 172.21.15.71:0/881120907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee0112b90 msgr2=0x7f8ee01a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edf400640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee0112b90 0x7f8ee01a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edf400640 1 -- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0074590 msgr2=0x7f8ee019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edf400640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0074590 0x7f8ee019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.017+0000 7f8edf400640 1 -- 172.21.15.71:0/881120907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee01a3b40 con 0x7f8ee0074eb0 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.018+0000 7f8edf400640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074eb0 0x7f8ee019f050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00e9f0 tx=0x7f8ecc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:06.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.018+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc00ccc0 con 0x7f8ee0074eb0 2024-09-05T12:24:06.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.018+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ecc005020 con 0x7f8ee0074eb0 2024-09-05T12:24:06.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.018+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee01a3e30 con 0x7f8ee0074eb0 2024-09-05T12:24:06.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.018+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0049e0 con 0x7f8ee0074eb0 2024-09-05T12:24:06.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.018+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee01ab6c0 con 0x7f8ee0074eb0 2024-09-05T12:24:06.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.019+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee0074970 con 0x7f8ee0074eb0 2024-09-05T12:24:06.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.023+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ecc005560 con 0x7f8ee0074eb0 2024-09-05T12:24:06.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.024+0000 7f8edcc00640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8ec40a2c00 0x7f8ec40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:06.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.024+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f8ecc0ceac0 con 0x7f8ee0074eb0 2024-09-05T12:24:06.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.024+0000 7f8edfe00640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8ec40a2c00 0x7f8ec40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:06.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.025+0000 7f8edfe00640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8ec40a2c00 0x7f8ec40a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009c90 tx=0x7f8ed0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:06.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.028+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ecc0051d0 con 0x7f8ee0074eb0 2024-09-05T12:24:06.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.269+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ee0104fd0 con 0x7f8ee0074eb0 2024-09-05T12:24:06.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.269+0000 7f8edcc00640 1 -- 172.21.15.71:0/881120907 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7f8ecc09be20 con 0x7f8ee0074eb0 2024-09-05T12:24:06.271 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:06.272 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"btime":"2024-09-05T12:24:06:178952+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T12:24:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 3 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.271+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8ec40a2c00 msgr2=0x7f8ec40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.271+0000 7f8ee7000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8ec40a2c00 0x7f8ec40a50c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0009c90 tx=0x7f8ed0005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074eb0 msgr2=0x7f8ee019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074eb0 0x7f8ee019f050 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00e9f0 tx=0x7f8ecc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 shutdown_connections 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8ec40a2c00 0x7f8ec40a50c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee0112b90 0x7f8ee01a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ee0074eb0 0x7f8ee019f050 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 --2- 172.21.15.71:0/881120907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ee0074590 0x7f8ee019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 >> 172.21.15.71:0/881120907 conn(0x7f8ee00fd8c0 msgr2=0x7f8ee00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 shutdown_connections 2024-09-05T12:24:06.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:06.272+0000 7f8ee7000640 1 -- 172.21.15.71:0/881120907 wait complete. 2024-09-05T12:24:06.460 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:06.460 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.460 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.460 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.460 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:06.461 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:06.461 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:06 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:06.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:06 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:06.585 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:06.585 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.585 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.586 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:06.586 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:06.586 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:06.586 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:06 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:07.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:07 smithi071 ceph-mon[29845]: Deploying daemon mds.j on smithi071 2024-09-05T12:24:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:07 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6832/1436525377,v1:172.21.15.123:6833/1436525377] up:boot 2024-09-05T12:24:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:07 smithi071 ceph-mon[29845]: fsmap 2 up:standby 2024-09-05T12:24:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:07 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:24:07.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/881120907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:07 smithi123 ceph-mon[34321]: Deploying daemon mds.j on smithi071 2024-09-05T12:24:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:07 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6832/1436525377,v1:172.21.15.123:6833/1436525377] up:boot 2024-09-05T12:24:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:07 smithi123 ceph-mon[34321]: fsmap 2 up:standby 2024-09-05T12:24:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:07 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:24:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/881120907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:07 smithi174 ceph-mon[34313]: Deploying daemon mds.j on smithi071 2024-09-05T12:24:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:07 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6832/1436525377,v1:172.21.15.123:6833/1436525377] up:boot 2024-09-05T12:24:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:07 smithi174 ceph-mon[34313]: fsmap 2 up:standby 2024-09-05T12:24:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:07 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:24:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/881120907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:08.451 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:08 smithi071 ceph-mon[29845]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:08 smithi174 ceph-mon[34313]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:08 smithi123 ceph-mon[34321]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:09.306 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:08 smithi071 systemd[1]: Starting Ceph mds.j for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:09.628 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 podman[69352]: 2024-09-05 12:24:09.209582497 +0000 UTC m=+0.027451516 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:09.628 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 podman[69352]: 2024-09-05 12:24:09.332948452 +0000 UTC m=+0.150817469 container create ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-05T12:24:09.628 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 podman[69352]: 2024-09-05 12:24:09.62753127 +0000 UTC m=+0.445400287 container init ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:24:09.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 podman[69352]: 2024-09-05 12:24:09.635582596 +0000 UTC m=+0.453451618 container start ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-05T12:24:09.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 ceph-mds[69371]: -- 172.21.15.71:0/1338297565 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5630dfe35860 con 0x5630dfe13400 2024-09-05T12:24:09.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j[69367]: starting mds.j at 2024-09-05T12:24:09.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 bash[69352]: ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a 2024-09-05T12:24:09.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:24:09 smithi071 systemd[1]: Started Ceph mds.j for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:09.972 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:10.360 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:10.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:10.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:10 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:10 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:10 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:11 smithi071 ceph-mon[29845]: Deploying daemon mds.d on smithi071 2024-09-05T12:24:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:11 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6834/83855172,v1:172.21.15.71:6835/83855172] up:boot 2024-09-05T12:24:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:11 smithi071 ceph-mon[29845]: fsmap 3 up:standby 2024-09-05T12:24:11.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:11 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T12:24:11.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:11 smithi174 ceph-mon[34313]: Deploying daemon mds.d on smithi071 2024-09-05T12:24:11.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:11 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6834/83855172,v1:172.21.15.71:6835/83855172] up:boot 2024-09-05T12:24:11.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:11 smithi174 ceph-mon[34313]: fsmap 3 up:standby 2024-09-05T12:24:11.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:11 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T12:24:11.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:11 smithi123 ceph-mon[34321]: Deploying daemon mds.d on smithi071 2024-09-05T12:24:11.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:11 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6834/83855172,v1:172.21.15.71:6835/83855172] up:boot 2024-09-05T12:24:11.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:11 smithi123 ceph-mon[34321]: fsmap 3 up:standby 2024-09-05T12:24:11.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:11 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T12:24:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:12 smithi071 ceph-mon[29845]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:12 smithi174 ceph-mon[34313]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 -- 172.21.15.71:0/1650306094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 msgr2=0x7fd40c10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 --2- 172.21.15.71:0/1650306094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 0x7fd40c10e460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd404009ca0 tx=0x7fd40402a950 comp rx=0 tx=0).stop 2024-09-05T12:24:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 -- 172.21.15.71:0/1650306094 shutdown_connections 2024-09-05T12:24:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 --2- 172.21.15.71:0/1650306094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 0x7fd40c10e460 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 --2- 172.21.15.71:0/1650306094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c1034b0 0x7fd40c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 --2- 172.21.15.71:0/1650306094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c102b90 0x7fd40c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 -- 172.21.15.71:0/1650306094 >> 172.21.15.71:0/1650306094 conn(0x7fd40c0fc630 msgr2=0x7fd40c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:12.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.644+0000 7fd40be00640 1 -- 172.21.15.71:0/1650306094 shutdown_connections 2024-09-05T12:24:12.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.645+0000 7fd40be00640 1 -- 172.21.15.71:0/1650306094 wait complete. 2024-09-05T12:24:12.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.645+0000 7fd40be00640 1 Processor -- start 2024-09-05T12:24:12.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 -- start start 2024-09-05T12:24:12.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c102b90 0x7fd40c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:12.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 0x7fd40c19a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:12.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 0x7fd40c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:12.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c111e10 con 0x7fd40c10bfc0 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c111c90 con 0x7fd40c1034b0 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40be00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd40c111f90 con 0x7fd40c102b90 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 0x7fd40c19a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c102b90 0x7fd40c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 0x7fd40c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 0x7fd40c19a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:46642/0 (socket says 172.21.15.71:46642) 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.646+0000 7fd40a000640 1 -- 172.21.15.71:0/2837348961 learned_addr learned my addr 172.21.15.71:0/2837348961 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40a000640 1 -- 172.21.15.71:0/2837348961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c102b90 msgr2=0x7fd40c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40a000640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c102b90 0x7fd40c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40a000640 1 -- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 msgr2=0x7fd40c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40a000640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 0x7fd40c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40a000640 1 -- 172.21.15.71:0/2837348961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd40c19f410 con 0x7fd40c1034b0 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40aa00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c102b90 0x7fd40c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:12.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40a000640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 0x7fd40c19a9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800e9f0 tx=0x7fd3f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:12.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800ccc0 con 0x7fd40c1034b0 2024-09-05T12:24:12.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3f800ce60 con 0x7fd40c1034b0 2024-09-05T12:24:12.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.647+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40c19f700 con 0x7fd40c1034b0 2024-09-05T12:24:12.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.648+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8010690 con 0x7fd40c1034b0 2024-09-05T12:24:12.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.648+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd40c1a6fe0 con 0x7fd40c1034b0 2024-09-05T12:24:12.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.649+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40c102f70 con 0x7fd40c1034b0 2024-09-05T12:24:12.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.652+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3f80046c0 con 0x7fd40c1034b0 2024-09-05T12:24:12.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.653+0000 7fd403400640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3f00a2b80 0x7fd3f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:12.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.653+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fd3f80cef50 con 0x7fd40c1034b0 2024-09-05T12:24:12.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.654+0000 7fd40aa00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3f00a2b80 0x7fd3f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:12.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.654+0000 7fd40aa00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3f00a2b80 0x7fd3f00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009fc0 tx=0x7fd3fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:12.658 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.657+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3f809c2b0 con 0x7fd40c1034b0 2024-09-05T12:24:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:12 smithi123 ceph-mon[34321]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:12.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.897+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd40c1a7240 con 0x7fd40c1034b0 2024-09-05T12:24:12.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.898+0000 7fd403400640 1 -- 172.21.15.71:0/2837348961 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fd3f809bbb0 con 0x7fd40c1034b0 2024-09-05T12:24:12.899 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:12.899 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":4,"btime":"2024-09-05T12:24:10:194032+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-05T12:24:12.899 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 4 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3f00a2b80 msgr2=0x7fd3f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3f00a2b80 0x7fd3f00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009fc0 tx=0x7fd3fc005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 msgr2=0x7fd40c19a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 0x7fd40c19a9a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800e9f0 tx=0x7fd3f800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 shutdown_connections 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3f00a2b80 0x7fd3f00a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd40c10bfc0 0x7fd40c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.901 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd40c1034b0 0x7fd40c19a9a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 --2- 172.21.15.71:0/2837348961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd40c102b90 0x7fd40c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:12.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 >> 172.21.15.71:0/2837348961 conn(0x7fd40c0fc630 msgr2=0x7fd40c10d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:12.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 shutdown_connections 2024-09-05T12:24:12.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:12.900+0000 7fd40be00640 1 -- 172.21.15.71:0/2837348961 wait complete. 2024-09-05T12:24:13.507 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:13 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2837348961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:13.507 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:24:13 smithi071 podman[69746]: 2024-09-05 12:24:13.469366893 +0000 UTC m=+0.685653391 container start 1b24c4f7298180850da42e69a3bc2d4e657d52cd9006d91786897e66294e6316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:24:13.508 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:24:13 smithi071 ceph-mds[69777]: -- 172.21.15.71:0/1711222613 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56084e4cfc20 con 0x56084e4ad400 2024-09-05T12:24:13.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:13 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2837348961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:13.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:13 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2837348961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:13.835 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:24:13 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d[69773]: starting mds.d at 2024-09-05T12:24:13.836 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:24:13 smithi071 bash[69746]: 1b24c4f7298180850da42e69a3bc2d4e657d52cd9006d91786897e66294e6316 2024-09-05T12:24:13.836 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:24:13 smithi071 systemd[1]: Started Ceph mds.d for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:14.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:14.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.358 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.358 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.358 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:14.358 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:14.358 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:14 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:14.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:14 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:14.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:14 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:15.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:15 smithi123 ceph-mon[34321]: Deploying daemon mds.e on smithi123 2024-09-05T12:24:15.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:15 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6836/2997614259,v1:172.21.15.71:6837/2997614259] up:boot 2024-09-05T12:24:15.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:15 smithi123 ceph-mon[34321]: fsmap 4 up:standby 2024-09-05T12:24:15.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:15 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T12:24:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:15 smithi174 ceph-mon[34313]: Deploying daemon mds.e on smithi123 2024-09-05T12:24:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:15 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6836/2997614259,v1:172.21.15.71:6837/2997614259] up:boot 2024-09-05T12:24:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:15 smithi174 ceph-mon[34313]: fsmap 4 up:standby 2024-09-05T12:24:15.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:15 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T12:24:15.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:15 smithi071 ceph-mon[29845]: Deploying daemon mds.e on smithi123 2024-09-05T12:24:15.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:15 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6836/2997614259,v1:172.21.15.71:6837/2997614259] up:boot 2024-09-05T12:24:15.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:15 smithi071 ceph-mon[29845]: fsmap 4 up:standby 2024-09-05T12:24:15.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:15 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T12:24:15.893 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:15 smithi123 systemd[1]: Starting Ceph mds.e for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:16.180 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:15 smithi123 podman[64194]: 2024-09-05 12:24:15.795504242 +0000 UTC m=+0.015993846 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:16.180 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:15 smithi123 podman[64194]: 2024-09-05 12:24:15.955740727 +0000 UTC m=+0.176230331 container create d44c02bcb32f2463ea64f5bf648419895905426f75f01cc3aa6309a8e581c200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True) 2024-09-05T12:24:16.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:16 smithi174 ceph-mon[34313]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:16.562 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:16 smithi071 ceph-mon[29845]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:16.614 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:16 smithi123 ceph-mon[34321]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:16.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:16 smithi123 podman[64194]: 2024-09-05 12:24:16.183515176 +0000 UTC m=+0.404004779 container init d44c02bcb32f2463ea64f5bf648419895905426f75f01cc3aa6309a8e581c200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-05T12:24:16.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:16 smithi123 podman[64194]: 2024-09-05 12:24:16.186364557 +0000 UTC m=+0.406854164 container start d44c02bcb32f2463ea64f5bf648419895905426f75f01cc3aa6309a8e581c200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-05T12:24:16.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:16 smithi123 ceph-mds[64214]: -- 172.21.15.123:0/3755590419 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fa0b8f860 con 0x564fa0b6c800 2024-09-05T12:24:16.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:16 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e[64210]: starting mds.e at 2024-09-05T12:24:16.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:16 smithi123 bash[64194]: d44c02bcb32f2463ea64f5bf648419895905426f75f01cc3aa6309a8e581c200 2024-09-05T12:24:16.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:24:16 smithi123 systemd[1]: Started Ceph mds.e for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:17.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 -- 172.21.15.71:0/2521104913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc077040 msgr2=0x7fa2fc112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:17.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 --2- 172.21.15.71:0/2521104913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc077040 0x7fa2fc112510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009900 tx=0x7fa2e802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 -- 172.21.15.71:0/2521104913 shutdown_connections 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 --2- 172.21.15.71:0/2521104913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2fc112a50 0x7fa2fc114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 --2- 172.21.15.71:0/2521104913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc077040 0x7fa2fc112510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 --2- 172.21.15.71:0/2521104913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc076720 0x7fa2fc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 -- 172.21.15.71:0/2521104913 >> 172.21.15.71:0/2521104913 conn(0x7fa2fc0ffa50 msgr2=0x7fa2fc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 -- 172.21.15.71:0/2521104913 shutdown_connections 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.247+0000 7fa301400640 1 -- 172.21.15.71:0/2521104913 wait complete. 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.248+0000 7fa301400640 1 Processor -- start 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.248+0000 7fa301400640 1 -- start start 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa301400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 0x7fa2fc1a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa301400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc077040 0x7fa2fc1a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa301400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2fc112a50 0x7fa2fc19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:17.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa301400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc1185e0 con 0x7fa2fc076720 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa301400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc118460 con 0x7fa2fc112a50 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa301400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2fc118760 con 0x7fa2fc077040 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 0x7fa2fc1a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 0x7fa2fc1a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48092/0 (socket says 172.21.15.71:48092) 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2f9e00640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc077040 0x7fa2fc1a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 -- 172.21.15.71:0/1210398182 learned_addr learned my addr 172.21.15.71:0/1210398182 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fb200640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2fc112a50 0x7fa2fc19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 -- 172.21.15.71:0/1210398182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc077040 msgr2=0x7fa2fc1a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc077040 0x7fa2fc1a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 -- 172.21.15.71:0/1210398182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2fc112a50 msgr2=0x7fa2fc19e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2fc112a50 0x7fa2fc19e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.249+0000 7fa2fa800640 1 -- 172.21.15.71:0/1210398182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2fc19f0c0 con 0x7fa2fc076720 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.250+0000 7fa2f9e00640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc077040 0x7fa2fc1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:17.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.250+0000 7fa2fa800640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 0x7fa2fc1a47d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00e9f0 tx=0x7fa2ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:17.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.252+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec00cd80 con 0x7fa2fc076720 2024-09-05T12:24:17.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.252+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2ec005500 con 0x7fa2fc076720 2024-09-05T12:24:17.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.252+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2fc19f3b0 con 0x7fa2fc076720 2024-09-05T12:24:17.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.252+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec010690 con 0x7fa2fc076720 2024-09-05T12:24:17.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.253+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2fc1ab6a0 con 0x7fa2fc076720 2024-09-05T12:24:17.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.254+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2fc076b00 con 0x7fa2fc076720 2024-09-05T12:24:17.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.257+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa2ec0046c0 con 0x7fa2fc076720 2024-09-05T12:24:17.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.258+0000 7fa2f3400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2e00a2b80 0x7fa2e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:17.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.258+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa2ec0cefe0 con 0x7fa2fc076720 2024-09-05T12:24:17.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.259+0000 7fa2f9e00640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2e00a2b80 0x7fa2e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:17.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.259+0000 7fa2f9e00640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2e00a2b80 0x7fa2e00a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009230 tx=0x7fa2e80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:17.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.267+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2ec09c380 con 0x7fa2fc076720 2024-09-05T12:24:17.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.485+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2fc10dc40 con 0x7fa2fc076720 2024-09-05T12:24:17.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.486+0000 7fa2f3400640 1 -- 172.21.15.71:0/1210398182 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7fa2ec09bc80 con 0x7fa2fc076720 2024-09-05T12:24:17.487 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:17.487 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":6,"btime":"2024-09-05T12:24:16:241209+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-05T12:24:17.487 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 6 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.488+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2e00a2b80 msgr2=0x7fa2e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.488+0000 7fa301400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2e00a2b80 0x7fa2e00a5040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009230 tx=0x7fa2e80058d0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.488+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 msgr2=0x7fa2fc1a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.488+0000 7fa301400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 0x7fa2fc1a47d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec00e9f0 tx=0x7fa2ec00edf0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 shutdown_connections 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2e00a2b80 0x7fa2e00a5040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2fc112a50 0x7fa2fc19e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2fc077040 0x7fa2fc1a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 --2- 172.21.15.71:0/1210398182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa2fc076720 0x7fa2fc1a47d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 >> 172.21.15.71:0/1210398182 conn(0x7fa2fc0ffa50 msgr2=0x7fa2fc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 shutdown_connections 2024-09-05T12:24:17.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:17.489+0000 7fa301400640 1 -- 172.21.15.71:0/1210398182 wait complete. 2024-09-05T12:24:17.504 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6834/2774200097,v1:172.21.15.123:6835/2774200097] up:boot 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: fsmap 5 up:standby 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:17.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:17 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:17.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6834/2774200097,v1:172.21.15.123:6835/2774200097] up:boot 2024-09-05T12:24:17.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: fsmap 5 up:standby 2024-09-05T12:24:17.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:24:17.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:17.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:17.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:17 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:17.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6834/2774200097,v1:172.21.15.123:6835/2774200097] up:boot 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: fsmap 5 up:standby 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:17.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:17 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:18.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:18 smithi123 ceph-mon[34321]: Deploying daemon mds.f on smithi174 2024-09-05T12:24:18.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:18 smithi123 ceph-mon[34321]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:18.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:18 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1210398182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:18 smithi174 ceph-mon[34313]: Deploying daemon mds.f on smithi174 2024-09-05T12:24:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:18 smithi174 ceph-mon[34313]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:18.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:18 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1210398182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:18.511 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 systemd[1]: Starting Ceph mds.f for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:18.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:18 smithi071 ceph-mon[29845]: Deploying daemon mds.f on smithi174 2024-09-05T12:24:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:18 smithi071 ceph-mon[29845]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:18 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1210398182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:18.786 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 podman[63778]: 2024-09-05 12:24:18.42054001 +0000 UTC m=+0.015683915 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:18.787 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 podman[63778]: 2024-09-05 12:24:18.533524056 +0000 UTC m=+0.128667999 container create 7d7208420c38c0b9710b9ec94b81f9fa3b1f5657b9acb76649af89a3c433ba26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-05T12:24:19.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 podman[63778]: 2024-09-05 12:24:18.786246435 +0000 UTC m=+0.381390345 container init 7d7208420c38c0b9710b9ec94b81f9fa3b1f5657b9acb76649af89a3c433ba26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T12:24:19.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 podman[63778]: 2024-09-05 12:24:18.789121969 +0000 UTC m=+0.384265867 container start 7d7208420c38c0b9710b9ec94b81f9fa3b1f5657b9acb76649af89a3c433ba26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=) 2024-09-05T12:24:19.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 ceph-mds[63797]: -- 172.21.15.174:0/2596850345 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559d6ca15860 con 0x559d6c9f2800 2024-09-05T12:24:19.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: starting mds.f at 2024-09-05T12:24:19.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 bash[63778]: 7d7208420c38c0b9710b9ec94b81f9fa3b1f5657b9acb76649af89a3c433ba26 2024-09-05T12:24:19.261 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:24:18 smithi174 systemd[1]: Started Ceph mds.f for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: Deploying daemon mds.c on smithi174 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6834/626911382,v1:172.21.15.174:6835/626911382] up:boot 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: fsmap 6 up:standby 2024-09-05T12:24:20.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:19 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:24:20.308 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: Deploying daemon mds.c on smithi174 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6834/626911382,v1:172.21.15.174:6835/626911382] up:boot 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: fsmap 6 up:standby 2024-09-05T12:24:20.365 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:19 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: Deploying daemon mds.c on smithi174 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6834/626911382,v1:172.21.15.174:6835/626911382] up:boot 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: fsmap 6 up:standby 2024-09-05T12:24:20.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:19 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:24:20.686 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:21.010 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:20 smithi174 systemd[1]: Starting Ceph mds.c for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:21.325 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 podman[64080]: 2024-09-05 12:24:20.948256018 +0000 UTC m=+0.015900809 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:21.325 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 podman[64080]: 2024-09-05 12:24:21.055129964 +0000 UTC m=+0.122774793 container create 0482b6e873df6258f9d1956e172e580c3c9a1128d4947345dc6e93da828fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 -- 172.21.15.71:0/1089054683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e7010e3b0 msgr2=0x7f4e70110800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 --2- 172.21.15.71:0/1089054683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e7010e3b0 0x7f4e70110800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e68009ca0 tx=0x7f4e6802a950 comp rx=0 tx=0).stop 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 -- 172.21.15.71:0/1089054683 shutdown_connections 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 --2- 172.21.15.71:0/1089054683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e7010e3b0 0x7f4e70110800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 --2- 172.21.15.71:0/1089054683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e70074f40 0x7f4e7010de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 --2- 172.21.15.71:0/1089054683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 0x7f4e70074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.338+0000 7f4e76e00640 1 -- 172.21.15.71:0/1089054683 >> 172.21.15.71:0/1089054683 conn(0x7f4e700fd8b0 msgr2=0x7f4e700ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:21.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.339+0000 7f4e76e00640 1 -- 172.21.15.71:0/1089054683 shutdown_connections 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.339+0000 7f4e76e00640 1 -- 172.21.15.71:0/1089054683 wait complete. 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 Processor -- start 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 -- start start 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 0x7f4e7019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 0x7f4e7019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e7010e3b0 0x7f4e7019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70114190 con 0x7f4e70074620 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70114010 con 0x7f4e7010e3b0 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e76e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e70114310 con 0x7f4e70074f40 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e6fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 0x7f4e7019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.340+0000 7f4e67400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 0x7f4e7019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 0x7f4e7019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58116/0 (socket says 172.21.15.71:58116) 2024-09-05T12:24:21.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e6fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 0x7f4e7019a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48110/0 (socket says 172.21.15.71:48110) 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 -- 172.21.15.71:0/2892633679 learned_addr learned my addr 172.21.15.71:0/2892633679 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e74e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e7010e3b0 0x7f4e7019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 -- 172.21.15.71:0/2892633679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e7010e3b0 msgr2=0x7f4e7019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e7010e3b0 0x7f4e7019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 -- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 msgr2=0x7f4e7019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 0x7f4e7019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 -- 172.21.15.71:0/2892633679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7019f640 con 0x7f4e70074f40 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e67400640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 0x7f4e7019abd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800d950 tx=0x7f4e5800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58017030 con 0x7f4e70074f40 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e58004020 con 0x7f4e70074f40 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7019f930 con 0x7f4e70074f40 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580047a0 con 0x7f4e70074f40 2024-09-05T12:24:21.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.341+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e701a7210 con 0x7f4e70074f40 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.342+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e70074a00 con 0x7f4e70074f40 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.344+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4e580041d0 con 0x7f4e70074f40 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.345+0000 7f4e6d600640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e540a2b80 0x7f4e540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.346+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f4e580ce530 con 0x7f4e70074f40 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.346+0000 7f4e6fe00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e540a2b80 0x7f4e540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.346+0000 7f4e6fe00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e540a2b80 0x7f4e540a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009bd0 tx=0x7f4e60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:21.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.349+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e5809b890 con 0x7f4e70074f40 2024-09-05T12:24:21.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.567+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e70108fa0 con 0x7f4e70074f40 2024-09-05T12:24:21.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.568+0000 7f4e6d600640 1 -- 172.21.15.71:0/2892633679 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5560 (secure 0 0 0) 0x7f4e5809b190 con 0x7f4e70074f40 2024-09-05T12:24:21.569 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:21.569 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":7,"btime":"2024-09-05T12:24:19:250611+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-05T12:24:21.569 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 7 2024-09-05T12:24:21.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e540a2b80 msgr2=0x7f4e540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:21.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e540a2b80 0x7f4e540a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009bd0 tx=0x7f4e60005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 msgr2=0x7f4e7019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 0x7f4e7019abd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800d950 tx=0x7f4e5800de10 comp rx=0 tx=0).stop 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 shutdown_connections 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e540a2b80 0x7f4e540a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e7010e3b0 0x7f4e7019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e70074f40 0x7f4e7019abd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 --2- 172.21.15.71:0/2892633679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e70074620 0x7f4e7019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.570+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 >> 172.21.15.71:0/2892633679 conn(0x7f4e700fd8b0 msgr2=0x7f4e700fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.571+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 shutdown_connections 2024-09-05T12:24:21.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:21.571+0000 7f4e76e00640 1 -- 172.21.15.71:0/2892633679 wait complete. 2024-09-05T12:24:21.760 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 podman[64080]: 2024-09-05 12:24:21.324637743 +0000 UTC m=+0.392282544 container init 0482b6e873df6258f9d1956e172e580c3c9a1128d4947345dc6e93da828fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:24:21.760 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 podman[64080]: 2024-09-05 12:24:21.327756499 +0000 UTC m=+0.395401291 container start 0482b6e873df6258f9d1956e172e580c3c9a1128d4947345dc6e93da828fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T12:24:21.760 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 ceph-mds[64099]: -- 172.21.15.174:0/3472155856 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f8ea55c20 con 0x556f8ea33400 2024-09-05T12:24:21.760 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c[64095]: starting mds.c at 2024-09-05T12:24:21.761 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 bash[64080]: 0482b6e873df6258f9d1956e172e580c3c9a1128d4947345dc6e93da828fe34f 2024-09-05T12:24:21.761 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:24:21 smithi174 systemd[1]: Started Ceph mds.c for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:22.301 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:22 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2892633679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:22.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:22.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:22 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2892633679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:22 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2892633679' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:23.387 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:23 smithi123 ceph-mon[34321]: Deploying daemon mds.h on smithi123 2024-09-05T12:24:23.388 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:23 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] up:boot 2024-09-05T12:24:23.388 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:23 smithi123 ceph-mon[34321]: fsmap 7 up:standby 2024-09-05T12:24:23.388 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:23 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:24:23.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:23 smithi174 ceph-mon[34313]: Deploying daemon mds.h on smithi123 2024-09-05T12:24:23.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:23 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] up:boot 2024-09-05T12:24:23.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:23 smithi174 ceph-mon[34313]: fsmap 7 up:standby 2024-09-05T12:24:23.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:23 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:24:23.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:23 smithi071 ceph-mon[29845]: Deploying daemon mds.h on smithi123 2024-09-05T12:24:23.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:23 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] up:boot 2024-09-05T12:24:23.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:23 smithi071 ceph-mon[29845]: fsmap 7 up:standby 2024-09-05T12:24:23.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:23 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:24:23.679 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:23 smithi123 systemd[1]: Starting Ceph mds.h for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:24.073 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:23 smithi123 podman[64504]: 2024-09-05 12:24:23.684675033 +0000 UTC m=+0.016218412 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:24.073 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:23 smithi123 podman[64504]: 2024-09-05 12:24:23.819769978 +0000 UTC m=+0.151313399 container create 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T12:24:24.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:24.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:24 smithi123 ceph-mon[34321]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:24.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:24 smithi123 podman[64504]: 2024-09-05 12:24:24.072520305 +0000 UTC m=+0.404063693 container init 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T12:24:24.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:24 smithi123 podman[64504]: 2024-09-05 12:24:24.075801417 +0000 UTC m=+0.407344799 container start 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:24:24.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:24 smithi123 ceph-mds[64523]: -- 172.21.15.123:0/2215738040 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a5e2a63860 con 0x55a5e2a40800 2024-09-05T12:24:24.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h[64519]: starting mds.h at 2024-09-05T12:24:24.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:24 smithi123 bash[64504]: 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 2024-09-05T12:24:24.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:24:24 smithi123 systemd[1]: Started Ceph mds.h for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:24 smithi174 ceph-mon[34313]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:24 smithi071 ceph-mon[29845]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:24.713 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 -- 172.21.15.71:0/1290228120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308103ff0 msgr2=0x7fa3081043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 --2- 172.21.15.71:0/1290228120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308103ff0 0x7fa3081043d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009900 tx=0x7fa2f802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 -- 172.21.15.71:0/1290228120 shutdown_connections 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 --2- 172.21.15.71:0/1290228120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308109120 0x7fa308110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 --2- 172.21.15.71:0/1290228120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3081049a0 0x7fa3081089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 --2- 172.21.15.71:0/1290228120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308103ff0 0x7fa3081043d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 -- 172.21.15.71:0/1290228120 >> 172.21.15.71:0/1290228120 conn(0x7fa3080ffa90 msgr2=0x7fa308101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 -- 172.21.15.71:0/1290228120 shutdown_connections 2024-09-05T12:24:25.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.307+0000 7fa30e400640 1 -- 172.21.15.71:0/1290228120 wait complete. 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.308+0000 7fa30e400640 1 Processor -- start 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.308+0000 7fa30e400640 1 -- start start 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa30e400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 0x7fa30819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa30e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3081049a0 0x7fa30819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa30e400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308109120 0x7fa30819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa30e400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa308114190 con 0x7fa308103ff0 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa30e400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa308114010 con 0x7fa3081049a0 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa30e400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa308114310 con 0x7fa308109120 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa307400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 0x7fa30819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:25.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa307400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 0x7fa30819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50510/0 (socket says 172.21.15.71:50510) 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa307400640 1 -- 172.21.15.71:0/109255447 learned_addr learned my addr 172.21.15.71:0/109255447 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa307e00640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308109120 0x7fa30819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.309+0000 7fa2ffe00640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3081049a0 0x7fa30819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307e00640 1 -- 172.21.15.71:0/109255447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3081049a0 msgr2=0x7fa30819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307e00640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3081049a0 0x7fa30819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307e00640 1 -- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 msgr2=0x7fa30819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307e00640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 0x7fa30819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307e00640 1 -- 172.21.15.71:0/109255447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30819f5c0 con 0x7fa308109120 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 0x7fa30819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa307e00640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308109120 0x7fa30819eee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa300009fd0 tx=0x7fa30000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:25.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30000ce20 con 0x7fa308109120 2024-09-05T12:24:25.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa300005020 con 0x7fa308109120 2024-09-05T12:24:25.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30819f8b0 con 0x7fa308109120 2024-09-05T12:24:25.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300010720 con 0x7fa308109120 2024-09-05T12:24:25.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.310+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3081a7190 con 0x7fa308109120 2024-09-05T12:24:25.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.311+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3081043d0 con 0x7fa308109120 2024-09-05T12:24:25.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.313+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa300005540 con 0x7fa308109120 2024-09-05T12:24:25.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.314+0000 7fa304c00640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:25.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.314+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa3000cf200 con 0x7fa308109120 2024-09-05T12:24:25.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.315+0000 7fa307400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:25.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.315+0000 7fa307400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80096c0 tx=0x7fa2f80046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:25.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.318+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa30009c560 con 0x7fa308109120 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] up:boot 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: fsmap 8 up:standby 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:25.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:25.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:25.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:25 smithi174 ceph-mon[34313]: Deploying daemon mds.i on smithi174 2024-09-05T12:24:25.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.527+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa30802d7c0 con 0x7fa308109120 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] up:boot 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: fsmap 8 up:standby 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:25.529 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:25.530 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:25 smithi071 ceph-mon[29845]: Deploying daemon mds.i on smithi174 2024-09-05T12:24:25.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.528+0000 7fa304c00640 1 -- 172.21.15.71:0/109255447 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7fa30009be60 con 0x7fa308109120 2024-09-05T12:24:25.530 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:25.531 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":9,"btime":"2024-09-05T12:24:24:194385+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-05T12:24:25.531 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 9 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2ec0a2c00 msgr2=0x7fa2ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80096c0 tx=0x7fa2f80046c0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308109120 msgr2=0x7fa30819eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308109120 0x7fa30819eee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa300009fd0 tx=0x7fa30000ef10 comp rx=0 tx=0).stop 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 shutdown_connections 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa2ec0a2c00 0x7fa2ec0a50c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa308109120 0x7fa30819eee0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3081049a0 0x7fa30819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 --2- 172.21.15.71:0/109255447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa308103ff0 0x7fa30819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:25.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 >> 172.21.15.71:0/109255447 conn(0x7fa3080ffa90 msgr2=0x7fa308100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:25.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 shutdown_connections 2024-09-05T12:24:25.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:25.531+0000 7fa30e400640 1 -- 172.21.15.71:0/109255447 wait complete. 2024-09-05T12:24:25.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] up:boot 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: fsmap 8 up:standby 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:25.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:25 smithi123 ceph-mon[34321]: Deploying daemon mds.i on smithi174 2024-09-05T12:24:26.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:26 smithi174 ceph-mon[34313]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:26.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:26 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/109255447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:26.491 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 systemd[1]: Starting Ceph mds.i for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:26.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:26 smithi071 ceph-mon[29845]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:26.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:26 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/109255447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:26 smithi123 ceph-mon[34321]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:26 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/109255447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:26.760 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 podman[64382]: 2024-09-05 12:24:26.392497749 +0000 UTC m=+0.015632374 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:26.760 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 podman[64382]: 2024-09-05 12:24:26.51903011 +0000 UTC m=+0.142164720 container create 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS) 2024-09-05T12:24:27.260 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 podman[64382]: 2024-09-05 12:24:26.781419382 +0000 UTC m=+0.404553994 container init 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-05T12:24:27.260 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 podman[64382]: 2024-09-05 12:24:26.784263757 +0000 UTC m=+0.407398379 container start 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-05T12:24:27.260 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 ceph-mds[64401]: -- 172.21.15.174:0/4041274035 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa666af860 con 0x55aa6668c800 2024-09-05T12:24:27.260 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i[64397]: starting mds.i at 2024-09-05T12:24:27.261 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 bash[64382]: 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd 2024-09-05T12:24:27.261 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:24:26 smithi174 systemd[1]: Started Ceph mds.i for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: Deploying daemon mds.g on smithi071 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:28.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] up:boot 2024-09-05T12:24:28.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: fsmap 9 up:standby 2024-09-05T12:24:28.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:27 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:24:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:28.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:28.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: Deploying daemon mds.g on smithi071 2024-09-05T12:24:28.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:28.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] up:boot 2024-09-05T12:24:28.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: fsmap 9 up:standby 2024-09-05T12:24:28.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:27 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:24:28.338 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: Deploying daemon mds.g on smithi071 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] up:boot 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: fsmap 9 up:standby 2024-09-05T12:24:28.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:27 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:24:28.936 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:29.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 -- 172.21.15.71:0/4172287766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a41047e0 msgr2=0x7fa9a4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:29.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/4172287766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a41047e0 0x7fa9a4108830 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009900 tx=0x7fa98c02a410 comp rx=0 tx=0).stop 2024-09-05T12:24:29.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 -- 172.21.15.71:0/4172287766 shutdown_connections 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/4172287766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a4108f60 0x7fa9a4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/4172287766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a41047e0 0x7fa9a4108830 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/4172287766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9a4103e30 0x7fa9a4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.733+0000 7fa9a8a00640 1 -- 172.21.15.71:0/4172287766 >> 172.21.15.71:0/4172287766 conn(0x7fa9a40ff910 msgr2=0x7fa9a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.734+0000 7fa9a8a00640 1 -- 172.21.15.71:0/4172287766 shutdown_connections 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.734+0000 7fa9a8a00640 1 -- 172.21.15.71:0/4172287766 wait complete. 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 Processor -- start 2024-09-05T12:24:29.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 -- start start 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9a4103e30 0x7fa9a419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 0x7fa9a419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a4108f60 0x7fa9a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4113fc0 con 0x7fa9a4108f60 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4113e40 con 0x7fa9a41047e0 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a8a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4114140 con 0x7fa9a4103e30 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa9a3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a4108f60 0x7fa9a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa99aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 0x7fa9a419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa99aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 0x7fa9a419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:56758/0 (socket says 172.21.15.71:56758) 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.735+0000 7fa99aa00640 1 -- 172.21.15.71:0/1447367100 learned_addr learned my addr 172.21.15.71:0/1447367100 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa9a3400640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9a4103e30 0x7fa9a419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa99aa00640 1 -- 172.21.15.71:0/1447367100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9a4103e30 msgr2=0x7fa9a419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:29.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa99aa00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9a4103e30 0x7fa9a419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa99aa00640 1 -- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a4108f60 msgr2=0x7fa9a419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa99aa00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a4108f60 0x7fa9a419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa99aa00640 1 -- 172.21.15.71:0/1447367100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a419f460 con 0x7fa9a41047e0 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa9a3e00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a4108f60 0x7fa9a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa99aa00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 0x7fa9a419a9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa98c006010 tx=0x7fa98c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.736+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c03f030 con 0x7fa9a41047e0 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.737+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a419f6f0 con 0x7fa9a41047e0 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.737+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98c004440 con 0x7fa9a41047e0 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.737+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c035070 con 0x7fa9a41047e0 2024-09-05T12:24:29.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.737+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a41a7030 con 0x7fa9a41047e0 2024-09-05T12:24:29.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.738+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a4105310 con 0x7fa9a41047e0 2024-09-05T12:24:29.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.740+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa98c042030 con 0x7fa9a41047e0 2024-09-05T12:24:29.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.741+0000 7fa9a0c00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9880a2b80 0x7fa9880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:29.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.741+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7fa98c0ef1f0 con 0x7fa9a41047e0 2024-09-05T12:24:29.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.741+0000 7fa9a3400640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9880a2b80 0x7fa9880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:29.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.742+0000 7fa9a3400640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9880a2b80 0x7fa9880a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa994009bd0 tx=0x7fa994005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:29.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:29.743+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa98c0bbf00 con 0x7fa9a41047e0 2024-09-05T12:24:30.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.006+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a4104210 con 0x7fa9a41047e0 2024-09-05T12:24:30.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.006+0000 7fa9a0c00640 1 -- 172.21.15.71:0/1447367100 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8040 (secure 0 0 0) 0x7fa9a4104210 con 0x7fa9a41047e0 2024-09-05T12:24:30.007 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:30.008 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":10,"btime":"2024-09-05T12:24:27:237632+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-05T12:24:30.008 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 10 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9880a2b80 msgr2=0x7fa9880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9880a2b80 0x7fa9880a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa994009bd0 tx=0x7fa994005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 msgr2=0x7fa9a419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 0x7fa9a419a9f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa98c006010 tx=0x7fa98c004040 comp rx=0 tx=0).stop 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 shutdown_connections 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9880a2b80 0x7fa9880a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9a4108f60 0x7fa9a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9a41047e0 0x7fa9a419a9f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 --2- 172.21.15.71:0/1447367100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9a4103e30 0x7fa9a419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 >> 172.21.15.71:0/1447367100 conn(0x7fa9a40ff910 msgr2=0x7fa9a4100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 shutdown_connections 2024-09-05T12:24:30.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:30.009+0000 7fa9a8a00640 1 -- 172.21.15.71:0/1447367100 wait complete. 2024-09-05T12:24:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:30 smithi071 ceph-mon[29845]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:30.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:30 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1447367100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:30 smithi174 ceph-mon[34313]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:30 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1447367100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:30 smithi123 ceph-mon[34321]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:30.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:30 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1447367100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:30.910 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:24:30 smithi071 systemd[1]: Started Ceph mds.g for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: Deploying daemon mds.a on smithi071 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] up:boot 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: fsmap 10 up:standby 2024-09-05T12:24:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:31 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:24:32.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: Deploying daemon mds.a on smithi071 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] up:boot 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: fsmap 10 up:standby 2024-09-05T12:24:32.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:31 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:24:32.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:32.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: Deploying daemon mds.a on smithi071 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] up:boot 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: fsmap 10 up:standby 2024-09-05T12:24:32.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:31 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:24:33.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:33.798 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:33 smithi071 systemd[1]: Starting Ceph mds.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:34.071 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:33 smithi071 podman[70847]: 2024-09-05 12:24:33.701663865 +0000 UTC m=+0.027572800 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:34.071 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:33 smithi071 podman[70847]: 2024-09-05 12:24:33.808733878 +0000 UTC m=+0.134642793 container create 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:24:34.071 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:34 smithi071 podman[70847]: 2024-09-05 12:24:34.070036668 +0000 UTC m=+0.395945583 container init 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:24:34.262 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:34.397 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:34 smithi071 ceph-mon[29845]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:34.397 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:34 smithi071 podman[70847]: 2024-09-05 12:24:34.076654817 +0000 UTC m=+0.402563733 container start 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:24:34.397 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:34 smithi071 ceph-mds[70884]: -- 172.21.15.71:0/2342195867 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56176cb19860 con 0x56176caf7400 2024-09-05T12:24:34.398 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:34 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a[70877]: starting mds.a at 2024-09-05T12:24:34.398 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:34 smithi071 bash[70847]: 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 2024-09-05T12:24:34.398 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:24:34 smithi071 systemd[1]: Started Ceph mds.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:34 smithi174 ceph-mon[34313]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:34 smithi123 ceph-mon[34321]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 -- 172.21.15.71:0/184248936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 msgr2=0x7facd410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 --2- 172.21.15.71:0/184248936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 0x7facd410eaa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faccc009be0 tx=0x7faccc02a6c0 comp rx=0 tx=0).stop 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 -- 172.21.15.71:0/184248936 shutdown_connections 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 --2- 172.21.15.71:0/184248936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 0x7facd410eaa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 --2- 172.21.15.71:0/184248936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd41027a0 0x7facd41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 --2- 172.21.15.71:0/184248936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:34.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 -- 172.21.15.71:0/184248936 >> 172.21.15.71:0/184248936 conn(0x7facd40fd8b0 msgr2=0x7facd40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:34.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 -- 172.21.15.71:0/184248936 shutdown_connections 2024-09-05T12:24:34.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.815+0000 7facdb400640 1 -- 172.21.15.71:0/184248936 wait complete. 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.816+0000 7facdb400640 1 Processor -- start 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 -- start start 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd41027a0 0x7facd419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 0x7facd419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7facd4111f30 con 0x7facd41027a0 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facd4111db0 con 0x7facd4106f20 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facdb400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7facd41120b0 con 0x7facd4101df0 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facd8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facd8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58998/0 (socket says 172.21.15.71:58998) 2024-09-05T12:24:34.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facd8a00640 1 -- 172.21.15.71:0/1921496941 learned_addr learned my addr 172.21.15.71:0/1921496941 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facd9400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 0x7facd419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.817+0000 7facd3e00640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd41027a0 0x7facd419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd8a00640 1 -- 172.21.15.71:0/1921496941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 msgr2=0x7facd419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd8a00640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 0x7facd419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd8a00640 1 -- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd41027a0 msgr2=0x7facd419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd8a00640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd41027a0 0x7facd419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd8a00640 1 -- 172.21.15.71:0/1921496941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd419f560 con 0x7facd4101df0 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd8a00640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd419a5b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facc800e9f0 tx=0x7facc800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc800cd80 con 0x7facd4101df0 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd419f850 con 0x7facd4101df0 2024-09-05T12:24:34.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facc8010910 con 0x7facd4101df0 2024-09-05T12:24:34.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc80188f0 con 0x7facd4101df0 2024-09-05T12:24:34.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.818+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd41a7130 con 0x7facd4101df0 2024-09-05T12:24:34.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.819+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd41021d0 con 0x7facd4101df0 2024-09-05T12:24:34.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.821+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7facc8010ac0 con 0x7facd4101df0 2024-09-05T12:24:34.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.822+0000 7facd1600640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7facbc0a2b80 0x7facbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:34.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.823+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7facc80cf530 con 0x7facd4101df0 2024-09-05T12:24:34.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.823+0000 7facd3e00640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7facbc0a2b80 0x7facbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:34.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.823+0000 7facd3e00640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7facbc0a2b80 0x7facbc0a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facc00096c0 tx=0x7facc0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:34.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:34.825+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7facc809c240 con 0x7facd4101df0 2024-09-05T12:24:35.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.033+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facd419b910 con 0x7facd4101df0 2024-09-05T12:24:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.034+0000 7facd1600640 1 -- 172.21.15.71:0/1921496941 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7facc80a0540 con 0x7facd4101df0 2024-09-05T12:24:35.035 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:35.035 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":12,"btime":"2024-09-05T12:24:34:184459+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-09-05T12:24:35.036 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 12 2024-09-05T12:24:35.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7facbc0a2b80 msgr2=0x7facbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:35.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7facbc0a2b80 0x7facbc0a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7facc00096c0 tx=0x7facc0009260 comp rx=0 tx=0).stop 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 msgr2=0x7facd419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd419a5b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facc800e9f0 tx=0x7facc800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 shutdown_connections 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7facbc0a2b80 0x7facbc0a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facd4106f20 0x7facd419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facd41027a0 0x7facd419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 --2- 172.21.15.71:0/1921496941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facd4101df0 0x7facd419a5b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 >> 172.21.15.71:0/1921496941 conn(0x7facd40fd8b0 msgr2=0x7facd40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 shutdown_connections 2024-09-05T12:24:35.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:35.037+0000 7facdb400640 1 -- 172.21.15.71:0/1921496941 wait complete. 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6840/991266062,v1:172.21.15.71:6841/991266062] up:boot 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: fsmap 11 up:standby 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:35.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: Deploying daemon mds.b on smithi123 2024-09-05T12:24:35.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:35 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1921496941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:35.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6840/991266062,v1:172.21.15.71:6841/991266062] up:boot 2024-09-05T12:24:35.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: fsmap 11 up:standby 2024-09-05T12:24:35.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: Deploying daemon mds.b on smithi123 2024-09-05T12:24:35.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:35 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1921496941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6840/991266062,v1:172.21.15.71:6841/991266062] up:boot 2024-09-05T12:24:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: fsmap 11 up:standby 2024-09-05T12:24:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:24:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:35.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-05T12:24:35.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-05T12:24:35.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:35.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: Deploying daemon mds.b on smithi123 2024-09-05T12:24:35.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:35 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1921496941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:36.602 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:36 smithi123 ceph-mon[34321]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:36.602 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 systemd[1]: Starting Ceph mds.b for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:24:36.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:36 smithi071 ceph-mon[29845]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:36 smithi174 ceph-mon[34313]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:36.903 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 podman[64803]: 2024-09-05 12:24:36.503843631 +0000 UTC m=+0.015671248 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:24:36.904 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 podman[64803]: 2024-09-05 12:24:36.642113598 +0000 UTC m=+0.153941211 container create 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-05T12:24:36.904 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 podman[64803]: 2024-09-05 12:24:36.903173107 +0000 UTC m=+0.415000724 container init 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:24:37.179 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 podman[64803]: 2024-09-05 12:24:36.906225681 +0000 UTC m=+0.418053291 container start 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-05T12:24:37.180 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 ceph-mds[64822]: -- 172.21.15.123:0/4118486919 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f15e59860 con 0x556f15e37000 2024-09-05T12:24:37.180 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: starting mds.b at 2024-09-05T12:24:37.180 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 bash[64803]: 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 2024-09-05T12:24:37.180 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:36 smithi123 systemd[1]: Started Ceph mds.b for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:24:37.845 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:24:38.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] up:boot 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: fsmap 12 up:standby 2024-09-05T12:24:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:38 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:24:38.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] up:boot 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: fsmap 12 up:standby 2024-09-05T12:24:38.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:38 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:24:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:38.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:38.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] up:boot 2024-09-05T12:24:38.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: fsmap 12 up:standby 2024-09-05T12:24:38.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:38 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:24:38.661 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 -- 172.21.15.71:0/994159310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4103e30 msgr2=0x7f84a4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:39.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 --2- 172.21.15.71:0/994159310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4103e30 0x7f84a4104210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8494009900 tx=0x7f849402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 -- 172.21.15.71:0/994159310 shutdown_connections 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 --2- 172.21.15.71:0/994159310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a4108f60 0x7f84a4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 --2- 172.21.15.71:0/994159310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84a41047e0 0x7f84a4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 --2- 172.21.15.71:0/994159310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4103e30 0x7f84a4104210 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 -- 172.21.15.71:0/994159310 >> 172.21.15.71:0/994159310 conn(0x7f84a40ff910 msgr2=0x7f84a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 -- 172.21.15.71:0/994159310 shutdown_connections 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.273+0000 7f84a9800640 1 -- 172.21.15.71:0/994159310 wait complete. 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.274+0000 7f84a9800640 1 Processor -- start 2024-09-05T12:24:39.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 -- start start 2024-09-05T12:24:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84a4103e30 0x7f84a419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a41047e0 0x7f84a419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 0x7f84a419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4114010 con 0x7f84a41047e0 2024-09-05T12:24:39.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4113e90 con 0x7f84a4108f60 2024-09-05T12:24:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a9800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84a4114190 con 0x7f84a4103e30 2024-09-05T12:24:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a41047e0 0x7f84a419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 0x7f84a419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.275+0000 7f84a3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84a4103e30 0x7f84a419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 0x7f84a419ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50182/0 (socket says 172.21.15.71:50182) 2024-09-05T12:24:39.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 -- 172.21.15.71:0/592380639 learned_addr learned my addr 172.21.15.71:0/592380639 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 -- 172.21.15.71:0/592380639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84a4103e30 msgr2=0x7f84a419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84a4103e30 0x7f84a419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 -- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a41047e0 msgr2=0x7f84a419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a41047e0 0x7f84a419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 -- 172.21.15.71:0/592380639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a419f430 con 0x7f84a4108f60 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.276+0000 7f84a8e00640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 0x7f84a419ed50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f849c00efd0 tx=0x7f849c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.277+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c003b70 con 0x7f84a4108f60 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.277+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f849c003d10 con 0x7f84a4108f60 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.277+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a419f720 con 0x7f84a4108f60 2024-09-05T12:24:39.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.277+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f849c0109f0 con 0x7f84a4108f60 2024-09-05T12:24:39.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.277+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a41a7000 con 0x7f84a4108f60 2024-09-05T12:24:39.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.278+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a4105310 con 0x7f84a4108f60 2024-09-05T12:24:39.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.280+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f849c004020 con 0x7f84a4108f60 2024-09-05T12:24:39.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.281+0000 7f84a0c00640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84880a2c00 0x7f84880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:39.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.281+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f849c0ceee0 con 0x7f84a4108f60 2024-09-05T12:24:39.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.281+0000 7f84a3e00640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84880a2c00 0x7f84880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:39.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.282+0000 7f84a3e00640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84880a2c00 0x7f84880a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f8494005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:39.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.285+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f849c09c2e0 con 0x7f84a4108f60 2024-09-05T12:24:39.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:39 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:39 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:39 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:39.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.562+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84a402d7c0 con 0x7f84a4108f60 2024-09-05T12:24:39.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.563+0000 7f84a0c00640 1 -- 172.21.15.71:0/592380639 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10515 (secure 0 0 0) 0x7f849c09bbe0 con 0x7f84a4108f60 2024-09-05T12:24:39.564 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:39.565 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":13,"btime":"2024-09-05T12:24:37:292691+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-05T12:24:39.565 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 13 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.565+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84880a2c00 msgr2=0x7f84880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.565+0000 7f84a9800640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84880a2c00 0x7f84880a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8494000c00 tx=0x7f8494005b50 comp rx=0 tx=0).stop 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.565+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 msgr2=0x7f84a419ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.565+0000 7f84a9800640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 0x7f84a419ed50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f849c00efd0 tx=0x7f849c00c6a0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 shutdown_connections 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84880a2c00 0x7f84880a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84a4108f60 0x7f84a419ed50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84a41047e0 0x7f84a419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 --2- 172.21.15.71:0/592380639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84a4103e30 0x7f84a419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 >> 172.21.15.71:0/592380639 conn(0x7f84a40ff910 msgr2=0x7f84a4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 shutdown_connections 2024-09-05T12:24:39.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:39.566+0000 7f84a9800640 1 -- 172.21.15.71:0/592380639 wait complete. 2024-09-05T12:24:40.375 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:40.375 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/592380639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:40.376 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.376 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.376 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:40.376 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:40.376 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:40 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.417 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-05T12:24:40.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:40.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/592380639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:40.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:40.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:40.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:40 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/592380639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:24:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:40.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:40 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:40.814 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:41.366 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:41 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:41.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:41 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:41.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:41 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 -- 172.21.15.71:0/3495772042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 msgr2=0x7f91281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 --2- 172.21.15.71:0/3495772042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 0x7f91281067e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9114009900 tx=0x7f911402a410 comp rx=0 tx=0).stop 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 -- 172.21.15.71:0/3495772042 shutdown_connections 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 --2- 172.21.15.71:0/3495772042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9128106f10 0x7f912810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 --2- 172.21.15.71:0/3495772042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 0x7f91281067e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 --2- 172.21.15.71:0/3495772042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f91281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 -- 172.21.15.71:0/3495772042 >> 172.21.15.71:0/3495772042 conn(0x7f91280fd8c0 msgr2=0x7f91280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.564+0000 7f912ce00640 1 -- 172.21.15.71:0/3495772042 shutdown_connections 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.565+0000 7f912ce00640 1 -- 172.21.15.71:0/3495772042 wait complete. 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.565+0000 7f912ce00640 1 Processor -- start 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.565+0000 7f912ce00640 1 -- start start 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f912ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f912819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f912ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 0x7f912819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f912ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9128106f10 0x7f912819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f912ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9128111fe0 con 0x7f9128102790 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f912ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9128111e60 con 0x7f9128101de0 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f912ce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9128112160 con 0x7f9128106f10 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f912819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:41.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f912819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50198/0 (socket says 172.21.15.71:50198) 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 -- 172.21.15.71:0/1678729622 learned_addr learned my addr 172.21.15.71:0/1678729622 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 0x7f912819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9126800640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9128106f10 0x7f912819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 -- 172.21.15.71:0/1678729622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9128106f10 msgr2=0x7f912819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9128106f10 0x7f912819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 -- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 msgr2=0x7f912819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 0x7f912819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.566+0000 7f9125e00640 1 -- 172.21.15.71:0/1678729622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912819f640 con 0x7f9128101de0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.567+0000 7f9125e00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f912819a690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f911800e9f0 tx=0x7f911800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.567+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911800cd80 con 0x7f9128101de0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.567+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9118010910 con 0x7f9128101de0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.567+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912819f930 con 0x7f9128101de0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.567+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9118018990 con 0x7f9128101de0 2024-09-05T12:24:41.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.567+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91281a7210 con 0x7f9128101de0 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.568+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91281021c0 con 0x7f9128101de0 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.570+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9118010ac0 con 0x7f9128101de0 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.571+0000 7f911ea00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f910c0a2c00 0x7f910c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.571+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f91180cf3c0 con 0x7f9128101de0 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.571+0000 7f9125400640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f910c0a2c00 0x7f910c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.572+0000 7f9125400640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f910c0a2c00 0x7f910c0a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9114000c00 tx=0x7f9114005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:41.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.573+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f911809c170 con 0x7f9128101de0 2024-09-05T12:24:41.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.757+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f91281a7410 con 0x7f9128101de0 2024-09-05T12:24:41.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.758+0000 7f911ea00640 1 -- 172.21.15.71:0/1678729622 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f91180a0470 con 0x7f9128101de0 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout:[ 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout: "poolnum": 1, 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout: "poolname": ".mgr" 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:24:41.760 INFO:teuthology.orchestra.run.smithi071.stdout:] 2024-09-05T12:24:41.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f910c0a2c00 msgr2=0x7f910c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f910c0a2c00 0x7f910c0a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9114000c00 tx=0x7f9114005b50 comp rx=0 tx=0).stop 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 msgr2=0x7f912819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f912819a690 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f911800e9f0 tx=0x7f911800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 shutdown_connections 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f910c0a2c00 0x7f910c0a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9128106f10 0x7f912819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9128102790 0x7f912819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 --2- 172.21.15.71:0/1678729622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9128101de0 0x7f912819a690 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.760+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 >> 172.21.15.71:0/1678729622 conn(0x7f91280fd8c0 msgr2=0x7f91280fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:41.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.761+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 shutdown_connections 2024-09-05T12:24:41.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:41.761+0000 7f912ce00640 1 -- 172.21.15.71:0/1678729622 wait complete. 2024-09-05T12:24:42.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:42 smithi071 ceph-mon[29845]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:42 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:42 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:42 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:42 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:42 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1678729622' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T12:24:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:42 smithi123 ceph-mon[34321]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:42 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:42 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:42 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:42 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:42 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1678729622' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T12:24:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:42 smithi174 ceph-mon[34313]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:42 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:42 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:42 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:42 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:42.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:42 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1678729622' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-05T12:24:42.987 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-05T12:24:42.987 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-05T12:24:43.376 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 -- 172.21.15.71:0/225584702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71341049a0 msgr2=0x7f71341089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 --2- 172.21.15.71:0/225584702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71341049a0 0x7f71341089f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f711c009900 tx=0x7f711c02a410 comp rx=0 tx=0).stop 2024-09-05T12:24:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 -- 172.21.15.71:0/225584702 shutdown_connections 2024-09-05T12:24:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 --2- 172.21.15.71:0/225584702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7134109120 0x7f7134110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 --2- 172.21.15.71:0/225584702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71341049a0 0x7f71341089f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:43.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 --2- 172.21.15.71:0/225584702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7134103ff0 0x7f71341043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 -- 172.21.15.71:0/225584702 >> 172.21.15.71:0/225584702 conn(0x7f71340ffa90 msgr2=0x7f7134101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 -- 172.21.15.71:0/225584702 shutdown_connections 2024-09-05T12:24:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.949+0000 7f713a600640 1 -- 172.21.15.71:0/225584702 wait complete. 2024-09-05T12:24:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.950+0000 7f713a600640 1 Processor -- start 2024-09-05T12:24:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.950+0000 7f713a600640 1 -- start start 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f713a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 0x7f713419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f713a600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71341049a0 0x7f713419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f713a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7134109120 0x7f713419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f713a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7134114190 con 0x7f7134103ff0 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f713a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7134114010 con 0x7f7134109120 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f713a600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7134114310 con 0x7f71341049a0 2024-09-05T12:24:43.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f7133400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 0x7f713419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f7133400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 0x7f713419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33688/0 (socket says 172.21.15.71:33688) 2024-09-05T12:24:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f7133400640 1 -- 172.21.15.71:0/3991111348 learned_addr learned my addr 172.21.15.71:0/3991111348 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f712aa00640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71341049a0 0x7f713419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.951+0000 7f7133e00640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7134109120 0x7f713419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:43.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.952+0000 7f7133400640 1 -- 172.21.15.71:0/3991111348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71341049a0 msgr2=0x7f713419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.952+0000 7f7133400640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71341049a0 0x7f713419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.952+0000 7f7133400640 1 -- 172.21.15.71:0/3991111348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7134109120 msgr2=0x7f713419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.952+0000 7f7133400640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7134109120 0x7f713419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.952+0000 7f7133400640 1 -- 172.21.15.71:0/3991111348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713419f5a0 con 0x7f7134103ff0 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.952+0000 7f7133400640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 0x7f713419a5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f712400e9f0 tx=0x7f712400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.953+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712400cd80 con 0x7f7134103ff0 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.953+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7124010910 con 0x7f7134103ff0 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.953+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71240188f0 con 0x7f7134103ff0 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.953+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713419f890 con 0x7f7134103ff0 2024-09-05T12:24:43.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.953+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71341a7170 con 0x7f7134103ff0 2024-09-05T12:24:43.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.954+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71341054d0 con 0x7f7134103ff0 2024-09-05T12:24:43.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.956+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7124010ac0 con 0x7f7134103ff0 2024-09-05T12:24:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.957+0000 7f7130c00640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71180a2c00 0x7f71180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.957+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7127+0+0 (secure 0 0 0) 0x7f71240cef00 con 0x7f7134103ff0 2024-09-05T12:24:43.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.958+0000 7f712aa00640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71180a2c00 0x7f71180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:43.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.958+0000 7f712aa00640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71180a2c00 0x7f71180a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:43.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:43.963+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f712409bc10 con 0x7f7134103ff0 2024-09-05T12:24:44.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:44.173+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f71341a7340 con 0x7f7134103ff0 2024-09-05T12:24:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:44 smithi174 ceph-mon[34313]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.014+0000 7f7130c00640 1 -- 172.21.15.71:0/3991111348 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f712409ff10 con 0x7f7134103ff0 2024-09-05T12:24:45.016 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' created 2024-09-05T12:24:45.016 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:44 smithi071 ceph-mon[29845]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:45.016 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.016 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.016 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:45.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:45.017 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71180a2c00 msgr2=0x7f71180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71180a2c00 0x7f71180a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f711c000c00 tx=0x7f711c005b50 comp rx=0 tx=0).stop 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 msgr2=0x7f713419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 0x7f713419a5f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f712400e9f0 tx=0x7f712400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 shutdown_connections 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f71180a2c00 0x7f71180a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7134109120 0x7f713419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71341049a0 0x7f713419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 --2- 172.21.15.71:0/3991111348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7134103ff0 0x7f713419a5f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 >> 172.21.15.71:0/3991111348 conn(0x7f71340ffa90 msgr2=0x7f7134101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:45.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 shutdown_connections 2024-09-05T12:24:45.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:45.017+0000 7f713a600640 1 -- 172.21.15.71:0/3991111348 wait complete. 2024-09-05T12:24:45.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:44 smithi123 ceph-mon[34321]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:45.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:24:45.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:24:45.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:24:45.819 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-05T12:24:46.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:45 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3991111348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T12:24:46.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:45 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3991111348' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T12:24:46.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:45 smithi174 ceph-mon[34313]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T12:24:46.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:45 smithi174 ceph-mon[34313]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:46.081 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:45 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3991111348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T12:24:46.081 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:45 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3991111348' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T12:24:46.081 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:45 smithi123 ceph-mon[34321]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T12:24:46.081 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:45 smithi123 ceph-mon[34321]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:46.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:45 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3991111348' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-05T12:24:46.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:45 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3991111348' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-05T12:24:46.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:45 smithi071 ceph-mon[29845]: osdmap e80: 12 total, 12 up, 12 in 2024-09-05T12:24:46.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:45 smithi071 ceph-mon[29845]: pgmap v286: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:46.200 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:46.796 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:46 smithi071 ceph-mon[29845]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T12:24:46.796 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:46 smithi071 ceph-mon[29845]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T12:24:46.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.793+0000 7f3c89c00640 1 -- 172.21.15.71:0/1413419385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 msgr2=0x7f3c84074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:46.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.793+0000 7f3c89c00640 1 --2- 172.21.15.71:0/1413419385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c84074a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009900 tx=0x7f3c7402a480 comp rx=0 tx=0).stop 2024-09-05T12:24:46.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 -- 172.21.15.71:0/1413419385 shutdown_connections 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 --2- 172.21.15.71:0/1413419385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c8410e3c0 0x7f3c84110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 --2- 172.21.15.71:0/1413419385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c84074f40 0x7f3c8410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 --2- 172.21.15.71:0/1413419385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c84074a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 -- 172.21.15.71:0/1413419385 >> 172.21.15.71:0/1413419385 conn(0x7f3c840fd8c0 msgr2=0x7f3c840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 -- 172.21.15.71:0/1413419385 shutdown_connections 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.794+0000 7f3c89c00640 1 -- 172.21.15.71:0/1413419385 wait complete. 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 Processor -- start 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 -- start start 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c8419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c84074f40 0x7f3c8419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c8410e3c0 0x7f3c8419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:46.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3c841141c0 con 0x7f3c8410e3c0 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84114040 con 0x7f3c84074f40 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c89c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c84114340 con 0x7f3c84074620 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c82a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c8419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.795+0000 7f3c82a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c8419a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39530/0 (socket says 172.21.15.71:39530) 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 -- 172.21.15.71:0/2622412991 learned_addr learned my addr 172.21.15.71:0/2622412991 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c83400640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c8410e3c0 0x7f3c8419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82000640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c84074f40 0x7f3c8419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 -- 172.21.15.71:0/2622412991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c84074f40 msgr2=0x7f3c8419ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c84074f40 0x7f3c8419ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 -- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c8410e3c0 msgr2=0x7f3c8419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c8410e3c0 0x7f3c8419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 -- 172.21.15.71:0/2622412991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8419f6d0 con 0x7f3c84074620 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c82a00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c8419a720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c74004770 tx=0x7f3c74003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:46.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.796+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7403f070 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.797+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c74031ca0 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.797+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8419f960 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.797+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74035030 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.797+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c841a72a0 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.798+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c84074a00 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.799+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3c74042070 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.801+0000 7f3c7b400640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3c680a2c00 0x7f3c680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.801+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7497+0+0 (secure 0 0 0) 0x7f3c740ef500 con 0x7f3c84074620 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.801+0000 7f3c82000640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3c680a2c00 0x7f3c680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.801+0000 7f3c82000640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3c680a2c00 0x7f3c680a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c8410f7d0 tx=0x7f3c70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:46.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.803+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c740bc160 con 0x7f3c84074620 2024-09-05T12:24:46.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:46.979+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f3c84109540 con 0x7f3c84074620 2024-09-05T12:24:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:46 smithi174 ceph-mon[34313]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T12:24:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:46 smithi174 ceph-mon[34313]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T12:24:47.093 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:46 smithi123 ceph-mon[34321]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T12:24:47.093 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:46 smithi123 ceph-mon[34321]: osdmap e81: 12 total, 12 up, 12 in 2024-09-05T12:24:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.499+0000 7f3c7b400640 1 -- 172.21.15.71:0/2622412991 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) ==== 171+0+0 (secure 0 0 0) 0x7f3c740c0460 con 0x7f3c84074620 2024-09-05T12:24:47.502 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' created 2024-09-05T12:24:47.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3c680a2c00 msgr2=0x7f3c680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:47.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3c680a2c00 0x7f3c680a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3c8410f7d0 tx=0x7f3c70005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:47.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 msgr2=0x7f3c8419a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:47.509 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c8419a720 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3c74004770 tx=0x7f3c74003c10 comp rx=0 tx=0).stop 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 shutdown_connections 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3c680a2c00 0x7f3c680a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c8410e3c0 0x7f3c8419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c84074f40 0x7f3c8419ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 --2- 172.21.15.71:0/2622412991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c84074620 0x7f3c8419a720 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 >> 172.21.15.71:0/2622412991 conn(0x7f3c840fd8c0 msgr2=0x7f3c840fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.507+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 shutdown_connections 2024-09-05T12:24:47.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:47.508+0000 7f3c89c00640 1 -- 172.21.15.71:0/2622412991 wait complete. 2024-09-05T12:24:47.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:47 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2622412991' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T12:24:47.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:47 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T12:24:47.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:47 smithi071 ceph-mon[29845]: pgmap v288: 33 pgs: 27 unknown, 6 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:47.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:47 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T12:24:47.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:47 smithi071 ceph-mon[29845]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T12:24:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:47 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2622412991' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T12:24:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:47 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T12:24:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:47 smithi174 ceph-mon[34313]: pgmap v288: 33 pgs: 27 unknown, 6 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:47 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T12:24:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:47 smithi174 ceph-mon[34313]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T12:24:48.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:47 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2622412991' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T12:24:48.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:47 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-05T12:24:48.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:47 smithi123 ceph-mon[34321]: pgmap v288: 33 pgs: 27 unknown, 6 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:48.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:47 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-05T12:24:48.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:47 smithi123 ceph-mon[34321]: osdmap e82: 12 total, 12 up, 12 in 2024-09-05T12:24:48.345 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-05T12:24:48.742 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:49.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.405+0000 7f75ba200640 1 -- 172.21.15.71:0/3410206594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 msgr2=0x7f75b41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:49.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.405+0000 7f75ba200640 1 --2- 172.21.15.71:0/3410206594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b41021c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009900 tx=0x7f75a402a410 comp rx=0 tx=0).stop 2024-09-05T12:24:49.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 -- 172.21.15.71:0/3410206594 shutdown_connections 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 --2- 172.21.15.71:0/3410206594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b4106f10 0x7f75b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 --2- 172.21.15.71:0/3410206594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4102790 0x7f75b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 --2- 172.21.15.71:0/3410206594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b41021c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 -- 172.21.15.71:0/3410206594 >> 172.21.15.71:0/3410206594 conn(0x7f75b40fd8c0 msgr2=0x7f75b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 -- 172.21.15.71:0/3410206594 shutdown_connections 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.406+0000 7f75ba200640 1 -- 172.21.15.71:0/3410206594 wait complete. 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 Processor -- start 2024-09-05T12:24:49.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 -- start start 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b4102790 0x7f75b419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4106f10 0x7f75b419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75b4111fc0 con 0x7f75b4101de0 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75b4111e40 con 0x7f75b4106f10 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75ba200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75b4112140 con 0x7f75b4102790 2024-09-05T12:24:49.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b419a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33720/0 (socket says 172.21.15.71:33720) 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75b3400640 1 -- 172.21.15.71:0/2407998576 learned_addr learned my addr 172.21.15.71:0/2407998576 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75b3e00640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4106f10 0x7f75b419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.407+0000 7f75b2a00640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b4102790 0x7f75b419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3400640 1 -- 172.21.15.71:0/2407998576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b4102790 msgr2=0x7f75b419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3400640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b4102790 0x7f75b419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3400640 1 -- 172.21.15.71:0/2407998576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4106f10 msgr2=0x7f75b419ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3400640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4106f10 0x7f75b419ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:49.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3400640 1 -- 172.21.15.71:0/2407998576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b419f610 con 0x7f75b4101de0 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3e00640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4106f10 0x7f75b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75b3400640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b419a660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75a4004770 tx=0x7f75a4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a403f070 con 0x7f75b4101de0 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75a4031ca0 con 0x7f75b4101de0 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4035030 con 0x7f75b4101de0 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.408+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b419f8a0 con 0x7f75b4101de0 2024-09-05T12:24:49.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.409+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b419fd30 con 0x7f75b4101de0 2024-09-05T12:24:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.410+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b41021c0 con 0x7f75b4101de0 2024-09-05T12:24:49.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.412+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75a4042070 con 0x7f75b4101de0 2024-09-05T12:24:49.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.413+0000 7f75abe00640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f75980a2b80 0x7f75980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:49.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.413+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f75a40efcb0 con 0x7f75b4101de0 2024-09-05T12:24:49.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.413+0000 7f75b2a00640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f75980a2b80 0x7f75980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:49.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.414+0000 7f75b2a00640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f75980a2b80 0x7f75980a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75b410eee0 tx=0x7f75a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:49.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.416+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75a40bcd20 con 0x7f75b4101de0 2024-09-05T12:24:49.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:49.629+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f75b41a7340 con 0x7f75b4101de0 2024-09-05T12:24:49.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:49 smithi071 ceph-mon[29845]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T12:24:49.632 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:49 smithi071 ceph-mon[29845]: pgmap v291: 97 pgs: 12 creating+peering, 69 unknown, 16 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:49.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:49 smithi174 ceph-mon[34313]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T12:24:49.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:49 smithi174 ceph-mon[34313]: pgmap v291: 97 pgs: 12 creating+peering, 69 unknown, 16 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:49.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:49 smithi123 ceph-mon[34321]: osdmap e83: 12 total, 12 up, 12 in 2024-09-05T12:24:49.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:49 smithi123 ceph-mon[34321]: pgmap v291: 97 pgs: 12 creating+peering, 69 unknown, 16 active+clean; 577 KiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:50.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.504+0000 7f75abe00640 1 -- 172.21.15.71:0/2407998576 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f75a40bc620 con 0x7f75b4101de0 2024-09-05T12:24:50.507 INFO:teuthology.orchestra.run.smithi071.stderr:new fs with metadata pool 2 and data pool 3 2024-09-05T12:24:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f75980a2b80 msgr2=0x7f75980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f75980a2b80 0x7f75980a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75b410eee0 tx=0x7f75a0005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 msgr2=0x7f75b419a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b419a660 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75a4004770 tx=0x7f75a4003c10 comp rx=0 tx=0).stop 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 shutdown_connections 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f75980a2b80 0x7f75980a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75b4106f10 0x7f75b419ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75b4102790 0x7f75b419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 --2- 172.21.15.71:0/2407998576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75b4101de0 0x7f75b419a660 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.509+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 >> 172.21.15.71:0/2407998576 conn(0x7f75b40fd8c0 msgr2=0x7f75b40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.510+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 shutdown_connections 2024-09-05T12:24:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:50.510+0000 7f75ba200640 1 -- 172.21.15.71:0/2407998576 wait complete. 2024-09-05T12:24:50.915 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:50 smithi071 ceph-mon[29845]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T12:24:50.916 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:50 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2407998576' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T12:24:50.916 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:50 smithi071 ceph-mon[29845]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T12:24:50.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:50 smithi123 ceph-mon[34321]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T12:24:50.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:50 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2407998576' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T12:24:50.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:50 smithi123 ceph-mon[34321]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T12:24:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:50 smithi174 ceph-mon[34313]: osdmap e84: 12 total, 12 up, 12 in 2024-09-05T12:24:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:50 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2407998576' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-05T12:24:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:50 smithi174 ceph-mon[34313]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-05T12:24:51.469 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-05T12:24:51.470 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-05T12:24:51.845 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:51.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:51 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2407998576' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T12:24:51.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:51 smithi071 ceph-mon[29845]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T12:24:51.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:51 smithi071 ceph-mon[29845]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T12:24:51.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:51 smithi071 ceph-mon[29845]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T12:24:51.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:51 smithi071 ceph-mon[29845]: pgmap v294: 97 pgs: 12 creating+peering, 5 unknown, 80 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:51.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:51 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2407998576' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T12:24:51.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:51 smithi123 ceph-mon[34321]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T12:24:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:51 smithi123 ceph-mon[34321]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T12:24:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:51 smithi123 ceph-mon[34321]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T12:24:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:51 smithi123 ceph-mon[34321]: pgmap v294: 97 pgs: 12 creating+peering, 5 unknown, 80 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:51 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2407998576' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-05T12:24:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:51 smithi174 ceph-mon[34313]: osdmap e85: 12 total, 12 up, 12 in 2024-09-05T12:24:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:51 smithi174 ceph-mon[34313]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-05T12:24:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:51 smithi174 ceph-mon[34313]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-05T12:24:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:51 smithi174 ceph-mon[34313]: pgmap v294: 97 pgs: 12 creating+peering, 5 unknown, 80 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:24:52.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 -- 172.21.15.71:0/3962365343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744103e30 msgr2=0x7f7744104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 --2- 172.21.15.71:0/3962365343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744103e30 0x7f7744104210 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7734009900 tx=0x7f773402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 -- 172.21.15.71:0/3962365343 shutdown_connections 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 --2- 172.21.15.71:0/3962365343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7744108f60 0x7f7744110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 --2- 172.21.15.71:0/3962365343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77441047e0 0x7f7744108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 --2- 172.21.15.71:0/3962365343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744103e30 0x7f7744104210 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 -- 172.21.15.71:0/3962365343 >> 172.21.15.71:0/3962365343 conn(0x7f77440ff8d0 msgr2=0x7f7744101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 -- 172.21.15.71:0/3962365343 shutdown_connections 2024-09-05T12:24:52.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.458+0000 7f7749a00640 1 -- 172.21.15.71:0/3962365343 wait complete. 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 Processor -- start 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 -- start start 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 0x7f774419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77441047e0 0x7f774419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744108f60 0x7f774419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7744113fc0 con 0x7f7744103e30 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7744113e40 con 0x7f7744108f60 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7744114140 con 0x7f77441047e0 2024-09-05T12:24:52.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7743e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 0x7f774419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7743e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 0x7f774419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33738/0 (socket says 172.21.15.71:33738) 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7743e00640 1 -- 172.21.15.71:0/1897423196 learned_addr learned my addr 172.21.15.71:0/1897423196 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f7749000640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744108f60 0x7f774419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.459+0000 7f773b400640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77441047e0 0x7f774419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7743e00640 1 -- 172.21.15.71:0/1897423196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77441047e0 msgr2=0x7f774419a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7743e00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77441047e0 0x7f774419a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7743e00640 1 -- 172.21.15.71:0/1897423196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744108f60 msgr2=0x7f774419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7743e00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744108f60 0x7f774419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7743e00640 1 -- 172.21.15.71:0/1897423196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774419f3f0 con 0x7f7744103e30 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7743e00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 0x7f774419a440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7734006010 tx=0x7f7734004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773403f030 con 0x7f7744103e30 2024-09-05T12:24:52.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774419f680 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7734004440 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.460+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734035070 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.461+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77441a6fc0 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.462+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7744105310 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.463+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7734042030 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.464+0000 7f7741600640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f77280a2b80 0x7f77280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.465+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f77340ef2d0 con 0x7f7744103e30 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.465+0000 7f773b400640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f77280a2b80 0x7f77280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:52.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.465+0000 7f773b400640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f77280a2b80 0x7f77280a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f772c0095f0 tx=0x7f772c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:52.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.469+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77340bbc90 con 0x7f7744103e30 2024-09-05T12:24:52.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:52.656+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f774410c140 con 0x7f7744103e30 2024-09-05T12:24:52.658 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:52 smithi071 ceph-mon[29845]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T12:24:52.658 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:52 smithi071 ceph-mon[29845]: Cluster is now healthy 2024-09-05T12:24:52.658 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:52 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] up:active 2024-09-05T12:24:52.658 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:52 smithi071 ceph-mon[29845]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:24:52.658 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:52 smithi071 ceph-mon[29845]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:24:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:52 smithi123 ceph-mon[34321]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T12:24:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:52 smithi123 ceph-mon[34321]: Cluster is now healthy 2024-09-05T12:24:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:52 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] up:active 2024-09-05T12:24:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:52 smithi123 ceph-mon[34321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:24:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:52 smithi123 ceph-mon[34321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:24:52.680 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:24:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: 2024-09-05T12:24:52.300+0000 7ff641ab1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-05T12:24:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:52 smithi174 ceph-mon[34313]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-05T12:24:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:52 smithi174 ceph-mon[34313]: Cluster is now healthy 2024-09-05T12:24:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:52 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] up:active 2024-09-05T12:24:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:52 smithi174 ceph-mon[34313]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:24:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:52 smithi174 ceph-mon[34313]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:24:53.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.531+0000 7f7741600640 1 -- 172.21.15.71:0/1897423196 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7f77340bff90 con 0x7f7744103e30 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.533+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f77280a2b80 msgr2=0x7f77280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.533+0000 7f7749a00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f77280a2b80 0x7f77280a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f772c0095f0 tx=0x7f772c005f90 comp rx=0 tx=0).stop 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 msgr2=0x7f774419a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 0x7f774419a440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7734006010 tx=0x7f7734004040 comp rx=0 tx=0).stop 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 shutdown_connections 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f77280a2b80 0x7f77280a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7744108f60 0x7f774419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77441047e0 0x7f774419a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 --2- 172.21.15.71:0/1897423196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7744103e30 0x7f774419a440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:53.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 >> 172.21.15.71:0/1897423196 conn(0x7f77440ff8d0 msgr2=0x7f7744100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:53.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 shutdown_connections 2024-09-05T12:24:53.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:53.534+0000 7f7749a00640 1 -- 172.21.15.71:0/1897423196 wait complete. 2024-09-05T12:24:53.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:53 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1897423196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T12:24:53.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:53 smithi071 ceph-mon[29845]: pgmap v295: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2024-09-05T12:24:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:53 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1897423196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T12:24:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:53 smithi123 ceph-mon[34321]: pgmap v295: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2024-09-05T12:24:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:53 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1897423196' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-05T12:24:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:53 smithi174 ceph-mon[34313]: pgmap v295: 97 pgs: 4 creating+peering, 93 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 363 B/s wr, 0 op/s 2024-09-05T12:24:54.599 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-05T12:24:54.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:54 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1897423196' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T12:24:54.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:54 smithi071 ceph-mon[29845]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T12:24:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:54 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1897423196' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T12:24:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:54 smithi123 ceph-mon[34321]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T12:24:54.964 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:54 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1897423196' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-05T12:24:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:54 smithi174 ceph-mon[34313]: osdmap e86: 12 total, 12 up, 12 in 2024-09-05T12:24:55.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 -- 172.21.15.71:0/2029261458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 msgr2=0x7f66e0104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 --2- 172.21.15.71:0/2029261458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e0104230 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009860 tx=0x7f66d002a340 comp rx=0 tx=0).stop 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 -- 172.21.15.71:0/2029261458 shutdown_connections 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 --2- 172.21.15.71:0/2029261458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e0109090 0x7f66e0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 --2- 172.21.15.71:0/2029261458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 0x7f66e0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 --2- 172.21.15.71:0/2029261458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e0104230 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 -- 172.21.15.71:0/2029261458 >> 172.21.15.71:0/2029261458 conn(0x7f66e00ff8b0 msgr2=0x7f66e0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.589+0000 7f66df200640 1 -- 172.21.15.71:0/2029261458 shutdown_connections 2024-09-05T12:24:55.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.590+0000 7f66df200640 1 -- 172.21.15.71:0/2029261458 wait complete. 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.590+0000 7f66df200640 1 Processor -- start 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 -- start start 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 0x7f66e019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e0109090 0x7f66e019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01140a0 con 0x7f66e0103e50 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0113f20 con 0x7f66e0104800 2024-09-05T12:24:55.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66df200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0114220 con 0x7f66e0109090 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66dde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66dde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55262/0 (socket says 172.21.15.71:55262) 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66dde00640 1 -- 172.21.15.71:0/2649994116 learned_addr learned my addr 172.21.15.71:0/2649994116 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66dd400640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 0x7f66e019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.591+0000 7f66de800640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e0109090 0x7f66e019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dde00640 1 -- 172.21.15.71:0/2649994116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e0109090 msgr2=0x7f66e019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dde00640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e0109090 0x7f66e019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dde00640 1 -- 172.21.15.71:0/2649994116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 msgr2=0x7f66e019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dde00640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 0x7f66e019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dde00640 1 -- 172.21.15.71:0/2649994116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e019f530 con 0x7f66e0103e50 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dd400640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 0x7f66e019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66dde00640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e019a580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66d0035040 tx=0x7f66d0033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d003f020 con 0x7f66e0103e50 2024-09-05T12:24:55.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66d0032770 con 0x7f66e0103e50 2024-09-05T12:24:55.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0037040 con 0x7f66e0103e50 2024-09-05T12:24:55.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e019f7c0 con 0x7f66e0103e50 2024-09-05T12:24:55.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.592+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01a7100 con 0x7f66e0103e50 2024-09-05T12:24:55.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.594+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e0105330 con 0x7f66e0103e50 2024-09-05T12:24:55.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.595+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f66d0042020 con 0x7f66e0103e50 2024-09-05T12:24:55.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.597+0000 7f66d6a00640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f66c40a2c00 0x7f66c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:55.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.597+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f66d00ef1a0 con 0x7f66e0103e50 2024-09-05T12:24:55.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.597+0000 7f66dd400640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f66c40a2c00 0x7f66c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:55.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.598+0000 7f66dd400640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f66c40a2c00 0x7f66c40a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ffcd0 tx=0x7f66cc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:55.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.602+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66d0049020 con 0x7f66e0103e50 2024-09-05T12:24:55.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:55.776+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f66e010c270 con 0x7f66e0103e50 2024-09-05T12:24:55.778 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:55 smithi071 ceph-mon[29845]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-05T12:24:55.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:55 smithi123 ceph-mon[34321]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-05T12:24:56.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:55 smithi174 ceph-mon[34313]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-05T12:24:56.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2649994116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T12:24:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2649994116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T12:24:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2649994116' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-05T12:24:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.551+0000 7f66d6a00640 1 -- 172.21.15.71:0/2649994116 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f66d00bc070 con 0x7f66e0103e50 2024-09-05T12:24:57.553 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data_ec' created 2024-09-05T12:24:57.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f66c40a2c00 msgr2=0x7f66c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f66c40a2c00 0x7f66c40a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f66e00ffcd0 tx=0x7f66cc009260 comp rx=0 tx=0).stop 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 msgr2=0x7f66e019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e019a580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f66d0035040 tx=0x7f66d0033d90 comp rx=0 tx=0).stop 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 shutdown_connections 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f66c40a2c00 0x7f66c40a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66e0109090 0x7f66e019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66e0104800 0x7f66e019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 --2- 172.21.15.71:0/2649994116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66e0103e50 0x7f66e019a580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.555+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 >> 172.21.15.71:0/2649994116 conn(0x7f66e00ff8b0 msgr2=0x7f66e0100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.556+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 shutdown_connections 2024-09-05T12:24:57.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:57.556+0000 7f66df200640 1 -- 172.21.15.71:0/2649994116 wait complete. 2024-09-05T12:24:57.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:57 smithi174 ceph-mon[34313]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T12:24:57.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:57 smithi174 ceph-mon[34313]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-05T12:24:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:57 smithi071 ceph-mon[29845]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T12:24:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:57 smithi071 ceph-mon[29845]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-05T12:24:57.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:57 smithi123 ceph-mon[34321]: osdmap e87: 12 total, 12 up, 12 in 2024-09-05T12:24:57.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:57 smithi123 ceph-mon[34321]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-05T12:24:58.529 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-05T12:24:58.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:58 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2649994116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T12:24:58.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:58 smithi071 ceph-mon[29845]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T12:24:58.912 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:24:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:58 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2649994116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T12:24:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:58 smithi123 ceph-mon[34321]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T12:24:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:58 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2649994116' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-05T12:24:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:58 smithi174 ceph-mon[34313]: osdmap e88: 12 total, 12 up, 12 in 2024-09-05T12:24:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 -- 172.21.15.71:0/2913242734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683410e210 msgr2=0x7f6834110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 --2- 172.21.15.71:0/2913242734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683410e210 0x7f6834110660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f682c009ca0 tx=0x7f682c02a950 comp rx=0 tx=0).stop 2024-09-05T12:24:59.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 -- 172.21.15.71:0/2913242734 shutdown_connections 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 --2- 172.21.15.71:0/2913242734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f683410e210 0x7f6834110660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 --2- 172.21.15.71:0/2913242734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6834074f40 0x7f683410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 --2- 172.21.15.71:0/2913242734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 0x7f6834074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 -- 172.21.15.71:0/2913242734 >> 172.21.15.71:0/2913242734 conn(0x7f68340fd710 msgr2=0x7f68340ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 -- 172.21.15.71:0/2913242734 shutdown_connections 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.491+0000 7f6838a00640 1 -- 172.21.15.71:0/2913242734 wait complete. 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 Processor -- start 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 -- start start 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 0x7f683419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 0x7f683419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683410e210 0x7f683419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:59.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6834113fa0 con 0x7f6834074f40 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6834113e20 con 0x7f6834074620 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.492+0000 7f6838a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6834114120 con 0x7f683410e210 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 0x7f683419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 0x7f683419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55284/0 (socket says 172.21.15.71:55284) 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 -- 172.21.15.71:0/2411146435 learned_addr learned my addr 172.21.15.71:0/2411146435 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6833400640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 0x7f683419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6833e00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683410e210 0x7f683419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 -- 172.21.15.71:0/2411146435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683410e210 msgr2=0x7f683419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683410e210 0x7f683419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 -- 172.21.15.71:0/2411146435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 msgr2=0x7f683419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 0x7f683419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:24:59.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 -- 172.21.15.71:0/2411146435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683419f3e0 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6833e00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683410e210 0x7f683419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6833400640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 0x7f683419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f6832a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 0x7f683419a970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9f0 tx=0x7f682000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.493+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682000cd80 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.494+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6820005500 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.494+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6820010690 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.494+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683419f6d0 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.494+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a6fb0 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.495+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6834074a00 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.497+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f68200046c0 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.499+0000 7f682be00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f68180a2c00 0x7f68180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.499+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f68200cf870 con 0x7f6834074f40 2024-09-05T12:24:59.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.499+0000 7f6833400640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f68180a2c00 0x7f68180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:24:59.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.499+0000 7f6833400640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f68180a2c00 0x7f68180a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6824009fc0 tx=0x7f6824005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:24:59.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.501+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68200541b0 con 0x7f6834074f40 2024-09-05T12:24:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:59 smithi071 ceph-mon[29845]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T12:24:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:24:59 smithi071 ceph-mon[29845]: pgmap v302: 129 pgs: 6 creating+peering, 22 unknown, 101 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 914 B/s wr, 6 op/s 2024-09-05T12:24:59.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:24:59.678+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f683419b790 con 0x7f6834074f40 2024-09-05T12:24:59.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:59 smithi123 ceph-mon[34321]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T12:24:59.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:24:59 smithi123 ceph-mon[34321]: pgmap v302: 129 pgs: 6 creating+peering, 22 unknown, 101 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 914 B/s wr, 6 op/s 2024-09-05T12:25:00.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:59 smithi174 ceph-mon[34313]: osdmap e89: 12 total, 12 up, 12 in 2024-09-05T12:25:00.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:24:59 smithi174 ceph-mon[34313]: pgmap v302: 129 pgs: 6 creating+peering, 22 unknown, 101 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 914 B/s wr, 6 op/s 2024-09-05T12:25:00.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.573+0000 7f682be00640 1 -- 172.21.15.71:0/2411146435 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v91) ==== 169+0+0 (secure 0 0 0) 0x7f683419b790 con 0x7f6834074f40 2024-09-05T12:25:00.575 INFO:teuthology.orchestra.run.smithi071.stderr:set pool 4 allow_ec_overwrites to true 2024-09-05T12:25:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f68180a2c00 msgr2=0x7f68180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f68180a2c00 0x7f68180a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6824009fc0 tx=0x7f6824005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 msgr2=0x7f683419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:00.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 0x7f683419a970 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f682000e9f0 tx=0x7f682000edf0 comp rx=0 tx=0).stop 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 shutdown_connections 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f68180a2c00 0x7f68180a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683410e210 0x7f683419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6834074f40 0x7f683419a970 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 --2- 172.21.15.71:0/2411146435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6834074620 0x7f683419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 >> 172.21.15.71:0/2411146435 conn(0x7f68340fd710 msgr2=0x7f68340fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 shutdown_connections 2024-09-05T12:25:00.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:00.577+0000 7f6838a00640 1 -- 172.21.15.71:0/2411146435 wait complete. 2024-09-05T12:25:00.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:00 smithi071 ceph-mon[29845]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T12:25:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:00 smithi071 ceph-mon[29845]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T12:25:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:00 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2411146435' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T12:25:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:00 smithi123 ceph-mon[34321]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T12:25:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:00 smithi123 ceph-mon[34321]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T12:25:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:00 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2411146435' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T12:25:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:00 smithi174 ceph-mon[34313]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-05T12:25:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:00 smithi174 ceph-mon[34313]: osdmap e90: 12 total, 12 up, 12 in 2024-09-05T12:25:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:00 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2411146435' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-05T12:25:01.389 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-05T12:25:01.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:01 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2411146435' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T12:25:01.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:01 smithi071 ceph-mon[29845]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T12:25:01.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:01 smithi071 ceph-mon[29845]: pgmap v305: 129 pgs: 6 creating+peering, 4 unknown, 119 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:01.758 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:01.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:01 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2411146435' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T12:25:01.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:01 smithi123 ceph-mon[34321]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T12:25:01.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:01 smithi123 ceph-mon[34321]: pgmap v305: 129 pgs: 6 creating+peering, 4 unknown, 119 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:02.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:01 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2411146435' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-05T12:25:02.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:01 smithi174 ceph-mon[34313]: osdmap e91: 12 total, 12 up, 12 in 2024-09-05T12:25:02.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:01 smithi174 ceph-mon[34313]: pgmap v305: 129 pgs: 6 creating+peering, 4 unknown, 119 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 -- 172.21.15.71:0/627029958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3674074620 msgr2=0x7f3674074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 --2- 172.21.15.71:0/627029958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3674074620 0x7f3674074a00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3664009900 tx=0x7f366402a480 comp rx=0 tx=0).stop 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 -- 172.21.15.71:0/627029958 shutdown_connections 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 --2- 172.21.15.71:0/627029958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f367410e3c0 0x7f3674110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 --2- 172.21.15.71:0/627029958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367410de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 --2- 172.21.15.71:0/627029958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3674074620 0x7f3674074a00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 -- 172.21.15.71:0/627029958 >> 172.21.15.71:0/627029958 conn(0x7f36740fd8c0 msgr2=0x7f36740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 -- 172.21.15.71:0/627029958 shutdown_connections 2024-09-05T12:25:02.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.364+0000 7f3679600640 1 -- 172.21.15.71:0/627029958 wait complete. 2024-09-05T12:25:02.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 Processor -- start 2024-09-05T12:25:02.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 -- start start 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3674074620 0x7f367419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f367410e3c0 0x7f367419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36741141c0 con 0x7f367410e3c0 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3674114040 con 0x7f3674074f40 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3679600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3674114340 con 0x7f3674074620 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.365+0000 7f3673400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f367410e3c0 0x7f367419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3674074620 0x7f367419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f366be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f366be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45234/0 (socket says 172.21.15.71:45234) 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f366be00640 1 -- 172.21.15.71:0/3473896178 learned_addr learned my addr 172.21.15.71:0/3473896178 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 -- 172.21.15.71:0/3473896178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 msgr2=0x7f367419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:02.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 -- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f367410e3c0 msgr2=0x7f367419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f367410e3c0 0x7f367419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 -- 172.21.15.71:0/3473896178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367419f640 con 0x7f3674074620 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f366be00640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.366+0000 7f3672a00640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3674074620 0x7f367419a690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f366402a9a0 tx=0x7f3664003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.367+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366403f070 con 0x7f3674074620 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.367+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367419f8d0 con 0x7f3674074620 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.367+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3664031c90 con 0x7f3674074620 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.367+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664035030 con 0x7f3674074620 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.367+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36741a7210 con 0x7f3674074620 2024-09-05T12:25:02.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.368+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3674074a00 con 0x7f3674074620 2024-09-05T12:25:02.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.370+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3664042070 con 0x7f3674074620 2024-09-05T12:25:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.371+0000 7f366b400640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f36580a2b80 0x7f36580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.371+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7f36640f0150 con 0x7f3674074620 2024-09-05T12:25:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.371+0000 7f366be00640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f36580a2b80 0x7f36580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:02.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.372+0000 7f366be00640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f36580a2b80 0x7f36580a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f367410f7b0 tx=0x7f365c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:02.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.373+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f36640bc840 con 0x7f3674074620 2024-09-05T12:25:02.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.586+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f367419b9f0 con 0x7f3674074620 2024-09-05T12:25:02.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.644+0000 7f366b400640 1 -- 172.21.15.71:0/3473896178 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f367419b9f0 con 0x7f3674074620 2024-09-05T12:25:02.646 INFO:teuthology.orchestra.run.smithi071.stderr:added data pool 4 to fsmap 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f36580a2b80 msgr2=0x7f36580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f36580a2b80 0x7f36580a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f367410f7b0 tx=0x7f365c005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3674074620 msgr2=0x7f367419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3674074620 0x7f367419a690 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f366402a9a0 tx=0x7f3664003b20 comp rx=0 tx=0).stop 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 shutdown_connections 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f36580a2b80 0x7f36580a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f367410e3c0 0x7f367419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3674074f40 0x7f367419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 --2- 172.21.15.71:0/3473896178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3674074620 0x7f367419a690 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:02.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 >> 172.21.15.71:0/3473896178 conn(0x7f36740fd8c0 msgr2=0x7f36740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:02.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 shutdown_connections 2024-09-05T12:25:02.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:02.647+0000 7f3679600640 1 -- 172.21.15.71:0/3473896178 wait complete. 2024-09-05T12:25:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:02 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3473896178' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T12:25:02.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:02 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T12:25:02.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:02 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3473896178' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T12:25:02.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:02 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T12:25:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:02 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3473896178' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T12:25:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:02 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-05T12:25:03.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:25:03.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:03 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T12:25:03.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:03 smithi071 ceph-mon[29845]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T12:25:03.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:03 smithi071 ceph-mon[29845]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:03.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:03 smithi071 ceph-mon[29845]: pgmap v307: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:03 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T12:25:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:03 smithi123 ceph-mon[34321]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T12:25:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:03 smithi123 ceph-mon[34321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:03 smithi123 ceph-mon[34321]: pgmap v307: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:03.966 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:03 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-05T12:25:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:03 smithi174 ceph-mon[34313]: osdmap e92: 12 total, 12 up, 12 in 2024-09-05T12:25:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:03 smithi174 ceph-mon[34313]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:03 smithi174 ceph-mon[34313]: pgmap v307: 129 pgs: 4 creating+peering, 125 active+clean; 579 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:04.691 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:04 smithi071 ceph-mon[29845]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T12:25:04.691 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:04 smithi071 ceph-mon[29845]: Cluster is now healthy 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 -- 172.21.15.71:0/926963362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 msgr2=0x7fd0dc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/926963362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc10e890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd0d4009930 tx=0x7fd0d402a420 comp rx=0 tx=0).stop 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 -- 172.21.15.71:0/926963362 shutdown_connections 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/926963362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc10e890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/926963362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0dc1019c0 0x7fd0dc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/926963362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc1079c0 0x7fd0dc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 -- 172.21.15.71:0/926963362 >> 172.21.15.71:0/926963362 conn(0x7fd0dc0fd6e0 msgr2=0x7fd0dc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 -- 172.21.15.71:0/926963362 shutdown_connections 2024-09-05T12:25:04.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.689+0000 7fd0e1c00640 1 -- 172.21.15.71:0/926963362 wait complete. 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.690+0000 7fd0e1c00640 1 Processor -- start 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.690+0000 7fd0e1c00640 1 -- start start 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.690+0000 7fd0e1c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0dc1019c0 0x7fd0dc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0e1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0e1c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc1079c0 0x7fd0dc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0e1c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111dc0 con 0x7fd0dc102360 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0e1c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111c40 con 0x7fd0dc1079c0 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0e1c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc111f40 con 0x7fd0dc1019c0 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38684/0 (socket says 172.21.15.71:38684) 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 -- 172.21.15.71:0/4260891107 learned_addr learned my addr 172.21.15.71:0/4260891107 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0dbe00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0dc1019c0 0x7fd0dc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0e1200640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc1079c0 0x7fd0dc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:04.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 -- 172.21.15.71:0/4260891107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0dc1019c0 msgr2=0x7fd0dc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0dc1019c0 0x7fd0dc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 -- 172.21.15.71:0/4260891107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc1079c0 msgr2=0x7fd0dc19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc1079c0 0x7fd0dc19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 -- 172.21.15.71:0/4260891107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc19f3f0 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.691+0000 7fd0d3400640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc19a980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400d950 tx=0x7fd0c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.692+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4017070 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.692+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0c4009c30 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.692+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c400f6b0 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.692+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc19f6e0 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.692+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc1a6fc0 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.693+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc101e20 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.695+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd0c40109d0 con 0x7fd0dc102360 2024-09-05T12:25:04.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.696+0000 7fd0d9600640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:04.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.696+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fd0c40ceed0 con 0x7fd0dc102360 2024-09-05T12:25:04.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.696+0000 7fd0dbe00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:04.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.697+0000 7fd0dbe00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005fd0 tx=0x7fd0cc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:04.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.701+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0c4009de0 con 0x7fd0dc102360 2024-09-05T12:25:04.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.910+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0dc1a7260 con 0x7fd0dc102360 2024-09-05T12:25:04.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.910+0000 7fd0d9600640 1 -- 172.21.15.71:0/4260891107 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11833 (secure 0 0 0) 0x7fd0c409bb90 con 0x7fd0dc102360 2024-09-05T12:25:04.913 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:04.913 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"btime":"2024-09-05T12:25:02:639285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:02.589019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-09-05T12:25:04.913 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd0c00a2c00 msgr2=0x7fd0c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc005fd0 tx=0x7fd0cc009500 comp rx=0 tx=0).stop 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 msgr2=0x7fd0dc19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc19a980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400d950 tx=0x7fd0c400de10 comp rx=0 tx=0).stop 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 shutdown_connections 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd0c00a2c00 0x7fd0c00a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0dc1079c0 0x7fd0dc19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd0dc102360 0x7fd0dc19a980 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 --2- 172.21.15.71:0/4260891107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0dc1019c0 0x7fd0dc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:04.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 >> 172.21.15.71:0/4260891107 conn(0x7fd0dc0fd6e0 msgr2=0x7fd0dc0fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:04.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 shutdown_connections 2024-09-05T12:25:04.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:04.913+0000 7fd0e1c00640 1 -- 172.21.15.71:0/4260891107 wait complete. 2024-09-05T12:25:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:04 smithi123 ceph-mon[34321]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T12:25:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:04 smithi123 ceph-mon[34321]: Cluster is now healthy 2024-09-05T12:25:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:04 smithi174 ceph-mon[34313]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-05T12:25:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:04 smithi174 ceph-mon[34313]: Cluster is now healthy 2024-09-05T12:25:05.816 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:25:05.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:05 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4260891107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:05 smithi071 ceph-mon[29845]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:05 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4260891107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:05 smithi123 ceph-mon[34321]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:05 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4260891107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:05 smithi174 ceph-mon[34313]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:06.185 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 -- 172.21.15.71:0/1066939924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4106f10 msgr2=0x7fa5c410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1066939924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4106f10 0x7fa5c410ea90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009ca0 tx=0x7fa5bc02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 -- 172.21.15.71:0/1066939924 shutdown_connections 2024-09-05T12:25:06.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1066939924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4106f10 0x7fa5c410ea90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1066939924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c4102790 0x7fa5c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1066939924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4101de0 0x7fa5c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 -- 172.21.15.71:0/1066939924 >> 172.21.15.71:0/1066939924 conn(0x7fa5c40fd8c0 msgr2=0x7fa5c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 -- 172.21.15.71:0/1066939924 shutdown_connections 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.766+0000 7fa5c9400640 1 -- 172.21.15.71:0/1066939924 wait complete. 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.767+0000 7fa5c9400640 1 Processor -- start 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.767+0000 7fa5c9400640 1 -- start start 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.767+0000 7fa5c9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 0x7fa5c419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4102790 0x7fa5c419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c4106f10 0x7fa5c419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:06.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4111f70 con 0x7fa5c4101de0 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c4111df0 con 0x7fa5c4102790 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c9400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5c41120f0 con 0x7fa5c4106f10 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 0x7fa5c419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 0x7fa5c419a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38702/0 (socket says 172.21.15.71:38702) 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 -- 172.21.15.71:0/1509003341 learned_addr learned my addr 172.21.15.71:0/1509003341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c1e00640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4102790 0x7fa5c419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c3200640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c4106f10 0x7fa5c419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 -- 172.21.15.71:0/1509003341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c4106f10 msgr2=0x7fa5c419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c4106f10 0x7fa5c419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 -- 172.21.15.71:0/1509003341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4102790 msgr2=0x7fa5c419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4102790 0x7fa5c419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 -- 172.21.15.71:0/1509003341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5c419f620 con 0x7fa5c4101de0 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c1e00640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4102790 0x7fa5c419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.768+0000 7fa5c2800640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 0x7fa5c419a670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.769+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4017030 con 0x7fa5c4101de0 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.769+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004020 con 0x7fa5c4101de0 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.769+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b40047a0 con 0x7fa5c4101de0 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.769+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5c419f910 con 0x7fa5c4101de0 2024-09-05T12:25:06.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.769+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5c41a71f0 con 0x7fa5c4101de0 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.770+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5c41021c0 con 0x7fa5c4101de0 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.772+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5b40041d0 con 0x7fa5c4101de0 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.773+0000 7fa5bb400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5a80a2c00 0x7fa5a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.773+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fa5b40ced00 con 0x7fa5c4101de0 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.773+0000 7fa5c1e00640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5a80a2c00 0x7fa5a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.774+0000 7fa5c1e00640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5a80a2c00 0x7fa5a80a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5c410ee50 tx=0x7fa5b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:06.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.777+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5b409b9c0 con 0x7fa5c4101de0 2024-09-05T12:25:06.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.958+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5c41a74a0 con 0x7fa5c4101de0 2024-09-05T12:25:06.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.960+0000 7fa5bb400640 1 -- 172.21.15.71:0/1509003341 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24214 (secure 0 0 0) 0x7fa5b409b2c0 con 0x7fa5c4101de0 2024-09-05T12:25:06.962 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:06.962 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":92,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:02.588840+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:25:06.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.962+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5a80a2c00 msgr2=0x7fa5a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:06.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.962+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5a80a2c00 0x7fa5a80a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa5c410ee50 tx=0x7fa5b0005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.962+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 msgr2=0x7fa5c419a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.962+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 0x7fa5c419a670 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5b400d950 tx=0x7fa5b400de10 comp rx=0 tx=0).stop 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 shutdown_connections 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5a80a2c00 0x7fa5a80a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5c4106f10 0x7fa5c419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5c4102790 0x7fa5c419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 --2- 172.21.15.71:0/1509003341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5c4101de0 0x7fa5c419a670 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 >> 172.21.15.71:0/1509003341 conn(0x7fa5c40fd8c0 msgr2=0x7fa5c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 shutdown_connections 2024-09-05T12:25:06.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:06.963+0000 7fa5c9400640 1 -- 172.21.15.71:0/1509003341 wait complete. 2024-09-05T12:25:07.363 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1509003341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:07.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1509003341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:07.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1509003341' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:07.901 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:25:08.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:08 smithi071 ceph-mon[29845]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:08.268 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:08 smithi123 ceph-mon[34321]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:08 smithi174 ceph-mon[34313]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:08.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 -- 172.21.15.71:0/1575460778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4103e30 msgr2=0x7febc4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:08.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 --2- 172.21.15.71:0/1575460778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4103e30 0x7febc4104210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febb4009900 tx=0x7febb402a410 comp rx=0 tx=0).stop 2024-09-05T12:25:08.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 -- 172.21.15.71:0/1575460778 shutdown_connections 2024-09-05T12:25:08.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 --2- 172.21.15.71:0/1575460778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc4108f60 0x7febc4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:08.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 --2- 172.21.15.71:0/1575460778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febc41047e0 0x7febc4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 --2- 172.21.15.71:0/1575460778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4103e30 0x7febc4104210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 -- 172.21.15.71:0/1575460778 >> 172.21.15.71:0/1575460778 conn(0x7febc40ff910 msgr2=0x7febc4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 -- 172.21.15.71:0/1575460778 shutdown_connections 2024-09-05T12:25:08.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.878+0000 7febc3e00640 1 -- 172.21.15.71:0/1575460778 wait complete. 2024-09-05T12:25:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 Processor -- start 2024-09-05T12:25:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 -- start start 2024-09-05T12:25:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc4103e30 0x7febc419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febc41047e0 0x7febc419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 0x7febc419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:08.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febc4113f80 con 0x7febc4108f60 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febc4113e00 con 0x7febc41047e0 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.879+0000 7febc3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febc4114100 con 0x7febc4103e30 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 0x7febc419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 0x7febc419ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38728/0 (socket says 172.21.15.71:38728) 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 -- 172.21.15.71:0/1043348955 learned_addr learned my addr 172.21.15.71:0/1043348955 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc2a00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc4103e30 0x7febc419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febba000640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febc41047e0 0x7febc419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 -- 172.21.15.71:0/1043348955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc4103e30 msgr2=0x7febc419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc4103e30 0x7febc419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 -- 172.21.15.71:0/1043348955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febc41047e0 msgr2=0x7febc419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febc41047e0 0x7febc419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.880+0000 7febc3400640 1 -- 172.21.15.71:0/1043348955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc419f410 con 0x7febc4108f60 2024-09-05T12:25:08.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.881+0000 7febc3400640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 0x7febc419ed30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7febbc009fd0 tx=0x7febbc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:08.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.881+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc017070 con 0x7febc4108f60 2024-09-05T12:25:08.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.881+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febbc003e50 con 0x7febc4108f60 2024-09-05T12:25:08.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.881+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc0106c0 con 0x7febc4108f60 2024-09-05T12:25:08.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.881+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc419f700 con 0x7febc4108f60 2024-09-05T12:25:08.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.881+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc41a6fe0 con 0x7febc4108f60 2024-09-05T12:25:08.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.882+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc4105310 con 0x7febc4108f60 2024-09-05T12:25:08.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.884+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7febbc003970 con 0x7febc4108f60 2024-09-05T12:25:08.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.886+0000 7febbbe00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feba80a2b80 0x7feba80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:08.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.886+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7febbc0cef40 con 0x7febc4108f60 2024-09-05T12:25:08.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.886+0000 7febc2a00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feba80a2b80 0x7feba80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:08.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.887+0000 7febc2a00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feba80a2b80 0x7feba80a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febb4009230 tx=0x7febb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:08.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:08.890+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febbc09b610 con 0x7febc4108f60 2024-09-05T12:25:09.065 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.064+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7febc4104210 con 0x7febc4108f60 2024-09-05T12:25:09.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.066+0000 7febbbe00640 1 -- 172.21.15.71:0/1043348955 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24214 (secure 0 0 0) 0x7febc4104210 con 0x7febc4108f60 2024-09-05T12:25:09.067 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:09.068 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":92,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:02.588840+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:25:09.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feba80a2b80 msgr2=0x7feba80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:09.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feba80a2b80 0x7feba80a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7febb4009230 tx=0x7febb40058d0 comp rx=0 tx=0).stop 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 msgr2=0x7febc419ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 0x7febc419ed30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7febbc009fd0 tx=0x7febbc00b260 comp rx=0 tx=0).stop 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 shutdown_connections 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feba80a2b80 0x7feba80a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4108f60 0x7febc419ed30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febc41047e0 0x7febc419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 --2- 172.21.15.71:0/1043348955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febc4103e30 0x7febc419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 >> 172.21.15.71:0/1043348955 conn(0x7febc40ff910 msgr2=0x7febc4100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.068+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 shutdown_connections 2024-09-05T12:25:09.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:09.069+0000 7febc3e00640 1 -- 172.21.15.71:0/1043348955 wait complete. 2024-09-05T12:25:09.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:09 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1043348955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:09.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:09 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1043348955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:09.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:09 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1043348955' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:10.001 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:25:10.370 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:10 smithi071 ceph-mon[29845]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:10.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:10 smithi123 ceph-mon[34321]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:10 smithi174 ceph-mon[34313]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:11.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 -- 172.21.15.71:0/2784084474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 msgr2=0x7f7dbc10ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 --2- 172.21.15.71:0/2784084474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 0x7f7dbc10ce00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7da4009860 tx=0x7f7da402a340 comp rx=0 tx=0).stop 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 -- 172.21.15.71:0/2784084474 shutdown_connections 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 --2- 172.21.15.71:0/2784084474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc10d530 0x7f7dbc1150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 --2- 172.21.15.71:0/2784084474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 0x7f7dbc10ce00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 --2- 172.21.15.71:0/2784084474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc108450 0x7f7dbc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 -- 172.21.15.71:0/2784084474 >> 172.21.15.71:0/2784084474 conn(0x7f7dbc077ba0 msgr2=0x7f7dbc079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 -- 172.21.15.71:0/2784084474 shutdown_connections 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.016+0000 7f7dc3000640 1 -- 172.21.15.71:0/2784084474 wait complete. 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 Processor -- start 2024-09-05T12:25:11.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 -- start start 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 0x7f7dbc1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 0x7f7dbc1a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc10d530 0x7f7dbc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc118530 con 0x7f7dbc108450 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc1183b0 con 0x7f7dbc10d530 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc3000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc1186b0 con 0x7f7dbc108e00 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dbbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 0x7f7dbc1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dbbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 0x7f7dbc1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38744/0 (socket says 172.21.15.71:38744) 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dbbe00640 1 -- 172.21.15.71:0/3138177298 learned_addr learned my addr 172.21.15.71:0/3138177298 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:11.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7dc1000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc10d530 0x7f7dbc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.017+0000 7f7db3e00640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 0x7f7dbc1a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dbbe00640 1 -- 172.21.15.71:0/3138177298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 msgr2=0x7f7dbc1a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dbbe00640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 0x7f7dbc1a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dbbe00640 1 -- 172.21.15.71:0/3138177298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc10d530 msgr2=0x7f7dbc19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dbbe00640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc10d530 0x7f7dbc19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dbbe00640 1 -- 172.21.15.71:0/3138177298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc19f050 con 0x7f7dbc108450 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dc1000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc10d530 0x7f7dbc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7dbbe00640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 0x7f7dbc1a4760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e9f0 tx=0x7f7dac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac00cd80 con 0x7f7dbc108450 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7dac010910 con 0x7f7dbc108450 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.018+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac018990 con 0x7f7dbc108450 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.019+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc19f340 con 0x7f7dbc108450 2024-09-05T12:25:11.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.019+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1ab630 con 0x7f7dbc108450 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.020+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc108830 con 0x7f7dbc108450 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.022+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7dac010ac0 con 0x7f7dbc108450 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.023+0000 7f7db9600640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7da00a2b80 0x7f7da00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.024+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f7dac0cf5f0 con 0x7f7dbc108450 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.024+0000 7f7db3e00640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7da00a2b80 0x7f7da00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.024+0000 7f7db3e00640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7da00a2b80 0x7f7da00a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7da402a930 tx=0x7f7da40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:11.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.028+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7dac09bc60 con 0x7f7dbc108450 2024-09-05T12:25:11.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.245+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dbc19fe50 con 0x7f7dbc108450 2024-09-05T12:25:11.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.245+0000 7f7db9600640 1 -- 172.21.15.71:0/3138177298 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11833 (secure 0 0 0) 0x7f7dac09ff60 con 0x7f7dbc108450 2024-09-05T12:25:11.247 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:11.247 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"btime":"2024-09-05T12:25:02:639285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14469,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24433,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24478,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:02.589019+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24524,"qdb_cluster":[24524]},"id":1}]} 2024-09-05T12:25:11.247 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 17 2024-09-05T12:25:11.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7da00a2b80 msgr2=0x7f7da00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7da00a2b80 0x7f7da00a5040 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7da402a930 tx=0x7f7da40058d0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 msgr2=0x7f7dbc1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 0x7f7dbc1a4760 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00e9f0 tx=0x7f7dac00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 shutdown_connections 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7da00a2b80 0x7f7da00a5040 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dbc10d530 0x7f7dbc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc108e00 0x7f7dbc1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 --2- 172.21.15.71:0/3138177298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7dbc108450 0x7f7dbc1a4760 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 >> 172.21.15.71:0/3138177298 conn(0x7f7dbc077ba0 msgr2=0x7f7dbc078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 shutdown_connections 2024-09-05T12:25:11.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:11.248+0000 7f7dc3000640 1 -- 172.21.15.71:0/3138177298 wait complete. 2024-09-05T12:25:12.034 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:25:12.402 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:12.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:12 smithi071 ceph-mon[29845]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:12.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:12 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3138177298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:12 smithi174 ceph-mon[34313]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:12 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3138177298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:12 smithi123 ceph-mon[34321]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:12 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3138177298' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 -- 172.21.15.71:0/2542020815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598112a30 msgr2=0x7fb598114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:13.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 --2- 172.21.15.71:0/2542020815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598112a30 0x7fb598114e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb590009ca0 tx=0x7fb59002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 -- 172.21.15.71:0/2542020815 shutdown_connections 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 --2- 172.21.15.71:0/2542020815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598112a30 0x7fb598114e20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 --2- 172.21.15.71:0/2542020815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb598077040 0x7fb5981124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 --2- 172.21.15.71:0/2542020815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb598076720 0x7fb598076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 -- 172.21.15.71:0/2542020815 >> 172.21.15.71:0/2542020815 conn(0x7fb5980ffa90 msgr2=0x7fb598101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.966+0000 7fb59d600640 1 -- 172.21.15.71:0/2542020815 shutdown_connections 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.967+0000 7fb59d600640 1 -- 172.21.15.71:0/2542020815 wait complete. 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.967+0000 7fb59d600640 1 Processor -- start 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 -- start start 2024-09-05T12:25:13.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb598076720 0x7fb5981a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 0x7fb5981a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb598112a30 0x7fb59819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb5981185c0 con 0x7fb598077040 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb598118440 con 0x7fb598112a30 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb59d600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb598118740 con 0x7fb598076720 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 0x7fb5981a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb58fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 0x7fb5981a4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33812/0 (socket says 172.21.15.71:33812) 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb58fe00640 1 -- 172.21.15.71:0/3602919188 learned_addr learned my addr 172.21.15.71:0/3602919188 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:13.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb596a00640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb598076720 0x7fb5981a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.968+0000 7fb597400640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb598112a30 0x7fb59819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58fe00640 1 -- 172.21.15.71:0/3602919188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb598076720 msgr2=0x7fb5981a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58fe00640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb598076720 0x7fb5981a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58fe00640 1 -- 172.21.15.71:0/3602919188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb598112a30 msgr2=0x7fb59819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58fe00640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb598112a30 0x7fb59819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58fe00640 1 -- 172.21.15.71:0/3602919188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59819efc0 con 0x7fb598077040 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58fe00640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 0x7fb5981a4c10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb58000e9f0 tx=0x7fb58000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58000cd80 con 0x7fb598077040 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb580010910 con 0x7fb598077040 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb580018990 con 0x7fb598077040 2024-09-05T12:25:13.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59819f2b0 con 0x7fb598077040 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.969+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5981ab5a0 con 0x7fb598077040 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.970+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb598076b00 con 0x7fb598077040 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.972+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb580010ac0 con 0x7fb598077040 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.974+0000 7fb58f400640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb57c0a2c00 0x7fb57c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.974+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb5800cf4b0 con 0x7fb598077040 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.974+0000 7fb596a00640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb57c0a2c00 0x7fb57c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.974+0000 7fb596a00640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb57c0a2c00 0x7fb57c0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb59819f9d0 tx=0x7fb588005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:13.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:12.977+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb58009bb20 con 0x7fb598077040 2024-09-05T12:25:13.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.153+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb59810dc20 con 0x7fb598077040 2024-09-05T12:25:13.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.155+0000 7fb58f400640 1 -- 172.21.15.71:0/3602919188 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24214 (secure 0 0 0) 0x7fb58009fe20 con 0x7fb598077040 2024-09-05T12:25:13.156 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:13.156 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":92,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:02.588840+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb57c0a2c00 msgr2=0x7fb57c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb57c0a2c00 0x7fb57c0a50c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb59819f9d0 tx=0x7fb588005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 msgr2=0x7fb5981a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 0x7fb5981a4c10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb58000e9f0 tx=0x7fb58000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 shutdown_connections 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb57c0a2c00 0x7fb57c0a50c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb598112a30 0x7fb59819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb598077040 0x7fb5981a4c10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 --2- 172.21.15.71:0/3602919188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb598076720 0x7fb5981a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:13.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 >> 172.21.15.71:0/3602919188 conn(0x7fb5980ffa90 msgr2=0x7fb598101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:13.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 shutdown_connections 2024-09-05T12:25:13.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:13.157+0000 7fb59d600640 1 -- 172.21.15.71:0/3602919188 wait complete. 2024-09-05T12:25:13.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:13 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3602919188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:13.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:13 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3602919188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:13.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:13 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3602919188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:14.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-05T12:24:49.631865+0000', 'modified': '2024-09-05T12:25:02.589019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/782979893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 782979893}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 782979893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24524, 'qdb_cluster': [24524]} 2024-09-05T12:25:14.043 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-05T12:25:14.044 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T12:25:14.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:14 smithi071 ceph-mon[29845]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:14.412 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:14 smithi174 ceph-mon[34313]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:14 smithi123 ceph-mon[34321]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.986+0000 7f867ce00640 1 -- 172.21.15.71:0/3375569504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678106f10 msgr2=0x7f867810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.986+0000 7f867ce00640 1 --2- 172.21.15.71:0/3375569504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678106f10 0x7f867810ea90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8670009ca0 tx=0x7f867002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 -- 172.21.15.71:0/3375569504 shutdown_connections 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 --2- 172.21.15.71:0/3375569504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678106f10 0x7f867810ea90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 --2- 172.21.15.71:0/3375569504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8678102790 0x7f86781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 --2- 172.21.15.71:0/3375569504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8678101de0 0x7f86781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 -- 172.21.15.71:0/3375569504 >> 172.21.15.71:0/3375569504 conn(0x7f86780fd8c0 msgr2=0x7f86780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 -- 172.21.15.71:0/3375569504 shutdown_connections 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.987+0000 7f867ce00640 1 -- 172.21.15.71:0/3375569504 wait complete. 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 Processor -- start 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 -- start start 2024-09-05T12:25:15.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 0x7f867819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8678102790 0x7f867819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8678106f10 0x7f867819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8678111fc0 con 0x7f8678101de0 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8678111e40 con 0x7f8678106f10 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f867ce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8678112140 con 0x7f8678102790 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.988+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 0x7f867819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 0x7f867819a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33842/0 (socket says 172.21.15.71:33842) 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 -- 172.21.15.71:0/1324580646 learned_addr learned my addr 172.21.15.71:0/1324580646 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675400640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8678102790 0x7f867819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8676800640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8678106f10 0x7f867819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 -- 172.21.15.71:0/1324580646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8678102790 msgr2=0x7f867819aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8678102790 0x7f867819aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 -- 172.21.15.71:0/1324580646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8678106f10 msgr2=0x7f867819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8678106f10 0x7f867819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 -- 172.21.15.71:0/1324580646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867819f610 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.989+0000 7f8675e00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 0x7f867819a660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f866800e9f0 tx=0x7f866800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.990+0000 7f866ea00640 1 -- 172.21.15.71:0/1324580646 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866800cd80 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.990+0000 7f866ea00640 1 -- 172.21.15.71:0/1324580646 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8668010910 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.990+0000 7f866ea00640 1 -- 172.21.15.71:0/1324580646 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668018990 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.990+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867819f900 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.990+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86781a71e0 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.993+0000 7f867fa70740 1 Processor -- start 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.993+0000 7f866ea00640 1 -- 172.21.15.71:0/1324580646 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8668010ac0 con 0x7f8678101de0 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.993+0000 7f867fa70740 1 -- start start 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.993+0000 7f867fa70740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f107d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.993+0000 7f867fa70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f107dc50 0x5649f1087fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.993+0000 7f867fa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f1088510 0x5649f108a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f867fa70740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5649f0fe30b0 con 0x5649f107d330 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f867fa70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649f0f8aab0 con 0x5649f107dc50 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f867fa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5649f0fdf070 con 0x5649f1088510 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8676800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f107d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8676800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f107d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33846/0 (socket says 172.21.15.71:33846) 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8676800640 1 -- 172.21.15.71:0/4262337569 learned_addr learned my addr 172.21.15.71:0/4262337569 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675400640 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f1088510 0x5649f108a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675e00640 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f107dc50 0x5649f1087fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675400640 1 -- 172.21.15.71:0/4262337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f107dc50 msgr2=0x5649f1087fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675400640 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f107dc50 0x5649f1087fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675400640 1 -- 172.21.15.71:0/4262337569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 msgr2=0x5649f107d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675400640 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f107d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8675400640 1 -- 172.21.15.71:0/4262337569 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f0f974a0 con 0x5649f1088510 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.994+0000 7f8676800640 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f107d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:15.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f8675400640 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f1088510 0x5649f108a900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8664009bd0 tx=0x7f8664011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12bfb1f7037b2b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f866ea00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f865c0a2c00 0x7f865c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f8663400640 1 -- 172.21.15.71:0/4262337569 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664004050 con 0x5649f1088510 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f8663400640 1 -- 172.21.15.71:0/4262337569 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664005500 con 0x5649f1088510 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f8663400640 1 -- 172.21.15.71:0/4262337569 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664004d40 con 0x5649f1088510 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f866ea00640 1 -- 172.21.15.71:0/1324580646 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f86680cf550 con 0x7f8678101de0 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.995+0000 7f8676800640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f865c0a2c00 0x7f865c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 -- 172.21.15.71:0/4262337569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f1088510 msgr2=0x5649f108a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f8676800640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f865c0a2c00 0x7f865c0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f867002aeb0 tx=0x7f8670005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f1088510 0x5649f108a900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8664009bd0 tx=0x7f8664011810 comp rx=0 tx=0).stop 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 -- 172.21.15.71:0/4262337569 shutdown_connections 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f1088510 0x5649f108a900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f107dc50 0x5649f1087fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 --2- 172.21.15.71:0/4262337569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f107d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 -- 172.21.15.71:0/4262337569 >> 172.21.15.71:0/4262337569 conn(0x5649f10798c0 msgr2=0x5649f1079ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 -- 172.21.15.71:0/4262337569 shutdown_connections 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 -- 172.21.15.71:0/4262337569 wait complete. 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.996+0000 7f867fa70740 1 -- start start 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 Processor -- start 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1085460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10859a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1085ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5649f108b0c0 con 0x5649f107d330 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649f0f6b920 con 0x5649f1088510 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5649f108b240 con 0x5649f107dc50 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f867fa70740 10 client.0 _ll_register_callbacks cb 0x5649f0f8f8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1085460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f8676800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1085ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f8675400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10859a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1085460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33860/0 (socket says 172.21.15.71:33860) 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.997+0000 7f8675e00640 1 -- 172.21.15.71:0/1771374541 learned_addr learned my addr 172.21.15.71:0/1771374541 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8676800640 1 -- 172.21.15.71:0/1771374541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 msgr2=0x5649f10859a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8676800640 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10859a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8676800640 1 -- 172.21.15.71:0/1771374541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 msgr2=0x5649f1085460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8676800640 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1085460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8676800640 1 -- 172.21.15.71:0/1771374541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f0d542b0 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8676800640 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1085ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8670006010 tx=0x7f867002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91ee5d1b28b5db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8657e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8657e00640 1 -- 172.21.15.71:0/1771374541 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670009880 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8664005500 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.998+0000 7f8657e00640 1 -- 172.21.15.71:0/1771374541 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f8670003fe0 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f8657e00640 1 -- 172.21.15.71:0/1771374541 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f86700319a0 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 10 client.24620 fetch_fsmap learned FSMap version 17 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5649f1096e30 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f8657e00640 1 -- 172.21.15.71:0/1771374541 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664005500 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f8657e00640 1 -- 172.21.15.71:0/1771374541 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f8670051020 con 0x5649f1088510 2024-09-05T12:25:15.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 10 client.24620 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 10 client.24620 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 20 client.24620 populate_metadata read hostname 'smithi071' 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f10994b0 0x5649f109a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 4 client.24620 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:14.999+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649f0ff0620 con 0x5649f10994b0 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.000+0000 7f8675e00640 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f10994b0 0x5649f109a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.000+0000 7f8675e00640 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f10994b0 0x5649f109a920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08d757d6c05f111 server_cookie=4bc81f2ccfce1cb1 in_seq=0 out_seq=0 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.000+0000 7f8657e00640 10 client.24620 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.001+0000 7f8657e00640 1 -- 172.21.15.71:0/1771374541 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5649f0ff0620 con 0x5649f10994b0 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.001+0000 7f8657e00640 10 client.24620 handle_command_reply: tid=0 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.001+0000 7f867fa70740 1 client.24620 shutdown 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 msgr2=0x5649f1085ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1085ee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8670006010 tx=0x7f867002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 shutdown_connections 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f10994b0 0x5649f109a920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1085ee0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10859a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 --2- 172.21.15.71:0/1771374541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1085460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f8657e00640 0 client.24620 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 >> 172.21.15.71:0/1771374541 conn(0x5649f10798c0 msgr2=0x5649f107f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 shutdown_connections 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 1 -- 172.21.15.71:0/1771374541 wait complete. 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.002+0000 7f867fa70740 20 client.24620 trim_cache size 0 max 16384 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 Processor -- start 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 -- start start 2024-09-05T12:25:15.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1080790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f1083d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1084270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5649f1098c60 con 0x5649f107d330 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649f1099330 con 0x5649f1088510 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.020+0000 7f867fa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5649f1099a00 con 0x5649f107dc50 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1080790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1080790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33874/0 (socket says 172.21.15.71:33874) 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 -- 172.21.15.71:0/481746887 learned_addr learned my addr 172.21.15.71:0/481746887 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675400640 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f1083d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8676800640 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1084270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 -- 172.21.15.71:0/481746887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 msgr2=0x5649f1083d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f1083d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 -- 172.21.15.71:0/481746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 msgr2=0x5649f1084270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1084270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 -- 172.21.15.71:0/481746887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649f1087dd0 con 0x5649f107d330 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8675e00640 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1080790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f866809b8c0 tx=0x7f8668000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36457a453da7a38f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8662a00640 1 -- 172.21.15.71:0/481746887 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680a4780 con 0x5649f107d330 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.021+0000 7f8662a00640 1 -- 172.21.15.71:0/481746887 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8668099c60 con 0x5649f107d330 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f8662a00640 1 -- 172.21.15.71:0/481746887 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8668018b30 con 0x5649f107d330 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 -- 172.21.15.71:0/481746887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 msgr2=0x5649f1080790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1080790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f866809b8c0 tx=0x7f8668000bc0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 -- 172.21.15.71:0/481746887 shutdown_connections 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f1084270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f1083d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 --2- 172.21.15.71:0/481746887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f1080790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 -- 172.21.15.71:0/481746887 >> 172.21.15.71:0/481746887 conn(0x5649f10798c0 msgr2=0x5649f1079ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 -- 172.21.15.71:0/481746887 shutdown_connections 2024-09-05T12:25:15.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.022+0000 7f867fa70740 1 -- 172.21.15.71:0/481746887 wait complete. 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 -- start start 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 Processor -- start 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f10a2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10a3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f109c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5649f109a7a0 con 0x5649f107d330 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649f109a0d0 con 0x5649f1088510 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5649f109ae70 con 0x5649f107dc50 2024-09-05T12:25:15.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f867fa70740 10 client.0 _ll_register_callbacks cb 0x7f8670003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.023+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f10a2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8675e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f10a2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33884/0 (socket says 172.21.15.71:33884) 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8675e00640 1 -- 172.21.15.71:0/2126143360 learned_addr learned my addr 172.21.15.71:0/2126143360 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f109c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8675400640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10a3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 -- 172.21.15.71:0/2126143360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 msgr2=0x5649f10a3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10a3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 -- 172.21.15.71:0/2126143360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 msgr2=0x5649f10a2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f10a2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 -- 172.21.15.71:0/2126143360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649f109c9a0 con 0x5649f1088510 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8675400640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8675e00640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f10a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:15.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8676800640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f109c170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8670009ca0 tx=0x7f867002fd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11e42e95efcc8d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8657e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8657e00640 1 -- 172.21.15.71:0/2126143360 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8670031bb0 con 0x5649f1088510 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8668099c60 con 0x5649f1088510 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.024+0000 7f8657e00640 1 -- 172.21.15.71:0/2126143360 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f867002fde0 con 0x5649f1088510 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f8657e00640 1 -- 172.21.15.71:0/2126143360 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8670069050 con 0x5649f1088510 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 10 client.24626 fetch_fsmap learned FSMap version 17 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5649f10a1480 con 0x5649f1088510 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f8657e00640 1 -- 172.21.15.71:0/2126143360 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8668099c60 con 0x5649f1088510 2024-09-05T12:25:15.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f8657e00640 1 -- 172.21.15.71:0/2126143360 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7f8670061020 con 0x5649f1088510 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 10 client.24626 fetch_fsmap finished waiting for FSMap version 17 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 10 client.24626 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 20 client.24626 populate_metadata read hostname 'smithi071' 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f1108340 0x5649f1107050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 4 client.24626 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.025+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "status"}) -- 0x5649f0feab30 con 0x5649f1108340 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f8675e00640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f1108340 0x5649f1107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f8675e00640 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f1108340 0x5649f1107050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf24171e38461b92 server_cookie=e51602387b834af4 in_seq=0 out_seq=0 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f8657e00640 10 client.24626 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f8657e00640 1 -- 172.21.15.71:0/2126143360 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5649f0feab30 con 0x5649f1108340 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f8657e00640 10 client.24626 handle_command_reply: tid=0 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f867fa70740 1 client.24626 shutdown 2024-09-05T12:25:15.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 msgr2=0x5649f109c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.026+0000 7f867fa70740 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f109c170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8670009ca0 tx=0x7f867002fd70 comp rx=0 tx=0).stop 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 shutdown_connections 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5649f1108340 0x5649f1107050 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649f1088510 0x5649f109c170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f8657e00640 0 client.24626 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649f107dc50 0x5649f10a3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 --2- 172.21.15.71:0/2126143360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5649f107d330 0x5649f10a2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 >> 172.21.15.71:0/2126143360 conn(0x5649f10798c0 msgr2=0x5649f1083c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24524, 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 24.518831463000001, 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 17, 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 92, 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 91, 2024-09-05T12:25:15.054 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 38.095058399000003 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 shutdown_connections 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 1 -- 172.21.15.71:0/2126143360 wait complete. 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867fa70740 20 client.24626 trim_cache size 0 max 16384 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f865c0a2c00 msgr2=0x7f865c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.027+0000 7f867ce00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f865c0a2c00 0x7f865c0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f867002aeb0 tx=0x7f8670005d80 comp rx=0 tx=0).stop 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 msgr2=0x7f867819a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:15.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 0x7f867819a660 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f866800e9f0 tx=0x7f866800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 shutdown_connections 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f865c0a2c00 0x7f865c0a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8678106f10 0x7f867819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8678102790 0x7f867819aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 --2- 172.21.15.71:0/1324580646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8678101de0 0x7f867819a660 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 >> 172.21.15.71:0/1324580646 conn(0x7f86780fd8c0 msgr2=0x7f86780fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 shutdown_connections 2024-09-05T12:25:15.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:15.028+0000 7f867ce00640 1 -- 172.21.15.71:0/1324580646 wait complete. 2024-09-05T12:25:16.095 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T12:25:16.095 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:16.095 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T12:25:16.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:16 smithi071 ceph-mon[29845]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:16.462 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:16.482 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:16 smithi123 ceph-mon[34321]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:16 smithi174 ceph-mon[34313]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 -- 172.21.15.71:0/986635944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c0745c0 msgr2=0x7f9f2c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 --2- 172.21.15.71:0/986635944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c0745c0 0x7f9f2c0749a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009900 tx=0x7f9f2002a480 comp rx=0 tx=0).stop 2024-09-05T12:25:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 -- 172.21.15.71:0/986635944 shutdown_connections 2024-09-05T12:25:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 --2- 172.21.15.71:0/986635944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2c10e260 0x7f9f2c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 --2- 172.21.15.71:0/986635944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 --2- 172.21.15.71:0/986635944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c0745c0 0x7f9f2c0749a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 -- 172.21.15.71:0/986635944 >> 172.21.15.71:0/986635944 conn(0x7f9f2c0fd740 msgr2=0x7f9f2c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:17.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 -- 172.21.15.71:0/986635944 shutdown_connections 2024-09-05T12:25:17.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.069+0000 7f9f32000640 1 -- 172.21.15.71:0/986635944 wait complete. 2024-09-05T12:25:17.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.070+0000 7f9f32000640 1 Processor -- start 2024-09-05T12:25:17.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.070+0000 7f9f32000640 1 -- start start 2024-09-05T12:25:17.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.070+0000 7f9f32000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2c0745c0 0x7f9f2c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:17.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f32000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f32000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2c10e260 0x7f9f2c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f32000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c114000 con 0x7f9f2c074ee0 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f32000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c113e80 con 0x7f9f2c0745c0 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f32000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2c114180 con 0x7f9f2c10e260 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33898/0 (socket says 172.21.15.71:33898) 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f30c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2c0745c0 0x7f9f2c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:17.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 -- 172.21.15.71:0/3815748272 learned_addr learned my addr 172.21.15.71:0/3815748272 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f31600640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2c10e260 0x7f9f2c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 -- 172.21.15.71:0/3815748272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2c10e260 msgr2=0x7f9f2c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2c10e260 0x7f9f2c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 -- 172.21.15.71:0/3815748272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2c0745c0 msgr2=0x7f9f2c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2c0745c0 0x7f9f2c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.075 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.071+0000 7f9f2be00640 1 -- 172.21.15.71:0/3815748272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2c19f450 con 0x7f9f2c074ee0 2024-09-05T12:25:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.072+0000 7f9f2be00640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c19a9e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d950 tx=0x7f9f1800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.072+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f18017030 con 0x7f9f2c074ee0 2024-09-05T12:25:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.072+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f18004020 con 0x7f9f2c074ee0 2024-09-05T12:25:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.072+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2c19f740 con 0x7f9f2c074ee0 2024-09-05T12:25:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.072+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f180047a0 con 0x7f9f2c074ee0 2024-09-05T12:25:17.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.072+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f2c1a7020 con 0x7f9f2c074ee0 2024-09-05T12:25:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.073+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2c0749a0 con 0x7f9f2c074ee0 2024-09-05T12:25:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.075+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9f180041d0 con 0x7f9f2c074ee0 2024-09-05T12:25:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.076+0000 7f9f29600640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9f140a2c00 0x7f9f140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.076+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f9f180ceba0 con 0x7f9f2c074ee0 2024-09-05T12:25:17.077 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.076+0000 7f9f30c00640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9f140a2c00 0x7f9f140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:17.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.077+0000 7f9f30c00640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9f140a2c00 0x7f9f140a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f20000c00 tx=0x7f9f20005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:17.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.081+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f1809b860 con 0x7f9f2c074ee0 2024-09-05T12:25:17.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.293+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9f2c108e50 con 0x7f9f2c074ee0 2024-09-05T12:25:17.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.293+0000 7f9f29600640 1 -- 172.21.15.71:0/3815748272 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f9f1809b160 con 0x7f9f2c074ee0 2024-09-05T12:25:17.294 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9f140a2c00 msgr2=0x7f9f140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9f140a2c00 0x7f9f140a50c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9f20000c00 tx=0x7f9f20005b50 comp rx=0 tx=0).stop 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 msgr2=0x7f9f2c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c19a9e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d950 tx=0x7f9f1800de10 comp rx=0 tx=0).stop 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 shutdown_connections 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9f140a2c00 0x7f9f140a50c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f2c10e260 0x7f9f2c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f2c074ee0 0x7f9f2c19a9e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 --2- 172.21.15.71:0/3815748272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f2c0745c0 0x7f9f2c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:17.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 >> 172.21.15.71:0/3815748272 conn(0x7f9f2c0fd740 msgr2=0x7f9f2c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.295+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 shutdown_connections 2024-09-05T12:25:17.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:17.296+0000 7f9f32000640 1 -- 172.21.15.71:0/3815748272 wait complete. 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T12:25:18.333 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T12:25:18.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:18.334 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-05T12:25:18.371 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-05T12:25:18.372 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-05T12:25:18.373 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T12:25:18.373 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:25:18.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:25:18.505 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:18 smithi071 ceph-mon[29845]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:18.506 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:18 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3815748272' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T12:25:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:18 smithi174 ceph-mon[34313]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:18 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3815748272' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T12:25:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:18 smithi123 ceph-mon[34321]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:18 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3815748272' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T12:25:18.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:18.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:18.807 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T12:25:18.864 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-05T12:25:18.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:18.868 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-05T12:25:18.923 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-05T12:25:18.923 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T12:25:18.923 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T12:25:18.923 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T12:25:18.923 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T12:25:18.924 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:25:18.924 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:25:18.924 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T12:25:18.924 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T12:25:18.924 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T12:25:18.924 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:25:19.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:25:19.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:19.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:19.520 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-05T12:25:19.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:19.556 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-09-05T12:25:19.632 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:25:19.633 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:25:19.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:19.633 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-05T12:25:19.633 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:25:19.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:25:19.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:19.810 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-05T12:25:19.810 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:25:19.810 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:25:19.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-05T12:25:19.810 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T12:25:19.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-05T12:25:19.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-05T12:25:19.811 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-05T12:25:19.811 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:25:19.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:25:20.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:20.052 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:25:20.052 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:25:20.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-05T12:25:20.052 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T12:25:20.052 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:25:20.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:25:20.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:20.220 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-05T12:25:20.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T12:25:20.220 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:20.255 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-05T12:25:20.256 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T12:25:20.256 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:20.330 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-09-05T12:25:20.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:20 smithi071 ceph-mon[29845]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:20.428 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T12:25:20.503 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-09-05T12:25:20.504 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-09-05T12:25:20.505 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/c9fd3c5e0124775f793eba7062198b4c08f4d03c0435be9e6ca44e1298f7c0d1/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/424af6d605e32142fa2afc0de651c76fe1707892ea2a6eb405c8a635446f8f03/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/0 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/577a8db6ba1498e215db2f004cd45d29d03b1263eb6babd976b227c5fa7646f9/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/6931ce7faaf6e98826ce999043eb501682ed8bb92c4755ef3533da0eec682cc8/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/27bc40fb17e74c23bae46a8dee2194b1f26b057ccc2d670e949d516cc6eab48c/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/1fe1d54d73d50cab801c281df7ab6c76a058b9cba086127248d08339ef33de1a/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/fa7c2cf7d5ec6b639dd0c6a718acef19d52b4fd02ee306e2fa49ba744bb32efd/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/084cf03b8559153e0a20c88385e789f5c3cafc168c172c97d71e702f275ca5d4/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/daccae1f6995cd36c62ac0d70a66ef4b59f791858d32134a499b9abb94284a16/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/2523d03fec22ebc399c76cd22b669ea46015fd037c982d287ea5ca8d625cb828/merged 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-09-05T12:25:20.506 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:20.507 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:20.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:20.507 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-09-05T12:25:20.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:20 smithi174 ceph-mon[34313]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:20.584 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-05T12:25:20.585 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-05T12:25:20.628 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-09-05T12:25:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:20 smithi123 ceph-mon[34321]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:20.685 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-09-05T12:25:20.722 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T12:25:20.723 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/c9fd3c5e0124775f793eba7062198b4c08f4d03c0435be9e6ca44e1298f7c0d1/merged 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/424af6d605e32142fa2afc0de651c76fe1707892ea2a6eb405c8a635446f8f03/merged 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/0 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/577a8db6ba1498e215db2f004cd45d29d03b1263eb6babd976b227c5fa7646f9/merged 2024-09-05T12:25:20.724 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/6931ce7faaf6e98826ce999043eb501682ed8bb92c4755ef3533da0eec682cc8/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/27bc40fb17e74c23bae46a8dee2194b1f26b057ccc2d670e949d516cc6eab48c/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/1fe1d54d73d50cab801c281df7ab6c76a058b9cba086127248d08339ef33de1a/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/fa7c2cf7d5ec6b639dd0c6a718acef19d52b4fd02ee306e2fa49ba744bb32efd/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/084cf03b8559153e0a20c88385e789f5c3cafc168c172c97d71e702f275ca5d4/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/daccae1f6995cd36c62ac0d70a66ef4b59f791858d32134a499b9abb94284a16/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/2523d03fec22ebc399c76cd22b669ea46015fd037c982d287ea5ca8d625cb828/merged 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:20.725 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:20.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:20.726 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-09-05T12:25:20.823 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:2024-09-05T12:25:20.822+0000 7fd43dc29480 -1 init, newargv = 0x558c42b58400 newargc=15 2024-09-05T12:25:20.823 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[74006]: starting ceph client 2024-09-05T12:25:20.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[74006]: starting fuse 2024-09-05T12:25:21.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo modprobe fuse 2024-09-05T12:25:21.969 DEBUG:teuthology.orchestra.run.smithi071:> cat /proc/self/mounts | awk '{print $2}' 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/proc 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/sys 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/dev 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/security 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/shm 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/pts 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/run 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/cgroup 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/pstore 2024-09-05T12:25:22.007 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/bpf 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/ 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/selinux 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/hugepages 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/dev/mqueue 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/debug 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/tracing 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/fs/fuse/connections 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/sys/kernel/config 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-sysctl.service 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-05T12:25:22.008 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/nfs/rpc_pipefs 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/1000 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/ceph 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/c9fd3c5e0124775f793eba7062198b4c08f4d03c0435be9e6ca44e1298f7c0d1/merged 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/424af6d605e32142fa2afc0de651c76fe1707892ea2a6eb405c8a635446f8f03/merged 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/run/user/0 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/proc/sys/fs/binfmt_misc 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/577a8db6ba1498e215db2f004cd45d29d03b1263eb6babd976b227c5fa7646f9/merged 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/6931ce7faaf6e98826ce999043eb501682ed8bb92c4755ef3533da0eec682cc8/merged 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/27bc40fb17e74c23bae46a8dee2194b1f26b057ccc2d670e949d516cc6eab48c/merged 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/1fe1d54d73d50cab801c281df7ab6c76a058b9cba086127248d08339ef33de1a/merged 2024-09-05T12:25:22.009 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/fa7c2cf7d5ec6b639dd0c6a718acef19d52b4fd02ee306e2fa49ba744bb32efd/merged 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/084cf03b8559153e0a20c88385e789f5c3cafc168c172c97d71e702f275ca5d4/merged 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/daccae1f6995cd36c62ac0d70a66ef4b59f791858d32134a499b9abb94284a16/merged 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/var/lib/containers/storage/overlay/2523d03fec22ebc399c76cd22b669ea46015fd037c982d287ea5ca8d625cb828/merged 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:25:22.010 INFO:teuthology.orchestra.run.smithi071.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:22.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:22.011 DEBUG:teuthology.orchestra.run.smithi071:> ls /sys/fs/fuse/connections 2024-09-05T12:25:22.088 INFO:teuthology.orchestra.run.smithi071.stdout:120 2024-09-05T12:25:22.088 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> sudo stdin-killer -- python3 -c ' 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> import glob 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> import re 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> import os 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> import subprocess 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> def _find_admin_socket(client_name): 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> files = glob.glob(asok_path) 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> # Given a non-glob path, it better be there 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> if "*" not in asok_path: 2024-09-05T12:25:22.089 DEBUG:teuthology.orchestra.run.smithi071:> assert(len(files) == 1) 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> return files[0] 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> for f in files: 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> contents = proc_f.read() 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> if mountpoint in contents: 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> return f 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> print(_find_admin_socket("client.admin")) 2024-09-05T12:25:22.090 DEBUG:teuthology.orchestra.run.smithi071:> ' 2024-09-05T12:25:22.251 INFO:teuthology.orchestra.run.smithi071.stdout:/var/run/ceph/ceph-client.admin.74006.asok 2024-09-05T12:25:22.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:22.266 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74006.asok 2024-09-05T12:25:22.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:22.266 DEBUG:teuthology.orchestra.run.smithi071:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74006.asok status 2024-09-05T12:25:22.448 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:25:22.448 INFO:teuthology.orchestra.run.smithi071.stdout: "metadata": { 2024-09-05T12:25:22.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_sha1": "e856d6b2462509a377ef39149a05475ed006ebac", 2024-09-05T12:25:22.448 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.1.1-261-ge856d6b2 (e856d6b2462509a377ef39149a05475ed006ebac) squid (stable)", 2024-09-05T12:25:22.448 INFO:teuthology.orchestra.run.smithi071.stdout: "entity_id": "admin", 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": "smithi071", 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pid": "74006", 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "root": "/" 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_count": 0, 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "dentry_pinned_count": 0, 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 14589, 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "inst": { 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "name": { 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "client", 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "num": 14589 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:25:22.449 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2674785167 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": { 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "192.168.0.1:0", 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2674785167 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "inst_str": "client.14589 192.168.0.1:0/2674785167", 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "addr_str": "192.168.0.1:0/2674785167", 2024-09-05T12:25:22.450 INFO:teuthology.orchestra.run.smithi071.stdout: "inode_count": 1, 2024-09-05T12:25:22.451 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_epoch": 17, 2024-09-05T12:25:22.451 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch": 92, 2024-09-05T12:25:22.451 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_epoch_barrier": 0, 2024-09-05T12:25:22.451 INFO:teuthology.orchestra.run.smithi071.stdout: "blocklisted": false, 2024-09-05T12:25:22.451 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs" 2024-09-05T12:25:22.451 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:25:22.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:22.460 DEBUG:teuthology.orchestra.run.smithi071:> stat --file-system '--printf=%T 2024-09-05T12:25:22.461 DEBUG:teuthology.orchestra.run.smithi071:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:22.497 INFO:teuthology.orchestra.run.smithi071.stdout:fuseblk 2024-09-05T12:25:22.497 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:22.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:22.497 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:22 smithi174 ceph-mon[34313]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:22.588 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:22 smithi071 ceph-mon[29845]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:22.597 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-05T12:25:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:22 smithi123 ceph-mon[34321]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:22.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:22 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T12:25:22.698 INFO:teuthology.orchestra.run.smithi071.stdout:# file: . 2024-09-05T12:25:22.698 INFO:teuthology.orchestra.run.smithi071.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-05T12:25:22.698 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:22.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:25:22.706 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:25:22.707 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:25:22.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T12:25:23.151 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 -- 172.21.15.71:0/3756759913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 msgr2=0x7f29f81043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 --2- 172.21.15.71:0/3756759913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 0x7f29f81043d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009900 tx=0x7f29e802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:25:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 -- 172.21.15.71:0/3756759913 shutdown_connections 2024-09-05T12:25:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 --2- 172.21.15.71:0/3756759913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f8109120 0x7f29f8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:23.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 --2- 172.21.15.71:0/3756759913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f81049a0 0x7f29f81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 --2- 172.21.15.71:0/3756759913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 0x7f29f81043d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 -- 172.21.15.71:0/3756759913 >> 172.21.15.71:0/3756759913 conn(0x7f29f80ffa90 msgr2=0x7f29f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 -- 172.21.15.71:0/3756759913 shutdown_connections 2024-09-05T12:25:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.843+0000 7f29fd000640 1 -- 172.21.15.71:0/3756759913 wait complete. 2024-09-05T12:25:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.844+0000 7f29fd000640 1 Processor -- start 2024-09-05T12:25:23.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.844+0000 7f29fd000640 1 -- start start 2024-09-05T12:25:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29fd000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 0x7f29f819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29fd000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 0x7f29f819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29fd000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 0x7f29f819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29fd000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29f81141a0 con 0x7f29f8103ff0 2024-09-05T12:25:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29fd000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8114020 con 0x7f29f8109120 2024-09-05T12:25:23.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29fd000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8114320 con 0x7f29f81049a0 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 0x7f29f819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 0x7f29f819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29f5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 0x7f29f819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29f6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 0x7f29f819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33600/0 (socket says 172.21.15.71:33600) 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29f5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 0x7f29f819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42876/0 (socket says 172.21.15.71:42876) 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.845+0000 7f29f6800640 1 -- 172.21.15.71:0/916776712 learned_addr learned my addr 172.21.15.71:0/916776712 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f6800640 1 -- 172.21.15.71:0/916776712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 msgr2=0x7f29f819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f6800640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 0x7f29f819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f6800640 1 -- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 msgr2=0x7f29f819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f6800640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 0x7f29f819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f6800640 1 -- 172.21.15.71:0/916776712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f819f5e0 con 0x7f29f8109120 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f5400640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 0x7f29f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:23.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29f6800640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 0x7f29f819ef00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29f000eb10 tx=0x7f29f000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0004b20 con 0x7f29f8109120 2024-09-05T12:25:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f819f8d0 con 0x7f29f8109120 2024-09-05T12:25:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29f0004cc0 con 0x7f29f8109120 2024-09-05T12:25:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0010760 con 0x7f29f8109120 2024-09-05T12:25:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.846+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f81a71b0 con 0x7f29f8109120 2024-09-05T12:25:23.849 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.847+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f81043d0 con 0x7f29f8109120 2024-09-05T12:25:23.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.851+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f29f0045bd0 con 0x7f29f8109120 2024-09-05T12:25:23.852 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.852+0000 7f29eea00640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29dc0a2b80 0x7f29dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:23.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.852+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f29f00cf940 con 0x7f29f8109120 2024-09-05T12:25:23.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.852+0000 7f29f5e00640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29dc0a2b80 0x7f29dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:23.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.853+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29f009c600 con 0x7f29f8109120 2024-09-05T12:25:23.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:23.853+0000 7f29f5e00640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29dc0a2b80 0x7f29dc0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f29e8000c00 tx=0x7f29e8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:24.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.041+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f29f819b990 con 0x7f29f8109120 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.042+0000 7f29eea00640 1 -- 172.21.15.71:0/916776712 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+592 (secure 0 0 0) 0x7f29f009bf00 con 0x7f29f8109120 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1492294353 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3648851306 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3648851306 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/1559694750 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2578458480 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:24.043 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3836892613 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:24.044 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/1559694750 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:24.044 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4092691810 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:24.044 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3671256491 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:24.044 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4234071631 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:24.044 INFO:teuthology.orchestra.run.smithi071.stderr:listed 10 entries 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.044+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29dc0a2b80 msgr2=0x7f29dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.044+0000 7f29fd000640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29dc0a2b80 0x7f29dc0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f29e8000c00 tx=0x7f29e8005b50 comp rx=0 tx=0).stop 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.044+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 msgr2=0x7f29f819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.044+0000 7f29fd000640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 0x7f29f819ef00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29f000eb10 tx=0x7f29f000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 shutdown_connections 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29dc0a2b80 0x7f29dc0a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29f8109120 0x7f29f819ef00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29f81049a0 0x7f29f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 --2- 172.21.15.71:0/916776712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29f8103ff0 0x7f29f819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 >> 172.21.15.71:0/916776712 conn(0x7f29f80ffa90 msgr2=0x7f29f8100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 shutdown_connections 2024-09-05T12:25:24.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:24.045+0000 7f29fd000640 1 -- 172.21.15.71:0/916776712 wait complete. 2024-09-05T12:25:24.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:24 smithi071 ceph-mon[29845]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:24.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:24 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/916776712' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:25:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:24 smithi174 ceph-mon[34313]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:24 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/916776712' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:25:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:24 smithi123 ceph-mon[34321]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:25:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:24 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/916776712' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:25:25.182 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:25:25.182 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:25:25.222 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T12:25:25.629 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:26.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.198+0000 7f8e5dc00640 1 -- 172.21.15.71:0/1704359649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e58103ff0 msgr2=0x7f8e581043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:26.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.198+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/1704359649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e58103ff0 0x7f8e581043d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8e48009900 tx=0x7f8e4802a410 comp rx=0 tx=0).stop 2024-09-05T12:25:26.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.198+0000 7f8e5dc00640 1 -- 172.21.15.71:0/1704359649 shutdown_connections 2024-09-05T12:25:26.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/1704359649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58109120 0x7f8e58110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/1704359649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e581049a0 0x7f8e581089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/1704359649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e58103ff0 0x7f8e581043d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 -- 172.21.15.71:0/1704359649 >> 172.21.15.71:0/1704359649 conn(0x7f8e580ffa90 msgr2=0x7f8e58101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 -- 172.21.15.71:0/1704359649 shutdown_connections 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 -- 172.21.15.71:0/1704359649 wait complete. 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.199+0000 7f8e5dc00640 1 Processor -- start 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 -- start start 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 0x7f8e5819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e581049a0 0x7f8e5819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e58109120 0x7f8e5819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58114140 con 0x7f8e581049a0 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58113fc0 con 0x7f8e58103ff0 2024-09-05T12:25:26.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e5dc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581142c0 con 0x7f8e58109120 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 0x7f8e5819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e56000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e581049a0 0x7f8e5819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 0x7f8e5819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33606/0 (socket says 172.21.15.71:33606) 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e56a00640 1 -- 172.21.15.71:0/826306417 learned_addr learned my addr 172.21.15.71:0/826306417 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.200+0000 7f8e57400640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e58109120 0x7f8e5819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56000640 1 -- 172.21.15.71:0/826306417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e58109120 msgr2=0x7f8e5819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56000640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e58109120 0x7f8e5819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56000640 1 -- 172.21.15.71:0/826306417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 msgr2=0x7f8e5819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56000640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 0x7f8e5819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56000640 1 -- 172.21.15.71:0/826306417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5819f5f0 con 0x7f8e581049a0 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56a00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 0x7f8e5819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:26.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e56000640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e581049a0 0x7f8e5819ab80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400d720 tx=0x7f8e4400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:26.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44017070 con 0x7f8e581049a0 2024-09-05T12:25:26.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e4400de30 con 0x7f8e581049a0 2024-09-05T12:25:26.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e440047a0 con 0x7f8e581049a0 2024-09-05T12:25:26.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5819f8e0 con 0x7f8e581049a0 2024-09-05T12:25:26.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.201+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e581a71c0 con 0x7f8e581049a0 2024-09-05T12:25:26.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.203+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e581043d0 con 0x7f8e581049a0 2024-09-05T12:25:26.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.205+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8e440099d0 con 0x7f8e581049a0 2024-09-05T12:25:26.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.206+0000 7f8e4f400640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:26.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.207+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f8e440cf300 con 0x7f8e581049a0 2024-09-05T12:25:26.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.207+0000 7f8e56a00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:26.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.207+0000 7f8e56a00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e480096c0 tx=0x7f8e48005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:26.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.209+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e44020020 con 0x7f8e581049a0 2024-09-05T12:25:26.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.396+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8e5819b9a0 con 0x7f8e581049a0 2024-09-05T12:25:26.397 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:26 smithi071 ceph-mon[29845]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.397+0000 7f8e4f400640 1 -- 172.21.15.71:0/826306417 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+592 (secure 0 0 0) 0x7f8e5819b9a0 con 0x7f8e581049a0 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1492294353 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3648851306 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3648851306 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/1559694750 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2578458480 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3836892613 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:26.398 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/1559694750 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:26.399 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4092691810 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:26.399 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3671256491 2024-09-06T12:17:33.045867+0000 2024-09-05T12:25:26.399 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4234071631 2024-09-06T12:17:04.816418+0000 2024-09-05T12:25:26.399 INFO:teuthology.orchestra.run.smithi071.stderr:listed 10 entries 2024-09-05T12:25:26.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8e3c0a2b80 msgr2=0x7f8e3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:26.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8e480096c0 tx=0x7f8e48005a20 comp rx=0 tx=0).stop 2024-09-05T12:25:26.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e581049a0 msgr2=0x7f8e5819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:26.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e581049a0 0x7f8e5819ab80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400d720 tx=0x7f8e4400dbe0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 shutdown_connections 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8e3c0a2b80 0x7f8e3c0a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e58109120 0x7f8e5819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e581049a0 0x7f8e5819ab80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 --2- 172.21.15.71:0/826306417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e58103ff0 0x7f8e5819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 >> 172.21.15.71:0/826306417 conn(0x7f8e580ffa90 msgr2=0x7f8e58100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.399+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 shutdown_connections 2024-09-05T12:25:26.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:26.400+0000 7f8e5dc00640 1 -- 172.21.15.71:0/826306417 wait complete. 2024-09-05T12:25:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:26 smithi174 ceph-mon[34313]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:26 smithi123 ceph-mon[34321]: pgmap v318: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:27.233 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi071.front.sepia.ceph.com... 2024-09-05T12:25:27.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:27.234 DEBUG:teuthology.orchestra.run.smithi071:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-05T12:25:27.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi071.stderr:ceph-fuse[74006]: fuse finished with error 0 and tester_r 0 2024-09-05T12:25:27.292 INFO:teuthology.orchestra.run:waiting for 300 2024-09-05T12:25:27.386 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:25:27.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:25:27.386 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-05T12:25:27.431 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:25:27.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:27 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/826306417' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:25:27.518 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:27 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/826306417' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:25:27.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:27 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/826306417' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:25:27.837 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 -- 172.21.15.71:0/1672843904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54112b90 msgr2=0x7f7a54114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1672843904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54112b90 0x7f7a54114f80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009ca0 tx=0x7f7a4c02a950 comp rx=0 tx=0).stop 2024-09-05T12:25:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 -- 172.21.15.71:0/1672843904 shutdown_connections 2024-09-05T12:25:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1672843904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54112b90 0x7f7a54114f80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1672843904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54074eb0 0x7f7a54112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1672843904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54074590 0x7f7a54074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 -- 172.21.15.71:0/1672843904 >> 172.21.15.71:0/1672843904 conn(0x7f7a540fd8c0 msgr2=0x7f7a540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:28.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 -- 172.21.15.71:0/1672843904 shutdown_connections 2024-09-05T12:25:28.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.449+0000 7f7a5a200640 1 -- 172.21.15.71:0/1672843904 wait complete. 2024-09-05T12:25:28.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.450+0000 7f7a5a200640 1 Processor -- start 2024-09-05T12:25:28.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 -- start start 2024-09-05T12:25:28.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 0x7f7a5419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54074eb0 0x7f7a5419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54112b90 0x7f7a541a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7a54118570 con 0x7f7a54074590 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541183f0 con 0x7f7a54074eb0 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a5a200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a541186f0 con 0x7f7a54112b90 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 0x7f7a5419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a53400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 0x7f7a5419eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40404/0 (socket says 172.21.15.71:40404) 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a53400640 1 -- 172.21.15.71:0/1138779019 learned_addr learned my addr 172.21.15.71:0/1138779019 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a4be00640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54074eb0 0x7f7a5419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.451+0000 7f7a53e00640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54112b90 0x7f7a541a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a53400640 1 -- 172.21.15.71:0/1138779019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54112b90 msgr2=0x7f7a541a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:28.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a53400640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54112b90 0x7f7a541a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a53400640 1 -- 172.21.15.71:0/1138779019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54074eb0 msgr2=0x7f7a5419f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a53400640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54074eb0 0x7f7a5419f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a53400640 1 -- 172.21.15.71:0/1138779019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a541a3af0 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a53400640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 0x7f7a5419eac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400e9f0 tx=0x7f7a4400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400cd80 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a44010910 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a541a3de0 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a44018990 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.452+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a541ab670 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.453+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a54074970 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.455+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7a44010ac0 con 0x7f7a54074590 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.456+0000 7f7a50c00640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a380a2b80 0x7f7a380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:28.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.457+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f7a440cf5f0 con 0x7f7a54074590 2024-09-05T12:25:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.457+0000 7f7a4be00640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a380a2b80 0x7f7a380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.457+0000 7f7a4be00640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a380a2b80 0x7f7a380a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a540fecf0 tx=0x7f7a3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:28.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.460+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a4409bc60 con 0x7f7a54074590 2024-09-05T12:25:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:28 smithi174 ceph-mon[34313]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:28.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.641+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a54105030 con 0x7f7a54074590 2024-09-05T12:25:28.643 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:28 smithi071 ceph-mon[29845]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:28.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.643+0000 7f7a50c00640 1 -- 172.21.15.71:0/1138779019 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24214 (secure 0 0 0) 0x7f7a4409ff60 con 0x7f7a54074590 2024-09-05T12:25:28.644 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:28.644 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":92,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:02.588840+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:25:28.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a380a2b80 msgr2=0x7f7a380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:28.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a380a2b80 0x7f7a380a5040 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7a540fecf0 tx=0x7f7a3c005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:28.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 msgr2=0x7f7a5419eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 0x7f7a5419eac0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400e9f0 tx=0x7f7a4400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 shutdown_connections 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a380a2b80 0x7f7a380a5040 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a54112b90 0x7f7a541a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a54074eb0 0x7f7a5419f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 --2- 172.21.15.71:0/1138779019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a54074590 0x7f7a5419eac0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.645+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 >> 172.21.15.71:0/1138779019 conn(0x7f7a540fd8c0 msgr2=0x7f7a540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.646+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 shutdown_connections 2024-09-05T12:25:28.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:28.646+0000 7f7a5a200640 1 -- 172.21.15.71:0/1138779019 wait complete. 2024-09-05T12:25:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:28 smithi123 ceph-mon[34321]: pgmap v319: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:29.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:29 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1138779019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:29.451 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:25:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:29 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1138779019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:29.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:29 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1138779019' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:30.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 -- 172.21.15.71:0/3301629676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 msgr2=0x7fbdac10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:30.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/3301629676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 0x7fbdac10ea80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009ca0 tx=0x7fbda402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 -- 172.21.15.71:0/3301629676 shutdown_connections 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/3301629676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 0x7fbdac10ea80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/3301629676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac102780 0x7fbdac1067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/3301629676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdac101dd0 0x7fbdac1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.375+0000 7fbdb0c00640 1 -- 172.21.15.71:0/3301629676 >> 172.21.15.71:0/3301629676 conn(0x7fbdac0fd8b0 msgr2=0x7fbdac0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.376+0000 7fbdb0c00640 1 -- 172.21.15.71:0/3301629676 shutdown_connections 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.376+0000 7fbdb0c00640 1 -- 172.21.15.71:0/3301629676 wait complete. 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 Processor -- start 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 -- start start 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 0x7fbdac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdac102780 0x7fbdac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 0x7fbdac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac111fb0 con 0x7fbdac106f00 2024-09-05T12:25:30.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac111e30 con 0x7fbdac101dd0 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdb0c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac112130 con 0x7fbdac102780 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbda9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 0x7fbdac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbda9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 0x7fbdac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33642/0 (socket says 172.21.15.71:33642) 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbda9e00640 1 -- 172.21.15.71:0/1746818848 learned_addr learned my addr 172.21.15.71:0/1746818848 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.377+0000 7fbdaa800640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 0x7fbdac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9400640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdac102780 0x7fbdac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9e00640 1 -- 172.21.15.71:0/1746818848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdac102780 msgr2=0x7fbdac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9e00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdac102780 0x7fbdac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9e00640 1 -- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 msgr2=0x7fbdac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9e00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 0x7fbdac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9e00640 1 -- 172.21.15.71:0/1746818848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac1a68a0 con 0x7fbdac101dd0 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda9e00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 0x7fbdac0744f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00e9f0 tx=0x7fbd9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:30.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.378+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c00cd80 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.379+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdac1a6b90 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.379+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd9c010910 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.379+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0188f0 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.379+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdac1a7120 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.380+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdac1021b0 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.382+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbd9c010ac0 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.383+0000 7fbda2a00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbd900a2c00 0x7fbd900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.383+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fbd9c0cfb00 con 0x7fbdac101dd0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.383+0000 7fbda9400640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbd900a2c00 0x7fbd900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.384+0000 7fbda9400640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbd900a2c00 0x7fbd900a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdac1087c0 tx=0x7fbd9800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:30.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.385+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd9c09c170 con 0x7fbdac101dd0 2024-09-05T12:25:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:30 smithi174 ceph-mon[34313]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:30.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.560+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdac075a20 con 0x7fbdac101dd0 2024-09-05T12:25:30.561 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:30 smithi071 ceph-mon[29845]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:30.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.562+0000 7fbda2a00640 1 -- 172.21.15.71:0/1746818848 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24214 (secure 0 0 0) 0x7fbd9c0a0470 con 0x7fbdac101dd0 2024-09-05T12:25:30.563 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:30.563 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":92,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:02.588840+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:25:30.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbd900a2c00 msgr2=0x7fbd900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:30.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbd900a2c00 0x7fbd900a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbdac1087c0 tx=0x7fbd9800b410 comp rx=0 tx=0).stop 2024-09-05T12:25:30.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 msgr2=0x7fbdac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:30.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 0x7fbdac0744f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c00e9f0 tx=0x7fbd9c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 shutdown_connections 2024-09-05T12:25:30.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbd900a2c00 0x7fbd900a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdac106f00 0x7fbdac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdac102780 0x7fbdac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 --2- 172.21.15.71:0/1746818848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbdac101dd0 0x7fbdac0744f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:30.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.564+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 >> 172.21.15.71:0/1746818848 conn(0x7fbdac0fd8b0 msgr2=0x7fbdac0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:30.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.565+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 shutdown_connections 2024-09-05T12:25:30.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:30.565+0000 7fbdb0c00640 1 -- 172.21.15.71:0/1746818848 wait complete. 2024-09-05T12:25:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:30 smithi123 ceph-mon[34321]: pgmap v320: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-05T12:25:31.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-05T12:25:31.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:31 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1746818848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:31.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:31 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1746818848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:31.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:31 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1746818848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:31.768 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:32.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 -- 172.21.15.71:0/775520566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118102360 msgr2=0x7f711810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:32.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 --2- 172.21.15.71:0/775520566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118102360 0x7f711810e890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7110009cd0 tx=0x7f711002a8e0 comp rx=0 tx=0).stop 2024-09-05T12:25:32.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 -- 172.21.15.71:0/775520566 shutdown_connections 2024-09-05T12:25:32.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 --2- 172.21.15.71:0/775520566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118102360 0x7f711810e890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:32.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 --2- 172.21.15.71:0/775520566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71181019c0 0x7f7118101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:32.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 --2- 172.21.15.71:0/775520566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71181079c0 0x7f7118107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:32.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 -- 172.21.15.71:0/775520566 >> 172.21.15.71:0/775520566 conn(0x7f71180fd740 msgr2=0x7f71180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:32.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 -- 172.21.15.71:0/775520566 shutdown_connections 2024-09-05T12:25:32.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.356+0000 7f711da00640 1 -- 172.21.15.71:0/775520566 wait complete. 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.357+0000 7f711da00640 1 Processor -- start 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.357+0000 7f711da00640 1 -- start start 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711da00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111e10 con 0x7f71181019c0 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111c90 con 0x7f71181079c0 2024-09-05T12:25:32.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711da00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7118111f90 con 0x7f7118102360 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40436/0 (socket says 172.21.15.71:40436) 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 -- 172.21.15.71:0/780133401 learned_addr learned my addr 172.21.15.71:0/780133401 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f710fe00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f711d000640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 -- 172.21.15.71:0/780133401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118102360 msgr2=0x7f7118074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 -- 172.21.15.71:0/780133401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71181079c0 msgr2=0x7f7118074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f7117e00640 1 -- 172.21.15.71:0/780133401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181a66f0 con 0x7f71181019c0 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.358+0000 7f710fe00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.359+0000 7f7117e00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 0x7f71180744f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9f0 tx=0x7f710800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:32.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.359+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710800cd80 con 0x7f71181019c0 2024-09-05T12:25:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.359+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7108005500 con 0x7f71181019c0 2024-09-05T12:25:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.359+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a69e0 con 0x7f71181019c0 2024-09-05T12:25:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.359+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108010690 con 0x7f71181019c0 2024-09-05T12:25:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.359+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a6ed0 con 0x7f71181019c0 2024-09-05T12:25:32.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.360+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7118101e20 con 0x7f71181019c0 2024-09-05T12:25:32.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.362+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f71080046c0 con 0x7f71181019c0 2024-09-05T12:25:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.363+0000 7f7115600640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70fc0a2b80 0x7f70fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.363+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f71080cf680 con 0x7f71181019c0 2024-09-05T12:25:32.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.364+0000 7f710fe00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70fc0a2b80 0x7f70fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:32.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.364+0000 7f710fe00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70fc0a2b80 0x7f70fc0a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7118075f80 tx=0x7f7100005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:32.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.368+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f710809c340 con 0x7f71181019c0 2024-09-05T12:25:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:32 smithi174 ceph-mon[34313]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T12:25:32.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:32.576+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f71181a7160 con 0x7f71181019c0 2024-09-05T12:25:32.577 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:32 smithi071 ceph-mon[29845]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T12:25:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:32 smithi123 ceph-mon[34321]: pgmap v321: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T12:25:33.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.258+0000 7f7115600640 1 -- 172.21.15.71:0/780133401 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f710809bc40 con 0x7f71181019c0 2024-09-05T12:25:33.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70fc0a2b80 msgr2=0x7f70fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:33.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70fc0a2b80 0x7f70fc0a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7118075f80 tx=0x7f7100005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:33.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 msgr2=0x7f71180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:33.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 0x7f71180744f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f710800e9f0 tx=0x7f710800edf0 comp rx=0 tx=0).stop 2024-09-05T12:25:33.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 shutdown_connections 2024-09-05T12:25:33.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70fc0a2b80 0x7f70fc0a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:33.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71181079c0 0x7f7118074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:33.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118102360 0x7f7118074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:33.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 --2- 172.21.15.71:0/780133401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71181019c0 0x7f71180744f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:33.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.261+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 >> 172.21.15.71:0/780133401 conn(0x7f71180fd740 msgr2=0x7f71180fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:33.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.262+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 shutdown_connections 2024-09-05T12:25:33.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:33.262+0000 7f711da00640 1 -- 172.21.15.71:0/780133401 wait complete. 2024-09-05T12:25:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/780133401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T12:25:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-05T12:25:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-05T12:25:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-05T12:25:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-05T12:25:33.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-05T12:25:33.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-05T12:25:33.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-09-05T12:25:33.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-05T12:25:33.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-05T12:25:33.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:33 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/780133401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-05T12:25:33.649 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:33 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/780133401' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]: dispatch 2024-09-05T12:25:33.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:33 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-05T12:25:34.056 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-09-05T12:25:34.056 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-05T12:25:34.434 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:34.572 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/780133401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-05T12:25:34.573 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T12:25:34.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:34 smithi071 ceph-mon[29845]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/780133401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-09-05T12:25:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-05T12:25:34.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-05T12:25:34.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-05T12:25:34.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T12:25:34.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:34 smithi123 ceph-mon[34321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/780133401' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 8]}]': finished 2024-09-05T12:25:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [5, 8]}]': finished 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [9, 8]}]': finished 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: osdmap e93: 12 total, 12 up, 12 in 2024-09-05T12:25:34.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:34 smithi174 ceph-mon[34313]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:35.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 -- 172.21.15.71:0/2780101787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c104010 msgr2=0x7f490c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 --2- 172.21.15.71:0/2780101787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c104010 0x7f490c1043f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4900009ca0 tx=0x7f490002a8b0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 -- 172.21.15.71:0/2780101787 shutdown_connections 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 --2- 172.21.15.71:0/2780101787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c109140 0x7f490c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 --2- 172.21.15.71:0/2780101787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c1049c0 0x7f490c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 --2- 172.21.15.71:0/2780101787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c104010 0x7f490c1043f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 -- 172.21.15.71:0/2780101787 >> 172.21.15.71:0/2780101787 conn(0x7f490c0ffa50 msgr2=0x7f490c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 -- 172.21.15.71:0/2780101787 shutdown_connections 2024-09-05T12:25:35.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.025+0000 7f4913e00640 1 -- 172.21.15.71:0/2780101787 wait complete. 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.026+0000 7f4913e00640 1 Processor -- start 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.026+0000 7f4913e00640 1 -- start start 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4913e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c104010 0x7f490c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4913e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 0x7f490c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4913e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c109140 0x7f490c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4913e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f490c114140 con 0x7f490c1049c0 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4913e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f490c113fc0 con 0x7f490c104010 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4913e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f490c1142c0 con 0x7f490c109140 2024-09-05T12:25:35.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4910a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 0x7f490c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4911400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c104010 0x7f490c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4910a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 0x7f490c19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55286/0 (socket says 172.21.15.71:55286) 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4910a00640 1 -- 172.21.15.71:0/4225154121 learned_addr learned my addr 172.21.15.71:0/4225154121 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4911400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c104010 0x7f490c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:56936/0 (socket says 172.21.15.71:56936) 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.027+0000 7f4911e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c109140 0x7f490c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4910a00640 1 -- 172.21.15.71:0/4225154121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c109140 msgr2=0x7f490c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4910a00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c109140 0x7f490c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4910a00640 1 -- 172.21.15.71:0/4225154121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c104010 msgr2=0x7f490c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4910a00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c104010 0x7f490c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4910a00640 1 -- 172.21.15.71:0/4225154121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c19f690 con 0x7f490c1049c0 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4910a00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 0x7f490c19ac20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e9f0 tx=0x7f48f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f800cd80 con 0x7f490c1049c0 2024-09-05T12:25:35.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48f8010910 con 0x7f490c1049c0 2024-09-05T12:25:35.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f490c19f980 con 0x7f490c1049c0 2024-09-05T12:25:35.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48f8018990 con 0x7f490c1049c0 2024-09-05T12:25:35.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.028+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f490c1a7260 con 0x7f490c1049c0 2024-09-05T12:25:35.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.029+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f490c1043f0 con 0x7f490c1049c0 2024-09-05T12:25:35.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.032+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f48f8010ac0 con 0x7f490c1049c0 2024-09-05T12:25:35.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.033+0000 7f490a000640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f48f40a2c00 0x7f48f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:35.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.033+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f48f80cf670 con 0x7f490c1049c0 2024-09-05T12:25:35.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.033+0000 7f4911400640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f48f40a2c00 0x7f48f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:35.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.034+0000 7f4911400640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f48f40a2c00 0x7f48f40a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4900004af0 tx=0x7f4900004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:35.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.038+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48f809bbc0 con 0x7f490c1049c0 2024-09-05T12:25:35.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.249+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f490c10bd90 con 0x7f490c1049c0 2024-09-05T12:25:35.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.275+0000 7f490a000640 1 -- 172.21.15.71:0/4225154121 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f48f809fec0 con 0x7f490c1049c0 2024-09-05T12:25:35.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f48f40a2c00 msgr2=0x7f48f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f48f40a2c00 0x7f48f40a50c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4900004af0 tx=0x7f4900004a40 comp rx=0 tx=0).stop 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 msgr2=0x7f490c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 0x7f490c19ac20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f48f800e9f0 tx=0x7f48f800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 shutdown_connections 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f48f40a2c00 0x7f48f40a50c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.278+0000 7f4913e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f490c109140 0x7f490c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.279+0000 7f4913e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f490c1049c0 0x7f490c19ac20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.279+0000 7f4913e00640 1 --2- 172.21.15.71:0/4225154121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f490c104010 0x7f490c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.279+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 >> 172.21.15.71:0/4225154121 conn(0x7f490c0ffa50 msgr2=0x7f490c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.279+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 shutdown_connections 2024-09-05T12:25:35.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:35.279+0000 7f4913e00640 1 -- 172.21.15.71:0/4225154121 wait complete. 2024-09-05T12:25:35.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:35 smithi071 ceph-mon[29845]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T12:25:35.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:35 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4225154121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T12:25:35.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:35 smithi123 ceph-mon[34321]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T12:25:35.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:35 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4225154121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T12:25:35.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:35 smithi174 ceph-mon[34313]: osdmap e94: 12 total, 12 up, 12 in 2024-09-05T12:25:35.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:35 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4225154121' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-05T12:25:36.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: pgmap v325: 129 pgs: 9 peering, 120 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4225154121' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-05T12:25:36.541 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:36 smithi071 ceph-mon[29845]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-05T12:25:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: pgmap v325: 129 pgs: 9 peering, 120 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-05T12:25:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T12:25:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4225154121' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T12:25:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T12:25:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-05T12:25:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:36 smithi123 ceph-mon[34321]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-05T12:25:36.680 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: pgmap v325: 129 pgs: 9 peering, 120 active+clean; 581 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 127 B/s wr, 0 op/s 2024-09-05T12:25:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-05T12:25:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4225154121' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-05T12:25:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-05T12:25:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-05T12:25:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-09-05T12:25:36.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:36 smithi174 ceph-mon[34313]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 -- 172.21.15.71:0/261365519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19881047e0 msgr2=0x7f1988108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 --2- 172.21.15.71:0/261365519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19881047e0 0x7f1988108830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1970009900 tx=0x7f197002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 -- 172.21.15.71:0/261365519 shutdown_connections 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 --2- 172.21.15.71:0/261365519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988108f60 0x7f1988110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 --2- 172.21.15.71:0/261365519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19881047e0 0x7f1988108830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 --2- 172.21.15.71:0/261365519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1988103e30 0x7f1988104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 -- 172.21.15.71:0/261365519 >> 172.21.15.71:0/261365519 conn(0x7f19880ff8d0 msgr2=0x7f1988101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.257+0000 7f198ca00640 1 -- 172.21.15.71:0/261365519 shutdown_connections 2024-09-05T12:25:37.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.258+0000 7f198ca00640 1 -- 172.21.15.71:0/261365519 wait complete. 2024-09-05T12:25:37.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.258+0000 7f198ca00640 1 Processor -- start 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 -- start start 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 0x7f198819a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19881047e0 0x7f198819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988108f60 0x7f198819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1988114010 con 0x7f1988103e30 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1988113e90 con 0x7f1988108f60 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f198ca00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1988114190 con 0x7f19881047e0 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f1987400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 0x7f198819a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f1987400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 0x7f198819a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55300/0 (socket says 172.21.15.71:55300) 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f1987400640 1 -- 172.21.15.71:0/798429894 learned_addr learned my addr 172.21.15.71:0/798429894 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:37.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f197ea00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19881047e0 0x7f198819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.259+0000 7f1987e00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988108f60 0x7f198819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1987400640 1 -- 172.21.15.71:0/798429894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19881047e0 msgr2=0x7f198819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1987400640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19881047e0 0x7f198819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1987400640 1 -- 172.21.15.71:0/798429894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988108f60 msgr2=0x7f198819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1987400640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988108f60 0x7f198819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1987400640 1 -- 172.21.15.71:0/798429894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198819f420 con 0x7f1988103e30 2024-09-05T12:25:37.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1987400640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 0x7f198819a470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9f0 tx=0x7f197800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800cd80 con 0x7f1988103e30 2024-09-05T12:25:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1978005500 con 0x7f1988103e30 2024-09-05T12:25:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198819f710 con 0x7f1988103e30 2024-09-05T12:25:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.260+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978010690 con 0x7f1988103e30 2024-09-05T12:25:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.261+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a6ff0 con 0x7f1988103e30 2024-09-05T12:25:37.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.262+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1988105310 con 0x7f1988103e30 2024-09-05T12:25:37.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.263+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f19780046c0 con 0x7f1988103e30 2024-09-05T12:25:37.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.264+0000 7f1984c00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f196c0a2b80 0x7f196c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:37.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.265+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f19780cf7e0 con 0x7f1988103e30 2024-09-05T12:25:37.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.265+0000 7f197ea00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f196c0a2b80 0x7f196c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:37.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.265+0000 7f197ea00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f196c0a2b80 0x7f196c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1970000c00 tx=0x7f1970005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:37.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.269+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f197809c380 con 0x7f1988103e30 2024-09-05T12:25:37.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:37 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] up:active 2024-09-05T12:25:37.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:37 smithi071 ceph-mon[29845]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T12:25:37.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:37 smithi071 ceph-mon[29845]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-05T12:25:37.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:37 smithi071 ceph-mon[29845]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-09-05T12:25:37.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:37 smithi071 ceph-mon[29845]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-05T12:25:37.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:37.485+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f198810c1a0 con 0x7f1988103e30 2024-09-05T12:25:37.679 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:25:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: 2024-09-05T12:25:37.303+0000 7ff641ab1640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-05T12:25:37.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:37 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] up:active 2024-09-05T12:25:37.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:37 smithi123 ceph-mon[34321]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T12:25:37.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:37 smithi123 ceph-mon[34321]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-05T12:25:37.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:37 smithi123 ceph-mon[34321]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-09-05T12:25:37.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:37 smithi123 ceph-mon[34321]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-05T12:25:37.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:37 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] up:active 2024-09-05T12:25:37.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:37 smithi174 ceph-mon[34313]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-05T12:25:37.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:37 smithi174 ceph-mon[34313]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-09-05T12:25:37.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:37 smithi174 ceph-mon[34313]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:creating} 9 up:standby 2024-09-05T12:25:37.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:37 smithi174 ceph-mon[34313]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-09-05T12:25:38.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.304+0000 7f1984c00640 1 -- 172.21.15.71:0/798429894 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f197809bc80 con 0x7f1988103e30 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f196c0a2b80 msgr2=0x7f196c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f196c0a2b80 0x7f196c0a5040 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1970000c00 tx=0x7f1970005b50 comp rx=0 tx=0).stop 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 msgr2=0x7f198819a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 0x7f198819a470 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9f0 tx=0x7f197800edf0 comp rx=0 tx=0).stop 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 shutdown_connections 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f196c0a2b80 0x7f196c0a5040 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1988108f60 0x7f198819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19881047e0 0x7f198819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 --2- 172.21.15.71:0/798429894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1988103e30 0x7f198819a470 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 >> 172.21.15.71:0/798429894 conn(0x7f19880ff8d0 msgr2=0x7f1988100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 shutdown_connections 2024-09-05T12:25:38.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:38.307+0000 7f198ca00640 1 -- 172.21.15.71:0/798429894 wait complete. 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: pgmap v326: 129 pgs: 9 peering, 120 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] up:active 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-05T12:25:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:38 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/798429894' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T12:25:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: pgmap v326: 129 pgs: 9 peering, 120 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-05T12:25:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] up:active 2024-09-05T12:25:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T12:25:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-09-05T12:25:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-05T12:25:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-05T12:25:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:38 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/798429894' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: pgmap v326: 129 pgs: 9 peering, 120 active+clean; 582 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] up:active 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=i=up:active} 9 up:standby 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-05T12:25:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:38 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/798429894' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-05T12:25:39.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/798429894' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] up:active 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: Cluster is now healthy 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-05T12:25:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:39 smithi071 ceph-mon[29845]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-05T12:25:39.508 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:39.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/798429894' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] up:active 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: Cluster is now healthy 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-05T12:25:39.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:39 smithi123 ceph-mon[34321]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/798429894' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] up:active 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: Cluster is now healthy 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: fsmap cephfs:4 {0=b=up:active,1=g=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-09-05T12:25:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-05T12:25:39.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:39 smithi174 ceph-mon[34313]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-05T12:25:40.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 -- 172.21.15.71:0/3133781375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e10102b90 msgr2=0x7f6e10102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:40.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 --2- 172.21.15.71:0/3133781375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e10102b90 0x7f6e10102f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009900 tx=0x7f6e0002a480 comp rx=0 tx=0).stop 2024-09-05T12:25:40.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 -- 172.21.15.71:0/3133781375 shutdown_connections 2024-09-05T12:25:40.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 --2- 172.21.15.71:0/3133781375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e1010bfc0 0x7f6e1010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 --2- 172.21.15.71:0/3133781375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e101034b0 0x7f6e1010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 --2- 172.21.15.71:0/3133781375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e10102b90 0x7f6e10102f70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 -- 172.21.15.71:0/3133781375 >> 172.21.15.71:0/3133781375 conn(0x7f6e100fc610 msgr2=0x7f6e100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 -- 172.21.15.71:0/3133781375 shutdown_connections 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.082+0000 7f6e15800640 1 -- 172.21.15.71:0/3133781375 wait complete. 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 Processor -- start 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 -- start start 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 0x7f6e1019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e101034b0 0x7f6e1019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1010bfc0 0x7f6e1019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111dc0 con 0x7f6e1010bfc0 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111c40 con 0x7f6e10102b90 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.083+0000 7f6e15800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e10111f40 con 0x7f6e101034b0 2024-09-05T12:25:40.106 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e14e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1010bfc0 0x7f6e1019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 0x7f6e1019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 0x7f6e1019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:56974/0 (socket says 172.21.15.71:56974) 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 -- 172.21.15.71:0/2650017810 learned_addr learned my addr 172.21.15.71:0/2650017810 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e07e00640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e101034b0 0x7f6e1019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 -- 172.21.15.71:0/2650017810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e101034b0 msgr2=0x7f6e1019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e101034b0 0x7f6e1019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 -- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1010bfc0 msgr2=0x7f6e1019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1010bfc0 0x7f6e1019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 -- 172.21.15.71:0/2650017810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1019f470 con 0x7f6e10102b90 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.084+0000 7f6e0fe00640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 0x7f6e1019a4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e00006010 tx=0x7f6e000042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.085+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0003f030 con 0x7f6e10102b90 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.085+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e1019f700 con 0x7f6e10102b90 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.085+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e00003ce0 con 0x7f6e10102b90 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.085+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00035070 con 0x7f6e10102b90 2024-09-05T12:25:40.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.085+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e101a7040 con 0x7f6e10102b90 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.086+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e10102f70 con 0x7f6e10102b90 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.088+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6e00042030 con 0x7f6e10102b90 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.089+0000 7f6e0d600640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6df40a2c00 0x7f6df40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.089+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f6e000efac0 con 0x7f6e10102b90 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.090+0000 7f6e07e00640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6df40a2c00 0x7f6df40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.090+0000 7f6e07e00640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6df40a2c00 0x7f6df40a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e10102810 tx=0x7f6df8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:40.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.092+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e000bc0d0 con 0x7f6e10102b90 2024-09-05T12:25:40.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.304+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f6e101a7210 con 0x7f6e10102b90 2024-09-05T12:25:40.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.326+0000 7f6e0d600640 1 -- 172.21.15.71:0/2650017810 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f6e000c03d0 con 0x7f6e10102b90 2024-09-05T12:25:40.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6df40a2c00 msgr2=0x7f6df40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6df40a2c00 0x7f6df40a50c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e10102810 tx=0x7f6df8005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 msgr2=0x7f6e1019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 0x7f6e1019a4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6e00006010 tx=0x7f6e000042f0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 shutdown_connections 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6df40a2c00 0x7f6df40a50c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.328+0000 7f6e15800640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6e1010bfc0 0x7f6e1019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.329+0000 7f6e15800640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e101034b0 0x7f6e1019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.329+0000 7f6e15800640 1 --2- 172.21.15.71:0/2650017810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e10102b90 0x7f6e1019a4c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.329+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 >> 172.21.15.71:0/2650017810 conn(0x7f6e100fc610 msgr2=0x7f6e100fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.329+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 shutdown_connections 2024-09-05T12:25:40.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:40.329+0000 7f6e15800640 1 -- 172.21.15.71:0/2650017810 wait complete. 2024-09-05T12:25:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:40 smithi071 ceph-mon[29845]: pgmap v327: 129 pgs: 9 peering, 120 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 5 op/s 2024-09-05T12:25:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:40 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] up:active 2024-09-05T12:25:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:40 smithi071 ceph-mon[29845]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-05T12:25:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:40 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:25:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2650017810' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T12:25:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:40 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T12:25:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:40 smithi123 ceph-mon[34321]: pgmap v327: 129 pgs: 9 peering, 120 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 5 op/s 2024-09-05T12:25:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:40 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] up:active 2024-09-05T12:25:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:40 smithi123 ceph-mon[34321]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-05T12:25:40.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:40 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:25:40.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2650017810' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T12:25:40.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:40 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T12:25:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:40 smithi174 ceph-mon[34313]: pgmap v327: 129 pgs: 9 peering, 120 active+clean; 584 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s wr, 5 op/s 2024-09-05T12:25:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:40 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] up:active 2024-09-05T12:25:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:40 smithi174 ceph-mon[34313]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-05T12:25:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:40 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:25:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2650017810' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T12:25:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:40 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-05T12:25:41.275 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-09-05T12:25:41.276 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-05T12:25:41.643 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:41.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:41 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T12:25:41.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:41 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:25:41.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:41 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T12:25:41.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:41 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:25:41.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:41 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-05T12:25:41.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:41 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:25:42.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 -- 172.21.15.71:0/1562498107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c103e30 msgr2=0x7f7c6c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:42.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 --2- 172.21.15.71:0/1562498107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c103e30 0x7f7c6c104210 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c009900 tx=0x7f7c5c02a400 comp rx=0 tx=0).stop 2024-09-05T12:25:42.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 -- 172.21.15.71:0/1562498107 shutdown_connections 2024-09-05T12:25:42.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 --2- 172.21.15.71:0/1562498107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c108f60 0x7f7c6c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 --2- 172.21.15.71:0/1562498107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c1047e0 0x7f7c6c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 --2- 172.21.15.71:0/1562498107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c103e30 0x7f7c6c104210 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 -- 172.21.15.71:0/1562498107 >> 172.21.15.71:0/1562498107 conn(0x7f7c6c0ff8d0 msgr2=0x7f7c6c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 -- 172.21.15.71:0/1562498107 shutdown_connections 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.255+0000 7f7c72200640 1 -- 172.21.15.71:0/1562498107 wait complete. 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 Processor -- start 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 -- start start 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c103e30 0x7f7c6c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c1047e0 0x7f7c6c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c108f60 0x7f7c6c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c114010 con 0x7f7c6c108f60 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c113e90 con 0x7f7c6c1047e0 2024-09-05T12:25:42.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c72200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6c114190 con 0x7f7c6c103e30 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.256+0000 7f7c71800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c108f60 0x7f7c6c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c70e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c103e30 0x7f7c6c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c70e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c103e30 0x7f7c6c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48058/0 (socket says 172.21.15.71:48058) 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c70e00640 1 -- 172.21.15.71:0/2651679896 learned_addr learned my addr 172.21.15.71:0/2651679896 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c63e00640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c1047e0 0x7f7c6c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c71800640 1 -- 172.21.15.71:0/2651679896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c103e30 msgr2=0x7f7c6c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c71800640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c103e30 0x7f7c6c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c71800640 1 -- 172.21.15.71:0/2651679896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c1047e0 msgr2=0x7f7c6c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c71800640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c1047e0 0x7f7c6c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c71800640 1 -- 172.21.15.71:0/2651679896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6c19f400 con 0x7f7c6c108f60 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c71800640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c108f60 0x7f7c6c19ed20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e940 tx=0x7f7c6400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.257+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c64004d20 con 0x7f7c6c108f60 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.258+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c64005020 con 0x7f7c6c108f60 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.258+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6c19f6f0 con 0x7f7c6c108f60 2024-09-05T12:25:42.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.258+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c640116b0 con 0x7f7c6c108f60 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.258+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6c1a6fd0 con 0x7f7c6c108f60 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.259+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c6c104210 con 0x7f7c6c108f60 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.260+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7c640054d0 con 0x7f7c6c108f60 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.262+0000 7f7c6a000640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7c500a2b80 0x7f7c500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.262+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7c640cf740 con 0x7f7c6c108f60 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.262+0000 7f7c70e00640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7c500a2b80 0x7f7c500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:42.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.263+0000 7f7c70e00640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7c500a2b80 0x7f7c500a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0096c0 tx=0x7f7c5c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:42.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.266+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c64016030 con 0x7f7c6c108f60 2024-09-05T12:25:42.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:42 smithi071 ceph-mon[29845]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s wr, 18 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:42.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.471+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7c6c062630 con 0x7f7c500a2b80 2024-09-05T12:25:42.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.502+0000 7f7c6a000640 1 -- 172.21.15.71:0/2651679896 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c6c062630 con 0x7f7c500a2b80 2024-09-05T12:25:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7c500a2b80 msgr2=0x7f7c500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7c500a2b80 0x7f7c500a5040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0096c0 tx=0x7f7c5c0046c0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c108f60 msgr2=0x7f7c6c19ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c108f60 0x7f7c6c19ed20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7c6400e940 tx=0x7f7c6400ee00 comp rx=0 tx=0).stop 2024-09-05T12:25:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 shutdown_connections 2024-09-05T12:25:42.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7c500a2b80 0x7f7c500a5040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c6c108f60 0x7f7c6c19ed20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c6c1047e0 0x7f7c6c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 --2- 172.21.15.71:0/2651679896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c6c103e30 0x7f7c6c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:42.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 >> 172.21.15.71:0/2651679896 conn(0x7f7c6c0ff8d0 msgr2=0x7f7c6c100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:42.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 shutdown_connections 2024-09-05T12:25:42.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:42.504+0000 7f7c72200640 1 -- 172.21.15.71:0/2651679896 wait complete. 2024-09-05T12:25:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:42 smithi123 ceph-mon[34321]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s wr, 18 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:42 smithi174 ceph-mon[34313]: pgmap v328: 129 pgs: 129 active+clean; 586 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s wr, 18 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:42.877 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:25:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:25:42.477+0000 7f16a0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:25:42.877 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:25:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:25:42.477+0000 7f16a0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:25:42.877 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:25:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:25:42.477+0000 7f16a0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:25:42.877 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:25:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:25:42.477+0000 7f16a0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:25:42.878 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:25:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:25:42.477+0000 7f16a0000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:25:43.291 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-05T12:25:43.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:43 smithi071 ceph-mon[29845]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:43.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:43 smithi071 ceph-mon[29845]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 724 B/s rd, 4.9 KiB/s wr, 16 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:43.661 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:43.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:43 smithi123 ceph-mon[34321]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:43.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:43 smithi123 ceph-mon[34321]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 724 B/s rd, 4.9 KiB/s wr, 16 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:43.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:43 smithi174 ceph-mon[34313]: from='client.14643 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:43.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:43 smithi174 ceph-mon[34313]: pgmap v329: 129 pgs: 129 active+clean; 587 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 724 B/s rd, 4.9 KiB/s wr, 16 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 -- 172.21.15.71:0/2548966013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c109120 msgr2=0x7fca4c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 --2- 172.21.15.71:0/2548966013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c109120 0x7fca4c110ca0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca44009ca0 tx=0x7fca4402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 -- 172.21.15.71:0/2548966013 shutdown_connections 2024-09-05T12:25:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 --2- 172.21.15.71:0/2548966013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c109120 0x7fca4c110ca0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 --2- 172.21.15.71:0/2548966013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c1049a0 0x7fca4c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 --2- 172.21.15.71:0/2548966013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c103ff0 0x7fca4c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 -- 172.21.15.71:0/2548966013 >> 172.21.15.71:0/2548966013 conn(0x7fca4c0ffa90 msgr2=0x7fca4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 -- 172.21.15.71:0/2548966013 shutdown_connections 2024-09-05T12:25:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.228+0000 7fca52e00640 1 -- 172.21.15.71:0/2548966013 wait complete. 2024-09-05T12:25:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.229+0000 7fca52e00640 1 Processor -- start 2024-09-05T12:25:44.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 -- start start 2024-09-05T12:25:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c103ff0 0x7fca4c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c1049a0 0x7fca4c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 0x7fca4c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c114150 con 0x7fca4c109120 2024-09-05T12:25:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c113fd0 con 0x7fca4c1049a0 2024-09-05T12:25:44.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.230+0000 7fca52e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca4c1142d0 con 0x7fca4c103ff0 2024-09-05T12:25:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 0x7fca4c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 0x7fca4c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32938/0 (socket says 172.21.15.71:32938) 2024-09-05T12:25:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 -- 172.21.15.71:0/537382643 learned_addr learned my addr 172.21.15.71:0/537382643 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:44.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca4b400640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c1049a0 0x7fca4c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca4be00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c103ff0 0x7fca4c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 -- 172.21.15.71:0/537382643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c103ff0 msgr2=0x7fca4c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c103ff0 0x7fca4c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 -- 172.21.15.71:0/537382643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c1049a0 msgr2=0x7fca4c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c1049a0 0x7fca4c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.231+0000 7fca50e00640 1 -- 172.21.15.71:0/537382643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4c19f560 con 0x7fca4c109120 2024-09-05T12:25:44.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.232+0000 7fca50e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 0x7fca4c19ee80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca44031a20 tx=0x7fca44031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:44.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.232+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca44003b70 con 0x7fca4c109120 2024-09-05T12:25:44.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.232+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca44003d10 con 0x7fca4c109120 2024-09-05T12:25:44.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.232+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4402fc40 con 0x7fca4c109120 2024-09-05T12:25:44.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.232+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca4c19f7f0 con 0x7fca4c109120 2024-09-05T12:25:44.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.233+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca4c1a7130 con 0x7fca4c109120 2024-09-05T12:25:44.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.234+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4c1043d0 con 0x7fca4c109120 2024-09-05T12:25:44.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.238+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fca44004050 con 0x7fca4c109120 2024-09-05T12:25:44.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.239+0000 7fca48c00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fca300a2c90 0x7fca300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:44.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.239+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fca440f0630 con 0x7fca4c109120 2024-09-05T12:25:44.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.239+0000 7fca4be00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fca300a2c90 0x7fca300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:44.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.240+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca44049060 con 0x7fca4c109120 2024-09-05T12:25:44.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.240+0000 7fca4be00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fca300a2c90 0x7fca300a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009bd0 tx=0x7fca3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:44.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:44 smithi071 ceph-mon[29845]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-05T12:25:44.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:44 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:25:44.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.450+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fca4c100d90 con 0x7fca300a2c90 2024-09-05T12:25:44.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.475+0000 7fca48c00640 1 -- 172.21.15.71:0/537382643 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca4c100d90 con 0x7fca300a2c90 2024-09-05T12:25:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.477+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fca300a2c90 msgr2=0x7fca300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.477+0000 7fca52e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fca300a2c90 0x7fca300a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009bd0 tx=0x7fca3c005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.477+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 msgr2=0x7fca4c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:44.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.477+0000 7fca52e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 0x7fca4c19ee80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fca44031a20 tx=0x7fca44031a50 comp rx=0 tx=0).stop 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 shutdown_connections 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fca300a2c90 0x7fca300a5150 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca4c109120 0x7fca4c19ee80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca4c1049a0 0x7fca4c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 --2- 172.21.15.71:0/537382643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca4c103ff0 0x7fca4c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 >> 172.21.15.71:0/537382643 conn(0x7fca4c0ffa90 msgr2=0x7fca4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:44.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 shutdown_connections 2024-09-05T12:25:44.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:44.478+0000 7fca52e00640 1 -- 172.21.15.71:0/537382643 wait complete. 2024-09-05T12:25:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:44 smithi123 ceph-mon[34321]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-05T12:25:44.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:44 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:25:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:44 smithi174 ceph-mon[34313]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-05T12:25:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:44 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:25:45.311 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-05T12:25:45.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:45 smithi071 ceph-mon[29845]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:45.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:45 smithi071 ceph-mon[29845]: pgmap v330: 129 pgs: 129 active+clean; 593 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:45.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:45 smithi123 ceph-mon[34321]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:45.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:45 smithi123 ceph-mon[34321]: pgmap v330: 129 pgs: 129 active+clean; 593 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:45.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:45 smithi174 ceph-mon[34313]: from='client.14655 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:45.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:45 smithi174 ceph-mon[34313]: pgmap v330: 129 pgs: 129 active+clean; 593 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.4 KiB/s wr, 22 op/s; 2 B/s, 0 objects/s recovering 2024-09-05T12:25:46.364 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:46.865 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:46 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.865 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:46 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.865 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:46 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.865 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:46 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:46 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:46 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:46 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.922 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:46 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:46 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:46 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:46 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:46 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:47.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 -- 172.21.15.71:0/4038178325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364106f00 msgr2=0x7fd36410ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:47.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 --2- 172.21.15.71:0/4038178325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364106f00 0x7fd36410ea80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd35c009be0 tx=0x7fd35c02a6c0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 -- 172.21.15.71:0/4038178325 shutdown_connections 2024-09-05T12:25:47.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 --2- 172.21.15.71:0/4038178325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364106f00 0x7fd36410ea80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 --2- 172.21.15.71:0/4038178325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd364102780 0x7fd3641067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 --2- 172.21.15.71:0/4038178325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd364101dd0 0x7fd3641021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 -- 172.21.15.71:0/4038178325 >> 172.21.15.71:0/4038178325 conn(0x7fd3640fd8b0 msgr2=0x7fd3640ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 -- 172.21.15.71:0/4038178325 shutdown_connections 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.982+0000 7fd368a00640 1 -- 172.21.15.71:0/4038178325 wait complete. 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 Processor -- start 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 -- start start 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd364101dd0 0x7fd36419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 0x7fd36419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd364106f00 0x7fd36419eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd364111f10 con 0x7fd364106f00 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd364111d90 con 0x7fd364101dd0 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.983+0000 7fd368a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd364112090 con 0x7fd364102780 2024-09-05T12:25:47.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 0x7fd36419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd362800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd364106f00 0x7fd36419eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 0x7fd36419ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:38534/0 (socket says 172.21.15.71:38534) 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 -- 172.21.15.71:0/2310281463 learned_addr learned my addr 172.21.15.71:0/2310281463 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361e00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd364101dd0 0x7fd36419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 -- 172.21.15.71:0/2310281463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd364101dd0 msgr2=0x7fd36419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd364101dd0 0x7fd36419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 -- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd364106f00 msgr2=0x7fd36419eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd364106f00 0x7fd36419eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 -- 172.21.15.71:0/2310281463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36419f5c0 con 0x7fd364102780 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.984+0000 7fd361400640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 0x7fd36419ab50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d950 tx=0x7fd35000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.985+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350017070 con 0x7fd364102780 2024-09-05T12:25:47.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.985+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd350009c30 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.985+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000f6b0 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.985+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36419f8b0 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.985+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641a7190 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.986+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3641021b0 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.988+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd3500109d0 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.989+0000 7fd35aa00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3480a2c90 0x7fd3480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.989+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fd3500ceff0 con 0x7fd364102780 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.990+0000 7fd361e00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3480a2c90 0x7fd3480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.990+0000 7fd361e00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3480a2c90 0x7fd3480a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd354009530 tx=0x7fd354009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:47.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:46.993+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd350009de0 con 0x7fd364102780 2024-09-05T12:25:47.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.210+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fd364075430 con 0x7fd3480a2c90 2024-09-05T12:25:47.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.215+0000 7fd35aa00640 1 -- 172.21.15.71:0/2310281463 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd364075430 con 0x7fd3480a2c90 2024-09-05T12:25:47.217 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/qa/sv_0/907454fc-92a1-40c0-8eb2-85c8bba76d45 2024-09-05T12:25:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3480a2c90 msgr2=0x7fd3480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3480a2c90 0x7fd3480a5150 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd354009530 tx=0x7fd354009260 comp rx=0 tx=0).stop 2024-09-05T12:25:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 msgr2=0x7fd36419ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 0x7fd36419ab50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d950 tx=0x7fd35000de10 comp rx=0 tx=0).stop 2024-09-05T12:25:47.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 shutdown_connections 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd3480a2c90 0x7fd3480a5150 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd364106f00 0x7fd36419eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd364102780 0x7fd36419ab50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 --2- 172.21.15.71:0/2310281463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd364101dd0 0x7fd36419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.218+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 >> 172.21.15.71:0/2310281463 conn(0x7fd3640fd8b0 msgr2=0x7fd3640febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.219+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 shutdown_connections 2024-09-05T12:25:47.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:47.219+0000 7fd368a00640 1 -- 172.21.15.71:0/2310281463 wait complete. 2024-09-05T12:25:47.865 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:47 smithi174 ceph-mon[34313]: pgmap v331: 129 pgs: 129 active+clean; 593 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.9 KiB/s wr, 20 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:47.865 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:47 smithi174 ceph-mon[34313]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:47.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:47 smithi071 ceph-mon[29845]: pgmap v331: 129 pgs: 129 active+clean; 593 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.9 KiB/s wr, 20 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:47.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:47 smithi071 ceph-mon[29845]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:47.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:47 smithi123 ceph-mon[34321]: pgmap v331: 129 pgs: 129 active+clean; 593 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 4.9 KiB/s wr, 20 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:47.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:47 smithi123 ceph-mon[34321]: from='client.24712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:48.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-05T12:25:48.488 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:49.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:25:49.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:25:49.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 -- 172.21.15.71:0/1903817710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c102360 msgr2=0x7fa51c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:49.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 --2- 172.21.15.71:0/1903817710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c102360 0x7fa51c10e890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa514009cd0 tx=0x7fa51402a8e0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 -- 172.21.15.71:0/1903817710 shutdown_connections 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 --2- 172.21.15.71:0/1903817710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c102360 0x7fa51c10e890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 --2- 172.21.15.71:0/1903817710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 --2- 172.21.15.71:0/1903817710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa51c1079c0 0x7fa51c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.157+0000 7fa51b200640 1 -- 172.21.15.71:0/1903817710 >> 172.21.15.71:0/1903817710 conn(0x7fa51c0fd740 msgr2=0x7fa51c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.158+0000 7fa51b200640 1 -- 172.21.15.71:0/1903817710 shutdown_connections 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.158+0000 7fa51b200640 1 -- 172.21.15.71:0/1903817710 wait complete. 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 Processor -- start 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 -- start start 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa51c102360 0x7fa51c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:49.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c1079c0 0x7fa51c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c111e10 con 0x7fa51c1079c0 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c111c90 con 0x7fa51c1019c0 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa51c111f90 con 0x7fa51c102360 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa519e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa51a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c1079c0 0x7fa51c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa519e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c19a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55188/0 (socket says 172.21.15.71:55188) 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.159+0000 7fa519e00640 1 -- 172.21.15.71:0/1929179844 learned_addr learned my addr 172.21.15.71:0/1929179844 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519400640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa51c102360 0x7fa51c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519e00640 1 -- 172.21.15.71:0/1929179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa51c102360 msgr2=0x7fa51c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519e00640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa51c102360 0x7fa51c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519e00640 1 -- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c1079c0 msgr2=0x7fa51c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:49.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519e00640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c1079c0 0x7fa51c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519e00640 1 -- 172.21.15.71:0/1929179844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51c19f400 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa519e00640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c19a450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00e9f0 tx=0x7fa50c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c00cd80 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa51c19f6f0 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa50c010910 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.160+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c0188f0 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.161+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51c1a6fd0 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.162+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa51c101e20 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.163+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa50c010ac0 con 0x7fa51c1019c0 2024-09-05T12:25:49.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.164+0000 7fa512a00640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5000a2c10 0x7fa5000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.165+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa50c0cf8d0 con 0x7fa51c1019c0 2024-09-05T12:25:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.165+0000 7fa519400640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5000a2c10 0x7fa5000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.165+0000 7fa519400640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5000a2c10 0x7fa5000a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5080095f0 tx=0x7fa508005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:49.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.168+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa50c09bcc0 con 0x7fa51c1019c0 2024-09-05T12:25:49.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.373+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa51c0fec80 con 0x7fa5000a2c10 2024-09-05T12:25:49.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.396+0000 7fa512a00640 1 -- 172.21.15.71:0/1929179844 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa51c0fec80 con 0x7fa5000a2c10 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5000a2c10 msgr2=0x7fa5000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5000a2c10 0x7fa5000a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa5080095f0 tx=0x7fa508005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 msgr2=0x7fa51c19a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c19a450 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa50c00e9f0 tx=0x7fa50c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 shutdown_connections 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa5000a2c10 0x7fa5000a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa51c1079c0 0x7fa51c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa51c102360 0x7fa51c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 --2- 172.21.15.71:0/1929179844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa51c1019c0 0x7fa51c19a450 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 >> 172.21.15.71:0/1929179844 conn(0x7fa51c0fd740 msgr2=0x7fa51c10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 shutdown_connections 2024-09-05T12:25:49.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:49.398+0000 7fa51b200640 1 -- 172.21.15.71:0/1929179844 wait complete. 2024-09-05T12:25:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:25:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:25:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:49.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:25:49.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:25:49.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:25:50.210 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-05T12:25:50.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:50 smithi071 ceph-mon[29845]: pgmap v332: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:50.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:50 smithi123 ceph-mon[34321]: pgmap v332: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:50 smithi174 ceph-mon[34313]: pgmap v332: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 6.5 KiB/s wr, 23 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:50.581 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:51.306 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:51 smithi071 ceph-mon[29845]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 -- 172.21.15.71:0/437697045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1019c0 msgr2=0x7f15ac101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 --2- 172.21.15.71:0/437697045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1019c0 0x7f15ac101e20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1598009930 tx=0x7f159802a420 comp rx=0 tx=0).stop 2024-09-05T12:25:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 -- 172.21.15.71:0/437697045 shutdown_connections 2024-09-05T12:25:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 --2- 172.21.15.71:0/437697045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ac102360 0x7f15ac10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 --2- 172.21.15.71:0/437697045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1019c0 0x7f15ac101e20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 --2- 172.21.15.71:0/437697045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1079c0 0x7f15ac107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 -- 172.21.15.71:0/437697045 >> 172.21.15.71:0/437697045 conn(0x7f15ac0fd6e0 msgr2=0x7f15ac0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 -- 172.21.15.71:0/437697045 shutdown_connections 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.304+0000 7f15ab200640 1 -- 172.21.15.71:0/437697045 wait complete. 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 Processor -- start 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 -- start start 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 0x7f15ac19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ac102360 0x7f15ac19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1079c0 0x7f15ac19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac111dc0 con 0x7f15ac1079c0 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac111c40 con 0x7f15ac1019c0 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.305+0000 7f15ab200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15ac111f40 con 0x7f15ac102360 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 0x7f15ac19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:51.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15aa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1079c0 0x7f15ac19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 0x7f15ac19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55202/0 (socket says 172.21.15.71:55202) 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 -- 172.21.15.71:0/1907330060 learned_addr learned my addr 172.21.15.71:0/1907330060 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9400640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ac102360 0x7f15ac19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 -- 172.21.15.71:0/1907330060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ac102360 msgr2=0x7f15ac19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ac102360 0x7f15ac19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 -- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1079c0 msgr2=0x7f15ac19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1079c0 0x7f15ac19ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 -- 172.21.15.71:0/1907330060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ac19f470 con 0x7f15ac1019c0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.306+0000 7f15a9e00640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 0x7f15ac19a4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f159c00e9f0 tx=0x7f159c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.307+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c00cd80 con 0x7f15ac1019c0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.307+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f159c010910 con 0x7f15ac1019c0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.307+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ac19f760 con 0x7f15ac1019c0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.307+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c018990 con 0x7f15ac1019c0 2024-09-05T12:25:51.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.307+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ac1a7040 con 0x7f15ac1019c0 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.308+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ac101e20 con 0x7f15ac1019c0 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.310+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f159c010ac0 con 0x7f15ac1019c0 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.311+0000 7f15a2a00640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f15900a2c10 0x7f15900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.311+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f159c0cf820 con 0x7f15ac1019c0 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.311+0000 7f15a9400640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f15900a2c10 0x7f15900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.312+0000 7f15a9400640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f15900a2c10 0x7f15900a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1598009900 tx=0x7f15980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:51.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.315+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f159c09bc10 con 0x7f15ac1019c0 2024-09-05T12:25:51.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:51 smithi123 ceph-mon[34321]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:51 smithi174 ceph-mon[34313]: from='client.24716 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:51.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.517+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f15ac0fecb0 con 0x7f15900a2c10 2024-09-05T12:25:51.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.523+0000 7f15a2a00640 1 -- 172.21.15.71:0/1907330060 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f15ac0fecb0 con 0x7f15900a2c10 2024-09-05T12:25:51.525 INFO:teuthology.orchestra.run.smithi071.stdout:/volumes/qa/sv_1/58aebd0b-d33c-4637-a6a0-e71d9cc8243f 2024-09-05T12:25:51.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f15900a2c10 msgr2=0x7f15900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:51.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f15900a2c10 0x7f15900a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1598009900 tx=0x7f15980058d0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 msgr2=0x7f15ac19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 0x7f15ac19a4c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f159c00e9f0 tx=0x7f159c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 shutdown_connections 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f15900a2c10 0x7f15900a50d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15ac1079c0 0x7f15ac19ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15ac102360 0x7f15ac19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 --2- 172.21.15.71:0/1907330060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ac1019c0 0x7f15ac19a4c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.525+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 >> 172.21.15.71:0/1907330060 conn(0x7f15ac0fd6e0 msgr2=0x7f15ac10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.526+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 shutdown_connections 2024-09-05T12:25:51.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:51.526+0000 7f15ab200640 1 -- 172.21.15.71:0/1907330060 wait complete. 2024-09-05T12:25:52.313 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:52 smithi071 ceph-mon[29845]: pgmap v333: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.2 KiB/s wr, 22 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:52.354 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:25:52.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:52 smithi123 ceph-mon[34321]: pgmap v333: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.2 KiB/s wr, 22 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:52 smithi174 ceph-mon[34313]: pgmap v333: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 6.2 KiB/s wr, 22 op/s; 1 B/s, 0 objects/s recovering 2024-09-05T12:25:52.728 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 -- 172.21.15.71:0/3516592429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb990108f60 msgr2=0x7fb990110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 --2- 172.21.15.71:0/3516592429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb990108f60 0x7fb990110ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb988009ca0 tx=0x7fb98802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 -- 172.21.15.71:0/3516592429 shutdown_connections 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 --2- 172.21.15.71:0/3516592429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb990108f60 0x7fb990110ae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 --2- 172.21.15.71:0/3516592429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9901047e0 0x7fb990108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 --2- 172.21.15.71:0/3516592429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990103e30 0x7fb990104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 -- 172.21.15.71:0/3516592429 >> 172.21.15.71:0/3516592429 conn(0x7fb9900ff8d0 msgr2=0x7fb990101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 -- 172.21.15.71:0/3516592429 shutdown_connections 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.344+0000 7fb994c00640 1 -- 172.21.15.71:0/3516592429 wait complete. 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.345+0000 7fb994c00640 1 Processor -- start 2024-09-05T12:25:53.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.345+0000 7fb994c00640 1 -- start start 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb994c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990103e30 0x7fb99019a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb994c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9901047e0 0x7fb99019a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb994c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990108f60 0x7fb99019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb994c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113fc0 con 0x7fb9901047e0 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb994c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb990113e40 con 0x7fb990103e30 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb994c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb990114140 con 0x7fb990108f60 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb987e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9901047e0 0x7fb99019a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990103e30 0x7fb99019a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb987e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9901047e0 0x7fb99019a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41172/0 (socket says 172.21.15.71:41172) 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb987e00640 1 -- 172.21.15.71:0/2311430160 learned_addr learned my addr 172.21.15.71:0/2311430160 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:53.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98fe00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990108f60 0x7fb99019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98f400640 1 -- 172.21.15.71:0/2311430160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990108f60 msgr2=0x7fb99019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98f400640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990108f60 0x7fb99019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98f400640 1 -- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9901047e0 msgr2=0x7fb99019a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98f400640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9901047e0 0x7fb99019a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.346+0000 7fb98f400640 1 -- 172.21.15.71:0/2311430160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99019f380 con 0x7fb990103e30 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.347+0000 7fb98f400640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990103e30 0x7fb99019a3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb98000e9f0 tx=0x7fb98000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.347+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98000ccc0 con 0x7fb990103e30 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.347+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99019f670 con 0x7fb990103e30 2024-09-05T12:25:53.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.347+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb980005020 con 0x7fb990103e30 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.347+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9800049e0 con 0x7fb990103e30 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.347+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901a6f50 con 0x7fb990103e30 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.348+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb990104210 con 0x7fb990103e30 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.350+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb980005560 con 0x7fb990103e30 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.351+0000 7fb98cc00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9740a2c10 0x7fb9740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.351+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fb9800cf4f0 con 0x7fb990103e30 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.351+0000 7fb987e00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9740a2c10 0x7fb9740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.352+0000 7fb987e00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9740a2c10 0x7fb9740a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb99010a8d0 tx=0x7fb978005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:53.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.353+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9800051d0 con 0x7fb990103e30 2024-09-05T12:25:53.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:53 smithi123 ceph-mon[34321]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:53.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:53 smithi174 ceph-mon[34313]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:53.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.562+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb99010c200 con 0x7fb990103e30 2024-09-05T12:25:53.564 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:53 smithi071 ceph-mon[29845]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:25:53.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.563+0000 7fb98cc00640 1 -- 172.21.15.71:0/2311430160 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11968 (secure 0 0 0) 0x7fb98009bf30 con 0x7fb990103e30 2024-09-05T12:25:53.565 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:53.565 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"btime":"2024-09-05T12:25:40:321146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:40.307362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:25:53.565 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:25:53.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9740a2c10 msgr2=0x7fb9740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:53.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9740a2c10 0x7fb9740a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb99010a8d0 tx=0x7fb978005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:53.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990103e30 msgr2=0x7fb99019a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990103e30 0x7fb99019a3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb98000e9f0 tx=0x7fb98000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 shutdown_connections 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9740a2c10 0x7fb9740a50d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990108f60 0x7fb99019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9901047e0 0x7fb99019a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 --2- 172.21.15.71:0/2311430160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990103e30 0x7fb99019a3d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 >> 172.21.15.71:0/2311430160 conn(0x7fb9900ff8d0 msgr2=0x7fb990100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.565+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 shutdown_connections 2024-09-05T12:25:53.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:53.566+0000 7fb994c00640 1 -- 172.21.15.71:0/2311430160 wait complete. 2024-09-05T12:25:54.282 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:54 smithi071 ceph-mon[29845]: pgmap v334: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-05T12:25:54.282 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:54 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2311430160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:54.320 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:25:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:54 smithi123 ceph-mon[34321]: pgmap v334: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-05T12:25:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:54 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2311430160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:54 smithi174 ceph-mon[34313]: pgmap v334: 129 pgs: 129 active+clean; 612 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-05T12:25:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:54 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2311430160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:25:54.685 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 -- 172.21.15.71:0/4111709105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 msgr2=0x7f56541150b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 --2- 172.21.15.71:0/4111709105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f56541150b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f564c009ca0 tx=0x7f564c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 -- 172.21.15.71:0/4111709105 shutdown_connections 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 --2- 172.21.15.71:0/4111709105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f56541150b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 --2- 172.21.15.71:0/4111709105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5654108e00 0x7f565410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 --2- 172.21.15.71:0/4111709105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f5654108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 -- 172.21.15.71:0/4111709105 >> 172.21.15.71:0/4111709105 conn(0x7f5654077ba0 msgr2=0x7f5654079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:55.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 -- 172.21.15.71:0/4111709105 shutdown_connections 2024-09-05T12:25:55.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.269+0000 7f5659000640 1 -- 172.21.15.71:0/4111709105 wait complete. 2024-09-05T12:25:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 Processor -- start 2024-09-05T12:25:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 -- start start 2024-09-05T12:25:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f56541a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5654108e00 0x7f56541a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f565419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:55.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5654118560 con 0x7f565410d530 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56541183e0 con 0x7f5654108e00 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.270+0000 7f5659000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56541186e0 con 0x7f5654108450 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5652800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f565419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5651e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f56541a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5651e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f56541a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42784/0 (socket says 172.21.15.71:42784) 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5651e00640 1 -- 172.21.15.71:0/2122815169 learned_addr learned my addr 172.21.15.71:0/2122815169 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f564be00640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5654108e00 0x7f56541a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5652800640 1 -- 172.21.15.71:0/2122815169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 msgr2=0x7f56541a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5652800640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f56541a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5652800640 1 -- 172.21.15.71:0/2122815169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5654108e00 msgr2=0x7f56541a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5652800640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5654108e00 0x7f56541a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5652800640 1 -- 172.21.15.71:0/2122815169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565419efd0 con 0x7f565410d530 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.271+0000 7f5651e00640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f56541a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:25:55.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.272+0000 7f5652800640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f565419e830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f564c02ade0 tx=0x7f564c031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:55.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.272+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c003b70 con 0x7f565410d530 2024-09-05T12:25:55.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.272+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f564c003d10 con 0x7f565410d530 2024-09-05T12:25:55.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.272+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564c02fba0 con 0x7f565410d530 2024-09-05T12:25:55.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.272+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565419f260 con 0x7f565410d530 2024-09-05T12:25:55.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.272+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56541ab510 con 0x7f565410d530 2024-09-05T12:25:55.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.273+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56541098e0 con 0x7f565410d530 2024-09-05T12:25:55.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.275+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f564c0046c0 con 0x7f565410d530 2024-09-05T12:25:55.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.276+0000 7f564aa00640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56380a2c90 0x7f56380a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:55.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.276+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f564c0f03e0 con 0x7f565410d530 2024-09-05T12:25:55.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.277+0000 7f5651e00640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56380a2c90 0x7f56380a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:55.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.277+0000 7f5651e00640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56380a2c90 0x7f56380a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5644011500 tx=0x7f5644005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:55.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.281+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f564c0bd0b0 con 0x7f565410d530 2024-09-05T12:25:55.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.458+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56541ab6e0 con 0x7f565410d530 2024-09-05T12:25:55.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.460+0000 7f564aa00640 1 -- 172.21.15.71:0/2122815169 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24741 (secure 0 0 0) 0x7f564c0bc9b0 con 0x7f565410d530 2024-09-05T12:25:55.462 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:55.462 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:34.259275+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:25:55.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.462+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56380a2c90 msgr2=0x7f56380a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.462+0000 7f5659000640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56380a2c90 0x7f56380a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5644011500 tx=0x7f5644005f90 comp rx=0 tx=0).stop 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 msgr2=0x7f565419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f565419e830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f564c02ade0 tx=0x7f564c031940 comp rx=0 tx=0).stop 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 shutdown_connections 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f56380a2c90 0x7f56380a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f565410d530 0x7f565419e830 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5654108e00 0x7f56541a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 --2- 172.21.15.71:0/2122815169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5654108450 0x7f56541a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 >> 172.21.15.71:0/2122815169 conn(0x7f5654077ba0 msgr2=0x7f5654079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 shutdown_connections 2024-09-05T12:25:55.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:55.463+0000 7f5659000640 1 -- 172.21.15.71:0/2122815169 wait complete. 2024-09-05T12:25:56.197 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T12:25:56.197 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:25:56.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:56 smithi071 ceph-mon[29845]: pgmap v335: 129 pgs: 129 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-09-05T12:25:56.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2122815169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:56.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:56 smithi123 ceph-mon[34321]: pgmap v335: 129 pgs: 129 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-09-05T12:25:56.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2122815169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:56 smithi174 ceph-mon[34313]: pgmap v335: 129 pgs: 129 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.2 KiB/s wr, 17 op/s 2024-09-05T12:25:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2122815169' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:25:56.565 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:57.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 -- 172.21.15.71:0/975502322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050103e30 msgr2=0x7ff050104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 --2- 172.21.15.71:0/975502322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050103e30 0x7ff050104210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff04002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 -- 172.21.15.71:0/975502322 shutdown_connections 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 --2- 172.21.15.71:0/975502322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff050108f60 0x7ff050110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 --2- 172.21.15.71:0/975502322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0501047e0 0x7ff050108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 --2- 172.21.15.71:0/975502322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff050103e30 0x7ff050104210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 -- 172.21.15.71:0/975502322 >> 172.21.15.71:0/975502322 conn(0x7ff0500ff8d0 msgr2=0x7ff050101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 -- 172.21.15.71:0/975502322 shutdown_connections 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.152+0000 7ff055200640 1 -- 172.21.15.71:0/975502322 wait complete. 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 Processor -- start 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 -- start start 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff050103e30 0x7ff05019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:57.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 0x7ff05019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff050108f60 0x7ff05019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff050114010 con 0x7ff0501047e0 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff050113e90 con 0x7ff050103e30 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.153+0000 7ff055200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff050114190 con 0x7ff050108f60 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 0x7ff05019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 0x7ff05019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41216/0 (socket says 172.21.15.71:41216) 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 -- 172.21.15.71:0/3877501655 learned_addr learned my addr 172.21.15.71:0/3877501655 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04f400640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff050103e30 0x7ff05019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04fe00640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff050108f60 0x7ff05019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 -- 172.21.15.71:0/3877501655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff050108f60 msgr2=0x7ff05019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff050108f60 0x7ff05019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 -- 172.21.15.71:0/3877501655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff050103e30 msgr2=0x7ff05019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:57.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff050103e30 0x7ff05019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04ea00640 1 -- 172.21.15.71:0/3877501655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05019f440 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.154+0000 7ff04fe00640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff050108f60 0x7ff05019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.155+0000 7ff04ea00640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 0x7ff05019a9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e9f0 tx=0x7ff03c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.155+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c00cd80 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.155+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05019f730 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.155+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff03c005500 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.155+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c010690 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.155+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501a7010 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.156+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff050104210 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.158+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff03c0046c0 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.160+0000 7ff047e00640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff0340a2c10 0x7ff0340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.160+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff03c0cfa20 con 0x7ff0501047e0 2024-09-05T12:25:57.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.160+0000 7ff04f400640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff0340a2c10 0x7ff0340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:57.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.161+0000 7ff04f400640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff0340a2c10 0x7ff0340a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff040000c00 tx=0x7ff040005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:57.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.164+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff03c09c460 con 0x7ff0501047e0 2024-09-05T12:25:57.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.338+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff050062630 con 0x7ff0340a2c10 2024-09-05T12:25:57.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.352+0000 7ff047e00640 1 -- 172.21.15.71:0/3877501655 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340416 (secure 0 0 0) 0x7ff050062630 con 0x7ff0340a2c10 2024-09-05T12:25:57.358 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:57.360 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:25:57.362 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff0340a2c10 msgr2=0x7ff0340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff0340a2c10 0x7ff0340a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff040000c00 tx=0x7ff040005b50 comp rx=0 tx=0).stop 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 msgr2=0x7ff05019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 0x7ff05019a9d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00e9f0 tx=0x7ff03c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 shutdown_connections 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff0340a2c10 0x7ff0340a50d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff050108f60 0x7ff05019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0501047e0 0x7ff05019a9d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 --2- 172.21.15.71:0/3877501655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff050103e30 0x7ff05019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 >> 172.21.15.71:0/3877501655 conn(0x7ff0500ff8d0 msgr2=0x7ff050100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 shutdown_connections 2024-09-05T12:25:57.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:57.357+0000 7ff055200640 1 -- 172.21.15.71:0/3877501655 wait complete. 2024-09-05T12:25:58.154 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-05T12:25:57.152327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649830,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":326,"num_read_kb":279,"num_write":422,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376508,"kb_used_data":14780,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565316,"statfs":{"total":1151940427776,"available":1151554883584,"internally_reserved":0,"allocated":15134720,"data_stored":8320376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42201,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":93,"num_write":25,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003109"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":66,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346062+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:25:52.346062+0000","last_peered":"2024-09-05T12:25:52.346062+0000","last_clean":"2024-09-05T12:25:52.346062+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:25:52.346062+0000","last_undegraded":"2024-09-05T12:25:52.346062+0000","last_fullsized":"2024-09-05T12:25:52.346062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263356+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:25:34.263356+0000","last_peered":"2024-09-05T12:25:34.263356+0000","last_clean":"2024-09-05T12:25:34.263356+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:25:34.263356+0000","last_undegraded":"2024-09-05T12:25:34.263356+0000","last_fullsized":"2024-09-05T12:25:34.263356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263685+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:25:34.263685+0000","last_peered":"2024-09-05T12:25:34.263685+0000","last_clean":"2024-09-05T12:25:34.263685+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:25:34.263685+0000","last_undegraded":"2024-09-05T12:25:34.263685+0000","last_fullsized":"2024-09-05T12:25:34.263685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263635+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:25:34.263635+0000","last_peered":"2024-09-05T12:25:34.263635+0000","last_clean":"2024-09-05T12:25:34.263635+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:25:34.263635+0000","last_undegraded":"2024-09-05T12:25:34.263635+0000","last_fullsized":"2024-09-05T12:25:34.263635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268221+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:25:34.268221+0000","last_peered":"2024-09-05T12:25:34.268221+0000","last_clean":"2024-09-05T12:25:34.268221+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:25:34.268221+0000","last_undegraded":"2024-09-05T12:25:34.268221+0000","last_fullsized":"2024-09-05T12:25:34.268221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264135+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:25:34.264135+0000","last_peered":"2024-09-05T12:25:34.264135+0000","last_clean":"2024-09-05T12:25:34.264135+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:25:34.264135+0000","last_undegraded":"2024-09-05T12:25:34.264135+0000","last_fullsized":"2024-09-05T12:25:34.264135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.317765+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:25:39.317765+0000","last_peered":"2024-09-05T12:25:39.317765+0000","last_clean":"2024-09-05T12:25:39.317765+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:25:39.317765+0000","last_undegraded":"2024-09-05T12:25:39.317765+0000","last_fullsized":"2024-09-05T12:25:39.317765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404504+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:25:34.404504+0000","last_peered":"2024-09-05T12:25:34.404504+0000","last_clean":"2024-09-05T12:25:34.404504+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:25:34.404504+0000","last_undegraded":"2024-09-05T12:25:34.404504+0000","last_fullsized":"2024-09-05T12:25:34.404504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264558+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:25:34.264558+0000","last_peered":"2024-09-05T12:25:34.264558+0000","last_clean":"2024-09-05T12:25:34.264558+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:25:34.264558+0000","last_undegraded":"2024-09-05T12:25:34.264558+0000","last_fullsized":"2024-09-05T12:25:34.264558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322603+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:25:38.322603+0000","last_peered":"2024-09-05T12:25:38.322603+0000","last_clean":"2024-09-05T12:25:38.322603+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:25:38.322603+0000","last_undegraded":"2024-09-05T12:25:38.322603+0000","last_fullsized":"2024-09-05T12:25:38.322603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271656+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:25:34.271656+0000","last_peered":"2024-09-05T12:25:34.271656+0000","last_clean":"2024-09-05T12:25:34.271656+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:25:34.271656+0000","last_undegraded":"2024-09-05T12:25:34.271656+0000","last_fullsized":"2024-09-05T12:25:34.271656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264243+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:25:34.264243+0000","last_peered":"2024-09-05T12:25:34.264243+0000","last_clean":"2024-09-05T12:25:34.264243+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:25:34.264243+0000","last_undegraded":"2024-09-05T12:25:34.264243+0000","last_fullsized":"2024-09-05T12:25:34.264243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268524+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:25:34.268524+0000","last_peered":"2024-09-05T12:25:34.268524+0000","last_clean":"2024-09-05T12:25:34.268524+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:25:34.268524+0000","last_undegraded":"2024-09-05T12:25:34.268524+0000","last_fullsized":"2024-09-05T12:25:34.268524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.294392+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:25:35.294392+0000","last_peered":"2024-09-05T12:25:35.294392+0000","last_clean":"2024-09-05T12:25:35.294392+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:25:35.294392+0000","last_undegraded":"2024-09-05T12:25:35.294392+0000","last_fullsized":"2024-09-05T12:25:35.294392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264525+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:25:34.264525+0000","last_peered":"2024-09-05T12:25:34.264525+0000","last_clean":"2024-09-05T12:25:34.264525+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:25:34.264525+0000","last_undegraded":"2024-09-05T12:25:34.264525+0000","last_fullsized":"2024-09-05T12:25:34.264525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263138+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:25:34.263138+0000","last_peered":"2024-09-05T12:25:34.263138+0000","last_clean":"2024-09-05T12:25:34.263138+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:25:34.263138+0000","last_undegraded":"2024-09-05T12:25:34.263138+0000","last_fullsized":"2024-09-05T12:25:34.263138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313403+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:25:37.313403+0000","last_peered":"2024-09-05T12:25:37.313403+0000","last_clean":"2024-09-05T12:25:37.313403+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:25:37.313403+0000","last_undegraded":"2024-09-05T12:25:37.313403+0000","last_fullsized":"2024-09-05T12:25:37.313403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271242+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:25:34.271242+0000","last_peered":"2024-09-05T12:25:34.271242+0000","last_clean":"2024-09-05T12:25:34.271242+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:25:34.271242+0000","last_undegraded":"2024-09-05T12:25:34.271242+0000","last_fullsized":"2024-09-05T12:25:34.271242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265378+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:25:34.265378+0000","last_peered":"2024-09-05T12:25:34.265378+0000","last_clean":"2024-09-05T12:25:34.265378+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:25:34.265378+0000","last_undegraded":"2024-09-05T12:25:34.265378+0000","last_fullsized":"2024-09-05T12:25:34.265378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323200+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:25:38.323200+0000","last_peered":"2024-09-05T12:25:38.323200+0000","last_clean":"2024-09-05T12:25:38.323200+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:25:38.323200+0000","last_undegraded":"2024-09-05T12:25:38.323200+0000","last_fullsized":"2024-09-05T12:25:38.323200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330571+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:25:34.330571+0000","last_peered":"2024-09-05T12:25:34.330571+0000","last_clean":"2024-09-05T12:25:34.330571+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:25:34.330571+0000","last_undegraded":"2024-09-05T12:25:34.330571+0000","last_fullsized":"2024-09-05T12:25:34.330571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265675+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:25:34.265675+0000","last_peered":"2024-09-05T12:25:34.265675+0000","last_clean":"2024-09-05T12:25:34.265675+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:25:34.265675+0000","last_undegraded":"2024-09-05T12:25:34.265675+0000","last_fullsized":"2024-09-05T12:25:34.265675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.293537+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:25:35.293537+0000","last_peered":"2024-09-05T12:25:35.293537+0000","last_clean":"2024-09-05T12:25:35.293537+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:25:35.293537+0000","last_undegraded":"2024-09-05T12:25:35.293537+0000","last_fullsized":"2024-09-05T12:25:35.293537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264519+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:25:34.264519+0000","last_peered":"2024-09-05T12:25:34.264519+0000","last_clean":"2024-09-05T12:25:34.264519+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:25:34.264519+0000","last_undegraded":"2024-09-05T12:25:34.264519+0000","last_fullsized":"2024-09-05T12:25:34.264519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268273+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:25:34.268273+0000","last_peered":"2024-09-05T12:25:34.268273+0000","last_clean":"2024-09-05T12:25:34.268273+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:25:34.268273+0000","last_undegraded":"2024-09-05T12:25:34.268273+0000","last_fullsized":"2024-09-05T12:25:34.268273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323273+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:25:38.323273+0000","last_peered":"2024-09-05T12:25:38.323273+0000","last_clean":"2024-09-05T12:25:38.323273+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:25:38.323273+0000","last_undegraded":"2024-09-05T12:25:38.323273+0000","last_fullsized":"2024-09-05T12:25:38.323273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263934+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:25:34.263934+0000","last_peered":"2024-09-05T12:25:34.263934+0000","last_clean":"2024-09-05T12:25:34.263934+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:25:34.263934+0000","last_undegraded":"2024-09-05T12:25:34.263934+0000","last_fullsized":"2024-09-05T12:25:34.263934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263632+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:25:34.263632+0000","last_peered":"2024-09-05T12:25:34.263632+0000","last_clean":"2024-09-05T12:25:34.263632+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:25:34.263632+0000","last_undegraded":"2024-09-05T12:25:34.263632+0000","last_fullsized":"2024-09-05T12:25:34.263632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'12","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:49.521658+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:25:49.521658+0000","last_peered":"2024-09-05T12:25:49.521658+0000","last_clean":"2024-09-05T12:25:49.521658+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:25:49.521658+0000","last_undegraded":"2024-09-05T12:25:49.521658+0000","last_fullsized":"2024-09-05T12:25:49.521658+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.262906+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:25:34.262906+0000","last_peered":"2024-09-05T12:25:34.262906+0000","last_clean":"2024-09-05T12:25:34.262906+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:25:34.262906+0000","last_undegraded":"2024-09-05T12:25:34.262906+0000","last_fullsized":"2024-09-05T12:25:34.262906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267595+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:25:34.267595+0000","last_peered":"2024-09-05T12:25:34.267595+0000","last_clean":"2024-09-05T12:25:34.267595+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:25:34.267595+0000","last_undegraded":"2024-09-05T12:25:34.267595+0000","last_fullsized":"2024-09-05T12:25:34.267595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323069+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:25:38.323069+0000","last_peered":"2024-09-05T12:25:38.323069+0000","last_clean":"2024-09-05T12:25:38.323069+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:25:38.323069+0000","last_undegraded":"2024-09-05T12:25:38.323069+0000","last_fullsized":"2024-09-05T12:25:38.323069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265409+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:25:34.265409+0000","last_peered":"2024-09-05T12:25:34.265409+0000","last_clean":"2024-09-05T12:25:34.265409+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:25:34.265409+0000","last_undegraded":"2024-09-05T12:25:34.265409+0000","last_fullsized":"2024-09-05T12:25:34.265409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267791+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:25:34.267791+0000","last_peered":"2024-09-05T12:25:34.267791+0000","last_clean":"2024-09-05T12:25:34.267791+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:25:34.267791+0000","last_undegraded":"2024-09-05T12:25:34.267791+0000","last_fullsized":"2024-09-05T12:25:34.267791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330392+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:25:34.330392+0000","last_peered":"2024-09-05T12:25:34.330392+0000","last_clean":"2024-09-05T12:25:34.330392+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:25:34.330392+0000","last_undegraded":"2024-09-05T12:25:34.330392+0000","last_fullsized":"2024-09-05T12:25:34.330392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266261+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:25:34.266261+0000","last_peered":"2024-09-05T12:25:34.266261+0000","last_clean":"2024-09-05T12:25:34.266261+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:25:34.266261+0000","last_undegraded":"2024-09-05T12:25:34.266261+0000","last_fullsized":"2024-09-05T12:25:34.266261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267674+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:25:34.267674+0000","last_peered":"2024-09-05T12:25:34.267674+0000","last_clean":"2024-09-05T12:25:34.267674+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:25:34.267674+0000","last_undegraded":"2024-09-05T12:25:34.267674+0000","last_fullsized":"2024-09-05T12:25:34.267674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346549+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:25:52.346549+0000","last_peered":"2024-09-05T12:25:52.346549+0000","last_clean":"2024-09-05T12:25:52.346549+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:25:52.346549+0000","last_undegraded":"2024-09-05T12:25:52.346549+0000","last_fullsized":"2024-09-05T12:25:52.346549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267415+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:25:34.267415+0000","last_peered":"2024-09-05T12:25:34.267415+0000","last_clean":"2024-09-05T12:25:34.267415+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:25:34.267415+0000","last_undegraded":"2024-09-05T12:25:34.267415+0000","last_fullsized":"2024-09-05T12:25:34.267415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269430+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:25:34.269430+0000","last_peered":"2024-09-05T12:25:34.269430+0000","last_clean":"2024-09-05T12:25:34.269430+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:25:34.269430+0000","last_undegraded":"2024-09-05T12:25:34.269430+0000","last_fullsized":"2024-09-05T12:25:34.269430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":59,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346062+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:25:52.346062+0000","last_peered":"2024-09-05T12:25:52.346062+0000","last_clean":"2024-09-05T12:25:52.346062+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:25:52.346062+0000","last_undegraded":"2024-09-05T12:25:52.346062+0000","last_fullsized":"2024-09-05T12:25:52.346062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271660+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:25:34.271660+0000","last_peered":"2024-09-05T12:25:34.271660+0000","last_clean":"2024-09-05T12:25:34.271660+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:25:34.271660+0000","last_undegraded":"2024-09-05T12:25:34.271660+0000","last_fullsized":"2024-09-05T12:25:34.271660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266417+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:25:34.266417+0000","last_peered":"2024-09-05T12:25:34.266417+0000","last_clean":"2024-09-05T12:25:34.266417+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:25:34.266417+0000","last_undegraded":"2024-09-05T12:25:34.266417+0000","last_fullsized":"2024-09-05T12:25:34.266417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.299617+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:25:36.299617+0000","last_peered":"2024-09-05T12:25:36.299617+0000","last_clean":"2024-09-05T12:25:36.299617+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:25:36.299617+0000","last_undegraded":"2024-09-05T12:25:36.299617+0000","last_fullsized":"2024-09-05T12:25:36.299617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272425+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:25:34.272425+0000","last_peered":"2024-09-05T12:25:34.272425+0000","last_clean":"2024-09-05T12:25:34.272425+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:25:34.272425+0000","last_undegraded":"2024-09-05T12:25:34.272425+0000","last_fullsized":"2024-09-05T12:25:34.272425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263271+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:25:34.263271+0000","last_peered":"2024-09-05T12:25:34.263271+0000","last_clean":"2024-09-05T12:25:34.263271+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:25:34.263271+0000","last_undegraded":"2024-09-05T12:25:34.263271+0000","last_fullsized":"2024-09-05T12:25:34.263271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.325619+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:25:39.325619+0000","last_peered":"2024-09-05T12:25:39.325619+0000","last_clean":"2024-09-05T12:25:39.325619+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:25:39.325619+0000","last_undegraded":"2024-09-05T12:25:39.325619+0000","last_fullsized":"2024-09-05T12:25:39.325619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269719+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:25:34.269719+0000","last_peered":"2024-09-05T12:25:34.269719+0000","last_clean":"2024-09-05T12:25:34.269719+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:25:34.269719+0000","last_undegraded":"2024-09-05T12:25:34.269719+0000","last_fullsized":"2024-09-05T12:25:34.269719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268797+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:25:34.268797+0000","last_peered":"2024-09-05T12:25:34.268797+0000","last_clean":"2024-09-05T12:25:34.268797+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:25:34.268797+0000","last_undegraded":"2024-09-05T12:25:34.268797+0000","last_fullsized":"2024-09-05T12:25:34.268797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269019+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:25:34.269019+0000","last_peered":"2024-09-05T12:25:34.269019+0000","last_clean":"2024-09-05T12:25:34.269019+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:25:34.269019+0000","last_undegraded":"2024-09-05T12:25:34.269019+0000","last_fullsized":"2024-09-05T12:25:34.269019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'6","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:51.345411+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:25:51.345411+0000","last_peered":"2024-09-05T12:25:51.345411+0000","last_clean":"2024-09-05T12:25:51.345411+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:25:51.345411+0000","last_undegraded":"2024-09-05T12:25:51.345411+0000","last_fullsized":"2024-09-05T12:25:51.345411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404323+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:25:34.404323+0000","last_peered":"2024-09-05T12:25:34.404323+0000","last_clean":"2024-09-05T12:25:34.404323+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:25:34.404323+0000","last_undegraded":"2024-09-05T12:25:34.404323+0000","last_fullsized":"2024-09-05T12:25:34.404323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269554+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:25:34.269554+0000","last_peered":"2024-09-05T12:25:34.269554+0000","last_clean":"2024-09-05T12:25:34.269554+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:25:34.269554+0000","last_undegraded":"2024-09-05T12:25:34.269554+0000","last_fullsized":"2024-09-05T12:25:34.269554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321412+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:25:38.321412+0000","last_peered":"2024-09-05T12:25:38.321412+0000","last_clean":"2024-09-05T12:25:38.321412+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:25:38.321412+0000","last_undegraded":"2024-09-05T12:25:38.321412+0000","last_fullsized":"2024-09-05T12:25:38.321412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267091+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:25:34.267091+0000","last_peered":"2024-09-05T12:25:34.267091+0000","last_clean":"2024-09-05T12:25:34.267091+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:25:34.267091+0000","last_undegraded":"2024-09-05T12:25:34.267091+0000","last_fullsized":"2024-09-05T12:25:34.267091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268658+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:25:34.268658+0000","last_peered":"2024-09-05T12:25:34.268658+0000","last_clean":"2024-09-05T12:25:34.268658+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:25:34.268658+0000","last_undegraded":"2024-09-05T12:25:34.268658+0000","last_fullsized":"2024-09-05T12:25:34.268658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323388+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:25:38.323388+0000","last_peered":"2024-09-05T12:25:38.323388+0000","last_clean":"2024-09-05T12:25:38.323388+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:25:38.323388+0000","last_undegraded":"2024-09-05T12:25:38.323388+0000","last_fullsized":"2024-09-05T12:25:38.323388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269057+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:25:34.269057+0000","last_peered":"2024-09-05T12:25:34.269057+0000","last_clean":"2024-09-05T12:25:34.269057+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:25:34.269057+0000","last_undegraded":"2024-09-05T12:25:34.269057+0000","last_fullsized":"2024-09-05T12:25:34.269057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266821+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:25:34.266821+0000","last_peered":"2024-09-05T12:25:34.266821+0000","last_clean":"2024-09-05T12:25:34.266821+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:25:34.266821+0000","last_undegraded":"2024-09-05T12:25:34.266821+0000","last_fullsized":"2024-09-05T12:25:34.266821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.296703+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:25:38.296703+0000","last_peered":"2024-09-05T12:25:38.296703+0000","last_clean":"2024-09-05T12:25:38.296703+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:25:38.296703+0000","last_undegraded":"2024-09-05T12:25:38.296703+0000","last_fullsized":"2024-09-05T12:25:38.296703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270206+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:25:34.270206+0000","last_peered":"2024-09-05T12:25:34.270206+0000","last_clean":"2024-09-05T12:25:34.270206+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:25:34.270206+0000","last_undegraded":"2024-09-05T12:25:34.270206+0000","last_fullsized":"2024-09-05T12:25:34.270206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270383+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:25:34.270383+0000","last_peered":"2024-09-05T12:25:34.270383+0000","last_clean":"2024-09-05T12:25:34.270383+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:25:34.270383+0000","last_undegraded":"2024-09-05T12:25:34.270383+0000","last_fullsized":"2024-09-05T12:25:34.270383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321005+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:25:38.321005+0000","last_peered":"2024-09-05T12:25:38.321005+0000","last_clean":"2024-09-05T12:25:38.321005+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:25:38.321005+0000","last_undegraded":"2024-09-05T12:25:38.321005+0000","last_fullsized":"2024-09-05T12:25:38.321005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267124+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:25:34.267124+0000","last_peered":"2024-09-05T12:25:34.267124+0000","last_clean":"2024-09-05T12:25:34.267124+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:25:34.267124+0000","last_undegraded":"2024-09-05T12:25:34.267124+0000","last_fullsized":"2024-09-05T12:25:34.267124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331448+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:25:34.331448+0000","last_peered":"2024-09-05T12:25:34.331448+0000","last_clean":"2024-09-05T12:25:34.331448+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:25:34.331448+0000","last_undegraded":"2024-09-05T12:25:34.331448+0000","last_fullsized":"2024-09-05T12:25:34.331448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":59,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346562+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:25:52.346562+0000","last_peered":"2024-09-05T12:25:52.346562+0000","last_clean":"2024-09-05T12:25:52.346562+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:25:52.346562+0000","last_undegraded":"2024-09-05T12:25:52.346562+0000","last_fullsized":"2024-09-05T12:25:52.346562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330666+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:25:34.330666+0000","last_peered":"2024-09-05T12:25:34.330666+0000","last_clean":"2024-09-05T12:25:34.330666+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:25:34.330666+0000","last_undegraded":"2024-09-05T12:25:34.330666+0000","last_fullsized":"2024-09-05T12:25:34.330666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268645+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:25:34.268645+0000","last_peered":"2024-09-05T12:25:34.268645+0000","last_clean":"2024-09-05T12:25:34.268645+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:25:34.268645+0000","last_undegraded":"2024-09-05T12:25:34.268645+0000","last_fullsized":"2024-09-05T12:25:34.268645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322440+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:25:38.322440+0000","last_peered":"2024-09-05T12:25:38.322440+0000","last_clean":"2024-09-05T12:25:38.322440+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:25:38.322440+0000","last_undegraded":"2024-09-05T12:25:38.322440+0000","last_fullsized":"2024-09-05T12:25:38.322440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271674+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:25:34.271674+0000","last_peered":"2024-09-05T12:25:34.271674+0000","last_clean":"2024-09-05T12:25:34.271674+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:25:34.271674+0000","last_undegraded":"2024-09-05T12:25:34.271674+0000","last_fullsized":"2024-09-05T12:25:34.271674+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266874+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:25:34.266874+0000","last_peered":"2024-09-05T12:25:34.266874+0000","last_clean":"2024-09-05T12:25:34.266874+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:25:34.266874+0000","last_undegraded":"2024-09-05T12:25:34.266874+0000","last_fullsized":"2024-09-05T12:25:34.266874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.326756+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:25:39.326756+0000","last_peered":"2024-09-05T12:25:39.326756+0000","last_clean":"2024-09-05T12:25:39.326756+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:25:39.326756+0000","last_undegraded":"2024-09-05T12:25:39.326756+0000","last_fullsized":"2024-09-05T12:25:39.326756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269780+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:25:34.269780+0000","last_peered":"2024-09-05T12:25:34.269780+0000","last_clean":"2024-09-05T12:25:34.269780+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:25:34.269780+0000","last_undegraded":"2024-09-05T12:25:34.269780+0000","last_fullsized":"2024-09-05T12:25:34.269780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266924+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:25:34.266924+0000","last_peered":"2024-09-05T12:25:34.266924+0000","last_clean":"2024-09-05T12:25:34.266924+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:25:34.266924+0000","last_undegraded":"2024-09-05T12:25:34.266924+0000","last_fullsized":"2024-09-05T12:25:34.266924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'31","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:53.350292+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:25:53.350292+0000","last_peered":"2024-09-05T12:25:53.350292+0000","last_clean":"2024-09-05T12:25:53.350292+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:25:53.350292+0000","last_undegraded":"2024-09-05T12:25:53.350292+0000","last_fullsized":"2024-09-05T12:25:53.350292+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52413,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":69,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330067+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:25:34.330067+0000","last_peered":"2024-09-05T12:25:34.330067+0000","last_clean":"2024-09-05T12:25:34.330067+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:25:34.330067+0000","last_undegraded":"2024-09-05T12:25:34.330067+0000","last_fullsized":"2024-09-05T12:25:34.330067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404529+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:25:34.404529+0000","last_peered":"2024-09-05T12:25:34.404529+0000","last_clean":"2024-09-05T12:25:34.404529+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:25:34.404529+0000","last_undegraded":"2024-09-05T12:25:34.404529+0000","last_fullsized":"2024-09-05T12:25:34.404529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313398+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:25:37.313398+0000","last_peered":"2024-09-05T12:25:37.313398+0000","last_clean":"2024-09-05T12:25:37.313398+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:25:37.313398+0000","last_undegraded":"2024-09-05T12:25:37.313398+0000","last_fullsized":"2024-09-05T12:25:37.313398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264502+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:25:34.264502+0000","last_peered":"2024-09-05T12:25:34.264502+0000","last_clean":"2024-09-05T12:25:34.264502+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:25:34.264502+0000","last_undegraded":"2024-09-05T12:25:34.264502+0000","last_fullsized":"2024-09-05T12:25:34.264502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268248+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:25:34.268248+0000","last_peered":"2024-09-05T12:25:34.268248+0000","last_clean":"2024-09-05T12:25:34.268248+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:25:34.268248+0000","last_undegraded":"2024-09-05T12:25:34.268248+0000","last_fullsized":"2024-09-05T12:25:34.268248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321510+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:25:38.321510+0000","last_peered":"2024-09-05T12:25:38.321510+0000","last_clean":"2024-09-05T12:25:38.321510+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:25:38.321510+0000","last_undegraded":"2024-09-05T12:25:38.321510+0000","last_fullsized":"2024-09-05T12:25:38.321510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272597+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:25:34.272597+0000","last_peered":"2024-09-05T12:25:34.272597+0000","last_clean":"2024-09-05T12:25:34.272597+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:25:34.272597+0000","last_undegraded":"2024-09-05T12:25:34.272597+0000","last_fullsized":"2024-09-05T12:25:34.272597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404720+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:25:34.404720+0000","last_peered":"2024-09-05T12:25:34.404720+0000","last_clean":"2024-09-05T12:25:34.404720+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:25:34.404720+0000","last_undegraded":"2024-09-05T12:25:34.404720+0000","last_fullsized":"2024-09-05T12:25:34.404720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:53.347692+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:25:53.347692+0000","last_peered":"2024-09-05T12:25:53.347692+0000","last_clean":"2024-09-05T12:25:53.347692+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:25:53.347692+0000","last_undegraded":"2024-09-05T12:25:53.347692+0000","last_fullsized":"2024-09-05T12:25:53.347692+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263017+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:25:34.263017+0000","last_peered":"2024-09-05T12:25:34.263017+0000","last_clean":"2024-09-05T12:25:34.263017+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:25:34.263017+0000","last_undegraded":"2024-09-05T12:25:34.263017+0000","last_fullsized":"2024-09-05T12:25:34.263017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268073+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:25:34.268073+0000","last_peered":"2024-09-05T12:25:34.268073+0000","last_clean":"2024-09-05T12:25:34.268073+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:25:34.268073+0000","last_undegraded":"2024-09-05T12:25:34.268073+0000","last_fullsized":"2024-09-05T12:25:34.268073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.329887+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:25:34.329887+0000","last_peered":"2024-09-05T12:25:34.329887+0000","last_clean":"2024-09-05T12:25:34.329887+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:25:34.329887+0000","last_undegraded":"2024-09-05T12:25:34.329887+0000","last_fullsized":"2024-09-05T12:25:34.329887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322579+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:25:38.322579+0000","last_peered":"2024-09-05T12:25:38.322579+0000","last_clean":"2024-09-05T12:25:38.322579+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:25:38.322579+0000","last_undegraded":"2024-09-05T12:25:38.322579+0000","last_fullsized":"2024-09-05T12:25:38.322579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267082+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:25:34.267082+0000","last_peered":"2024-09-05T12:25:34.267082+0000","last_clean":"2024-09-05T12:25:34.267082+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:25:34.267082+0000","last_undegraded":"2024-09-05T12:25:34.267082+0000","last_fullsized":"2024-09-05T12:25:34.267082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263148+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:25:34.263148+0000","last_peered":"2024-09-05T12:25:34.263148+0000","last_clean":"2024-09-05T12:25:34.263148+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:25:34.263148+0000","last_undegraded":"2024-09-05T12:25:34.263148+0000","last_fullsized":"2024-09-05T12:25:34.263148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.302852+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:25:36.302852+0000","last_peered":"2024-09-05T12:25:36.302852+0000","last_clean":"2024-09-05T12:25:36.302852+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:25:36.302852+0000","last_undegraded":"2024-09-05T12:25:36.302852+0000","last_fullsized":"2024-09-05T12:25:36.302852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313373+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:25:37.313373+0000","last_peered":"2024-09-05T12:25:37.313373+0000","last_clean":"2024-09-05T12:25:37.313373+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:25:37.313373+0000","last_undegraded":"2024-09-05T12:25:37.313373+0000","last_fullsized":"2024-09-05T12:25:37.313373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269899+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:25:34.269899+0000","last_peered":"2024-09-05T12:25:34.269899+0000","last_clean":"2024-09-05T12:25:34.269899+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:25:34.269899+0000","last_undegraded":"2024-09-05T12:25:34.269899+0000","last_fullsized":"2024-09-05T12:25:34.269899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264384+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:25:34.264384+0000","last_peered":"2024-09-05T12:25:34.264384+0000","last_clean":"2024-09-05T12:25:34.264384+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:25:34.264384+0000","last_undegraded":"2024-09-05T12:25:34.264384+0000","last_fullsized":"2024-09-05T12:25:34.264384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:55.350848+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:25:55.350848+0000","last_peered":"2024-09-05T12:25:55.350848+0000","last_clean":"2024-09-05T12:25:55.350848+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:25:55.350848+0000","last_undegraded":"2024-09-05T12:25:55.350848+0000","last_fullsized":"2024-09-05T12:25:55.350848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266460+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:25:34.266460+0000","last_peered":"2024-09-05T12:25:34.266460+0000","last_clean":"2024-09-05T12:25:34.266460+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:25:34.266460+0000","last_undegraded":"2024-09-05T12:25:34.266460+0000","last_fullsized":"2024-09-05T12:25:34.266460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264349+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:25:34.264349+0000","last_peered":"2024-09-05T12:25:34.264349+0000","last_clean":"2024-09-05T12:25:34.264349+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:25:34.264349+0000","last_undegraded":"2024-09-05T12:25:34.264349+0000","last_fullsized":"2024-09-05T12:25:34.264349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265242+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:25:34.265242+0000","last_peered":"2024-09-05T12:25:34.265242+0000","last_clean":"2024-09-05T12:25:34.265242+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:25:34.265242+0000","last_undegraded":"2024-09-05T12:25:34.265242+0000","last_fullsized":"2024-09-05T12:25:34.265242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264119+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:25:34.264119+0000","last_peered":"2024-09-05T12:25:34.264119+0000","last_clean":"2024-09-05T12:25:34.264119+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:25:34.264119+0000","last_undegraded":"2024-09-05T12:25:34.264119+0000","last_fullsized":"2024-09-05T12:25:34.264119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263751+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:25:34.263751+0000","last_peered":"2024-09-05T12:25:34.263751+0000","last_clean":"2024-09-05T12:25:34.263751+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:25:34.263751+0000","last_undegraded":"2024-09-05T12:25:34.263751+0000","last_fullsized":"2024-09-05T12:25:34.263751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270400+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:25:34.270400+0000","last_peered":"2024-09-05T12:25:34.270400+0000","last_clean":"2024-09-05T12:25:34.270400+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:25:34.270400+0000","last_undegraded":"2024-09-05T12:25:34.270400+0000","last_fullsized":"2024-09-05T12:25:34.270400+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331799+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:25:34.331799+0000","last_peered":"2024-09-05T12:25:34.331799+0000","last_clean":"2024-09-05T12:25:34.331799+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:25:34.331799+0000","last_undegraded":"2024-09-05T12:25:34.331799+0000","last_fullsized":"2024-09-05T12:25:34.331799+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266950+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:25:34.266950+0000","last_peered":"2024-09-05T12:25:34.266950+0000","last_clean":"2024-09-05T12:25:34.266950+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:25:34.266950+0000","last_undegraded":"2024-09-05T12:25:34.266950+0000","last_fullsized":"2024-09-05T12:25:34.266950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264875+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:25:34.264875+0000","last_peered":"2024-09-05T12:25:34.264875+0000","last_clean":"2024-09-05T12:25:34.264875+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:25:34.264875+0000","last_undegraded":"2024-09-05T12:25:34.264875+0000","last_fullsized":"2024-09-05T12:25:34.264875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263394+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:25:34.263394+0000","last_peered":"2024-09-05T12:25:34.263394+0000","last_clean":"2024-09-05T12:25:34.263394+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:25:34.263394+0000","last_undegraded":"2024-09-05T12:25:34.263394+0000","last_fullsized":"2024-09-05T12:25:34.263394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269818+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:25:34.269818+0000","last_peered":"2024-09-05T12:25:34.269818+0000","last_clean":"2024-09-05T12:25:34.269818+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:25:34.269818+0000","last_undegraded":"2024-09-05T12:25:34.269818+0000","last_fullsized":"2024-09-05T12:25:34.269818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263113+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:25:34.263113+0000","last_peered":"2024-09-05T12:25:34.263113+0000","last_clean":"2024-09-05T12:25:34.263113+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:25:34.263113+0000","last_undegraded":"2024-09-05T12:25:34.263113+0000","last_fullsized":"2024-09-05T12:25:34.263113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404441+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:25:34.404441+0000","last_peered":"2024-09-05T12:25:34.404441+0000","last_clean":"2024-09-05T12:25:34.404441+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:25:34.404441+0000","last_undegraded":"2024-09-05T12:25:34.404441+0000","last_fullsized":"2024-09-05T12:25:34.404441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267531+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:25:34.267531+0000","last_peered":"2024-09-05T12:25:34.267531+0000","last_clean":"2024-09-05T12:25:34.267531+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:25:34.267531+0000","last_undegraded":"2024-09-05T12:25:34.267531+0000","last_fullsized":"2024-09-05T12:25:34.267531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266222+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:25:34.266222+0000","last_peered":"2024-09-05T12:25:34.266222+0000","last_clean":"2024-09-05T12:25:34.266222+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:25:34.266222+0000","last_undegraded":"2024-09-05T12:25:34.266222+0000","last_fullsized":"2024-09-05T12:25:34.266222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264463+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:25:34.264463+0000","last_peered":"2024-09-05T12:25:34.264463+0000","last_clean":"2024-09-05T12:25:34.264463+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:25:34.264463+0000","last_undegraded":"2024-09-05T12:25:34.264463+0000","last_fullsized":"2024-09-05T12:25:34.264463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330539+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:25:34.330539+0000","last_peered":"2024-09-05T12:25:34.330539+0000","last_clean":"2024-09-05T12:25:34.330539+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:25:34.330539+0000","last_undegraded":"2024-09-05T12:25:34.330539+0000","last_fullsized":"2024-09-05T12:25:34.330539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267312+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:25:34.267312+0000","last_peered":"2024-09-05T12:25:34.267312+0000","last_clean":"2024-09-05T12:25:34.267312+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:25:34.267312+0000","last_undegraded":"2024-09-05T12:25:34.267312+0000","last_fullsized":"2024-09-05T12:25:34.267312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404202+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:25:34.404202+0000","last_peered":"2024-09-05T12:25:34.404202+0000","last_clean":"2024-09-05T12:25:34.404202+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:25:34.404202+0000","last_undegraded":"2024-09-05T12:25:34.404202+0000","last_fullsized":"2024-09-05T12:25:34.404202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266403+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:25:34.266403+0000","last_peered":"2024-09-05T12:25:34.266403+0000","last_clean":"2024-09-05T12:25:34.266403+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:25:34.266403+0000","last_undegraded":"2024-09-05T12:25:34.266403+0000","last_fullsized":"2024-09-05T12:25:34.266403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266842+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:25:34.266842+0000","last_peered":"2024-09-05T12:25:34.266842+0000","last_clean":"2024-09-05T12:25:34.266842+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:25:34.266842+0000","last_undegraded":"2024-09-05T12:25:34.266842+0000","last_fullsized":"2024-09-05T12:25:34.266842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267435+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:25:34.267435+0000","last_peered":"2024-09-05T12:25:34.267435+0000","last_clean":"2024-09-05T12:25:34.267435+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:25:34.267435+0000","last_undegraded":"2024-09-05T12:25:34.267435+0000","last_fullsized":"2024-09-05T12:25:34.267435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265181+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:25:34.265181+0000","last_peered":"2024-09-05T12:25:34.265181+0000","last_clean":"2024-09-05T12:25:34.265181+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:25:34.265181+0000","last_undegraded":"2024-09-05T12:25:34.265181+0000","last_fullsized":"2024-09-05T12:25:34.265181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268910+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:25:34.268910+0000","last_peered":"2024-09-05T12:25:34.268910+0000","last_clean":"2024-09-05T12:25:34.268910+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:25:34.268910+0000","last_undegraded":"2024-09-05T12:25:34.268910+0000","last_fullsized":"2024-09-05T12:25:34.268910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331174+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:25:34.331174+0000","last_peered":"2024-09-05T12:25:34.331174+0000","last_clean":"2024-09-05T12:25:34.331174+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:25:34.331174+0000","last_undegraded":"2024-09-05T12:25:34.331174+0000","last_fullsized":"2024-09-05T12:25:34.331174+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404780+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:25:34.404780+0000","last_peered":"2024-09-05T12:25:34.404780+0000","last_clean":"2024-09-05T12:25:34.404780+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:25:34.404780+0000","last_undegraded":"2024-09-05T12:25:34.404780+0000","last_fullsized":"2024-09-05T12:25:34.404780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263044+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:25:34.263044+0000","last_peered":"2024-09-05T12:25:34.263044+0000","last_clean":"2024-09-05T12:25:34.263044+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:25:34.263044+0000","last_undegraded":"2024-09-05T12:25:34.263044+0000","last_fullsized":"2024-09-05T12:25:34.263044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268375+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:25:34.268375+0000","last_peered":"2024-09-05T12:25:34.268375+0000","last_clean":"2024-09-05T12:25:34.268375+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:25:34.268375+0000","last_undegraded":"2024-09-05T12:25:34.268375+0000","last_fullsized":"2024-09-05T12:25:34.268375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267499+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:25:34.267499+0000","last_peered":"2024-09-05T12:25:34.267499+0000","last_clean":"2024-09-05T12:25:34.267499+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:25:34.267499+0000","last_undegraded":"2024-09-05T12:25:34.267499+0000","last_fullsized":"2024-09-05T12:25:34.267499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265767+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:25:34.265767+0000","last_peered":"2024-09-05T12:25:34.265767+0000","last_clean":"2024-09-05T12:25:34.265767+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:25:34.265767+0000","last_undegraded":"2024-09-05T12:25:34.265767+0000","last_fullsized":"2024-09-05T12:25:34.265767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265061+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:25:34.265061+0000","last_peered":"2024-09-05T12:25:34.265061+0000","last_clean":"2024-09-05T12:25:34.265061+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:25:34.265061+0000","last_undegraded":"2024-09-05T12:25:34.265061+0000","last_fullsized":"2024-09-05T12:25:34.265061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267876+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:25:34.267876+0000","last_peered":"2024-09-05T12:25:34.267876+0000","last_clean":"2024-09-05T12:25:34.267876+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:25:34.267876+0000","last_undegraded":"2024-09-05T12:25:34.267876+0000","last_fullsized":"2024-09-05T12:25:34.267876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268171+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:25:34.268171+0000","last_peered":"2024-09-05T12:25:34.268171+0000","last_clean":"2024-09-05T12:25:34.268171+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:25:34.268171+0000","last_undegraded":"2024-09-05T12:25:34.268171+0000","last_fullsized":"2024-09-05T12:25:34.268171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267268+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:25:34.267268+0000","last_peered":"2024-09-05T12:25:34.267268+0000","last_clean":"2024-09-05T12:25:34.267268+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:25:34.267268+0000","last_undegraded":"2024-09-05T12:25:34.267268+0000","last_fullsized":"2024-09-05T12:25:34.267268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59228,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":190,"num_read_kb":160,"num_write":239,"num_write_kb":128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":177684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":533938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678047,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28752,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716400,"statfs":{"total":95995035648,"available":95965593600,"internally_reserved":0,"allocated":1769472,"data_stored":1176848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841572,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":533453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":533809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1699840,"data_stored":1129937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":528957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1708032,"data_stored":1125519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":533031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182468,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":528665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1159168,"data_stored":583013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":582400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":530806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:25:58.161 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:25:58.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:58 smithi174 ceph-mon[34313]: pgmap v336: 129 pgs: 129 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-05T12:25:58.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:58 smithi071 ceph-mon[29845]: pgmap v336: 129 pgs: 129 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-05T12:25:58.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:58 smithi123 ceph-mon[34321]: pgmap v336: 129 pgs: 129 active+clean; 635 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-09-05T12:25:58.530 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 -- 172.21.15.71:0/4085743798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 msgr2=0x7f6b20108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 --2- 172.21.15.71:0/4085743798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 0x7f6b20108a10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 -- 172.21.15.71:0/4085743798 shutdown_connections 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 --2- 172.21.15.71:0/4085743798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20109140 0x7f6b20110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 --2- 172.21.15.71:0/4085743798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 0x7f6b20108a10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 --2- 172.21.15.71:0/4085743798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b20104010 0x7f6b201043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.169+0000 7f6b26e00640 1 -- 172.21.15.71:0/4085743798 >> 172.21.15.71:0/4085743798 conn(0x7f6b200ffa50 msgr2=0x7f6b20101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.170+0000 7f6b26e00640 1 -- 172.21.15.71:0/4085743798 shutdown_connections 2024-09-05T12:25:59.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.170+0000 7f6b26e00640 1 -- 172.21.15.71:0/4085743798 wait complete. 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.170+0000 7f6b26e00640 1 Processor -- start 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 -- start start 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 0x7f6b2019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 0x7f6b2019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b20109140 0x7f6b2019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20114190 con 0x7f6b201049c0 2024-09-05T12:25:59.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20114010 con 0x7f6b20104010 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b26e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b20114310 con 0x7f6b20109140 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 0x7f6b2019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b1fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 0x7f6b2019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58294/0 (socket says 172.21.15.71:58294) 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b24e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b20109140 0x7f6b2019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b1fe00640 1 -- 172.21.15.71:0/1299037567 learned_addr learned my addr 172.21.15.71:0/1299037567 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.171+0000 7f6b1f400640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 0x7f6b2019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1fe00640 1 -- 172.21.15.71:0/1299037567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b20109140 msgr2=0x7f6b2019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1fe00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b20109140 0x7f6b2019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1fe00640 1 -- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 msgr2=0x7f6b2019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1fe00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 0x7f6b2019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1fe00640 1 -- 172.21.15.71:0/1299037567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2019f5e0 con 0x7f6b20104010 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1fe00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 0x7f6b2019a630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000d950 tx=0x7f6b1000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:59.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b10017070 con 0x7f6b20104010 2024-09-05T12:25:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b10009c30 con 0x7f6b20104010 2024-09-05T12:25:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.172+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1000f6b0 con 0x7f6b20104010 2024-09-05T12:25:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.173+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2019f8d0 con 0x7f6b20104010 2024-09-05T12:25:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.173+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b201a71b0 con 0x7f6b20104010 2024-09-05T12:25:59.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.174+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b201043f0 con 0x7f6b20104010 2024-09-05T12:25:59.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.176+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6b100109d0 con 0x7f6b20104010 2024-09-05T12:25:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.177+0000 7f6b1cc00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b040a2c10 0x7f6b040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:25:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.177+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f6b100cf110 con 0x7f6b20104010 2024-09-05T12:25:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.177+0000 7f6b1f400640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b040a2c10 0x7f6b040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:25:59.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.178+0000 7f6b1f400640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b040a2c10 0x7f6b040a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c00 tx=0x7f6b0c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:25:59.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.179+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b10009de0 con 0x7f6b20104010 2024-09-05T12:25:59.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.354+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b2010c3f0 con 0x7f6b040a2c10 2024-09-05T12:25:59.356 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:25:59 smithi071 ceph-mon[29845]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:25:59.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.363+0000 7f6b1cc00640 1 -- 172.21.15.71:0/1299037567 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340417 (secure 0 0 0) 0x7f6b0c004270 con 0x7f6b040a2c10 2024-09-05T12:25:59.368 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:25:59.370 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:25:59.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b040a2c10 msgr2=0x7f6b040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:59.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b040a2c10 0x7f6b040a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c00 tx=0x7f6b0c005b50 comp rx=0 tx=0).stop 2024-09-05T12:25:59.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 msgr2=0x7f6b2019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:25:59.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 0x7f6b2019a630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6b1000d950 tx=0x7f6b1000de10 comp rx=0 tx=0).stop 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 shutdown_connections 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f6b040a2c10 0x7f6b040a50d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b20109140 0x7f6b2019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b201049c0 0x7f6b2019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 --2- 172.21.15.71:0/1299037567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20104010 0x7f6b2019a630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.367+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 >> 172.21.15.71:0/1299037567 conn(0x7f6b200ffa50 msgr2=0x7f6b20100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.368+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 shutdown_connections 2024-09-05T12:25:59.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:25:59.368+0000 7f6b26e00640 1 -- 172.21.15.71:0/1299037567 wait complete. 2024-09-05T12:25:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:25:59 smithi174 ceph-mon[34313]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:25:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:25:59 smithi123 ceph-mon[34321]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:00.246 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-05T12:25:59.152754+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654095,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":357,"num_read_kb":304,"num_write":427,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376516,"kb_used_data":14788,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565308,"statfs":{"total":1151940427776,"available":1151554875392,"internally_reserved":0,"allocated":15142912,"data_stored":8328906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":46466,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":30,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002936"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":71,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:57.352470+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:25:57.352470+0000","last_peered":"2024-09-05T12:25:57.352470+0000","last_clean":"2024-09-05T12:25:57.352470+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:25:57.352470+0000","last_undegraded":"2024-09-05T12:25:57.352470+0000","last_fullsized":"2024-09-05T12:25:57.352470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263356+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:25:34.263356+0000","last_peered":"2024-09-05T12:25:34.263356+0000","last_clean":"2024-09-05T12:25:34.263356+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:25:34.263356+0000","last_undegraded":"2024-09-05T12:25:34.263356+0000","last_fullsized":"2024-09-05T12:25:34.263356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263685+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:25:34.263685+0000","last_peered":"2024-09-05T12:25:34.263685+0000","last_clean":"2024-09-05T12:25:34.263685+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:25:34.263685+0000","last_undegraded":"2024-09-05T12:25:34.263685+0000","last_fullsized":"2024-09-05T12:25:34.263685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263635+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:25:34.263635+0000","last_peered":"2024-09-05T12:25:34.263635+0000","last_clean":"2024-09-05T12:25:34.263635+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:25:34.263635+0000","last_undegraded":"2024-09-05T12:25:34.263635+0000","last_fullsized":"2024-09-05T12:25:34.263635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268221+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:25:34.268221+0000","last_peered":"2024-09-05T12:25:34.268221+0000","last_clean":"2024-09-05T12:25:34.268221+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:25:34.268221+0000","last_undegraded":"2024-09-05T12:25:34.268221+0000","last_fullsized":"2024-09-05T12:25:34.268221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264135+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:25:34.264135+0000","last_peered":"2024-09-05T12:25:34.264135+0000","last_clean":"2024-09-05T12:25:34.264135+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:25:34.264135+0000","last_undegraded":"2024-09-05T12:25:34.264135+0000","last_fullsized":"2024-09-05T12:25:34.264135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.317765+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:25:39.317765+0000","last_peered":"2024-09-05T12:25:39.317765+0000","last_clean":"2024-09-05T12:25:39.317765+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:25:39.317765+0000","last_undegraded":"2024-09-05T12:25:39.317765+0000","last_fullsized":"2024-09-05T12:25:39.317765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404504+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:25:34.404504+0000","last_peered":"2024-09-05T12:25:34.404504+0000","last_clean":"2024-09-05T12:25:34.404504+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:25:34.404504+0000","last_undegraded":"2024-09-05T12:25:34.404504+0000","last_fullsized":"2024-09-05T12:25:34.404504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264558+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:25:34.264558+0000","last_peered":"2024-09-05T12:25:34.264558+0000","last_clean":"2024-09-05T12:25:34.264558+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:25:34.264558+0000","last_undegraded":"2024-09-05T12:25:34.264558+0000","last_fullsized":"2024-09-05T12:25:34.264558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322603+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:25:38.322603+0000","last_peered":"2024-09-05T12:25:38.322603+0000","last_clean":"2024-09-05T12:25:38.322603+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:25:38.322603+0000","last_undegraded":"2024-09-05T12:25:38.322603+0000","last_fullsized":"2024-09-05T12:25:38.322603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271656+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:25:34.271656+0000","last_peered":"2024-09-05T12:25:34.271656+0000","last_clean":"2024-09-05T12:25:34.271656+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:25:34.271656+0000","last_undegraded":"2024-09-05T12:25:34.271656+0000","last_fullsized":"2024-09-05T12:25:34.271656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264243+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:25:34.264243+0000","last_peered":"2024-09-05T12:25:34.264243+0000","last_clean":"2024-09-05T12:25:34.264243+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:25:34.264243+0000","last_undegraded":"2024-09-05T12:25:34.264243+0000","last_fullsized":"2024-09-05T12:25:34.264243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268524+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:25:34.268524+0000","last_peered":"2024-09-05T12:25:34.268524+0000","last_clean":"2024-09-05T12:25:34.268524+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:25:34.268524+0000","last_undegraded":"2024-09-05T12:25:34.268524+0000","last_fullsized":"2024-09-05T12:25:34.268524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.294392+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:25:35.294392+0000","last_peered":"2024-09-05T12:25:35.294392+0000","last_clean":"2024-09-05T12:25:35.294392+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:25:35.294392+0000","last_undegraded":"2024-09-05T12:25:35.294392+0000","last_fullsized":"2024-09-05T12:25:35.294392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264525+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:25:34.264525+0000","last_peered":"2024-09-05T12:25:34.264525+0000","last_clean":"2024-09-05T12:25:34.264525+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:25:34.264525+0000","last_undegraded":"2024-09-05T12:25:34.264525+0000","last_fullsized":"2024-09-05T12:25:34.264525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263138+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:25:34.263138+0000","last_peered":"2024-09-05T12:25:34.263138+0000","last_clean":"2024-09-05T12:25:34.263138+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:25:34.263138+0000","last_undegraded":"2024-09-05T12:25:34.263138+0000","last_fullsized":"2024-09-05T12:25:34.263138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313403+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:25:37.313403+0000","last_peered":"2024-09-05T12:25:37.313403+0000","last_clean":"2024-09-05T12:25:37.313403+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:25:37.313403+0000","last_undegraded":"2024-09-05T12:25:37.313403+0000","last_fullsized":"2024-09-05T12:25:37.313403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271242+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:25:34.271242+0000","last_peered":"2024-09-05T12:25:34.271242+0000","last_clean":"2024-09-05T12:25:34.271242+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:25:34.271242+0000","last_undegraded":"2024-09-05T12:25:34.271242+0000","last_fullsized":"2024-09-05T12:25:34.271242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265378+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:25:34.265378+0000","last_peered":"2024-09-05T12:25:34.265378+0000","last_clean":"2024-09-05T12:25:34.265378+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:25:34.265378+0000","last_undegraded":"2024-09-05T12:25:34.265378+0000","last_fullsized":"2024-09-05T12:25:34.265378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323200+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:25:38.323200+0000","last_peered":"2024-09-05T12:25:38.323200+0000","last_clean":"2024-09-05T12:25:38.323200+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:25:38.323200+0000","last_undegraded":"2024-09-05T12:25:38.323200+0000","last_fullsized":"2024-09-05T12:25:38.323200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330571+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:25:34.330571+0000","last_peered":"2024-09-05T12:25:34.330571+0000","last_clean":"2024-09-05T12:25:34.330571+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:25:34.330571+0000","last_undegraded":"2024-09-05T12:25:34.330571+0000","last_fullsized":"2024-09-05T12:25:34.330571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265675+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:25:34.265675+0000","last_peered":"2024-09-05T12:25:34.265675+0000","last_clean":"2024-09-05T12:25:34.265675+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:25:34.265675+0000","last_undegraded":"2024-09-05T12:25:34.265675+0000","last_fullsized":"2024-09-05T12:25:34.265675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.293537+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:25:35.293537+0000","last_peered":"2024-09-05T12:25:35.293537+0000","last_clean":"2024-09-05T12:25:35.293537+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:25:35.293537+0000","last_undegraded":"2024-09-05T12:25:35.293537+0000","last_fullsized":"2024-09-05T12:25:35.293537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264519+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:25:34.264519+0000","last_peered":"2024-09-05T12:25:34.264519+0000","last_clean":"2024-09-05T12:25:34.264519+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:25:34.264519+0000","last_undegraded":"2024-09-05T12:25:34.264519+0000","last_fullsized":"2024-09-05T12:25:34.264519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268273+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:25:34.268273+0000","last_peered":"2024-09-05T12:25:34.268273+0000","last_clean":"2024-09-05T12:25:34.268273+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:25:34.268273+0000","last_undegraded":"2024-09-05T12:25:34.268273+0000","last_fullsized":"2024-09-05T12:25:34.268273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323273+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:25:38.323273+0000","last_peered":"2024-09-05T12:25:38.323273+0000","last_clean":"2024-09-05T12:25:38.323273+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:25:38.323273+0000","last_undegraded":"2024-09-05T12:25:38.323273+0000","last_fullsized":"2024-09-05T12:25:38.323273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263934+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:25:34.263934+0000","last_peered":"2024-09-05T12:25:34.263934+0000","last_clean":"2024-09-05T12:25:34.263934+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:25:34.263934+0000","last_undegraded":"2024-09-05T12:25:34.263934+0000","last_fullsized":"2024-09-05T12:25:34.263934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263632+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:25:34.263632+0000","last_peered":"2024-09-05T12:25:34.263632+0000","last_clean":"2024-09-05T12:25:34.263632+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:25:34.263632+0000","last_undegraded":"2024-09-05T12:25:34.263632+0000","last_fullsized":"2024-09-05T12:25:34.263632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'14","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:54.522676+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:25:54.522676+0000","last_peered":"2024-09-05T12:25:54.522676+0000","last_clean":"2024-09-05T12:25:54.522676+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:25:54.522676+0000","last_undegraded":"2024-09-05T12:25:54.522676+0000","last_fullsized":"2024-09-05T12:25:54.522676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.262906+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:25:34.262906+0000","last_peered":"2024-09-05T12:25:34.262906+0000","last_clean":"2024-09-05T12:25:34.262906+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:25:34.262906+0000","last_undegraded":"2024-09-05T12:25:34.262906+0000","last_fullsized":"2024-09-05T12:25:34.262906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267595+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:25:34.267595+0000","last_peered":"2024-09-05T12:25:34.267595+0000","last_clean":"2024-09-05T12:25:34.267595+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:25:34.267595+0000","last_undegraded":"2024-09-05T12:25:34.267595+0000","last_fullsized":"2024-09-05T12:25:34.267595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323069+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:25:38.323069+0000","last_peered":"2024-09-05T12:25:38.323069+0000","last_clean":"2024-09-05T12:25:38.323069+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:25:38.323069+0000","last_undegraded":"2024-09-05T12:25:38.323069+0000","last_fullsized":"2024-09-05T12:25:38.323069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265409+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:25:34.265409+0000","last_peered":"2024-09-05T12:25:34.265409+0000","last_clean":"2024-09-05T12:25:34.265409+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:25:34.265409+0000","last_undegraded":"2024-09-05T12:25:34.265409+0000","last_fullsized":"2024-09-05T12:25:34.265409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267791+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:25:34.267791+0000","last_peered":"2024-09-05T12:25:34.267791+0000","last_clean":"2024-09-05T12:25:34.267791+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:25:34.267791+0000","last_undegraded":"2024-09-05T12:25:34.267791+0000","last_fullsized":"2024-09-05T12:25:34.267791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330392+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:25:34.330392+0000","last_peered":"2024-09-05T12:25:34.330392+0000","last_clean":"2024-09-05T12:25:34.330392+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:25:34.330392+0000","last_undegraded":"2024-09-05T12:25:34.330392+0000","last_fullsized":"2024-09-05T12:25:34.330392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266261+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:25:34.266261+0000","last_peered":"2024-09-05T12:25:34.266261+0000","last_clean":"2024-09-05T12:25:34.266261+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:25:34.266261+0000","last_undegraded":"2024-09-05T12:25:34.266261+0000","last_fullsized":"2024-09-05T12:25:34.266261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267674+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:25:34.267674+0000","last_peered":"2024-09-05T12:25:34.267674+0000","last_clean":"2024-09-05T12:25:34.267674+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:25:34.267674+0000","last_undegraded":"2024-09-05T12:25:34.267674+0000","last_fullsized":"2024-09-05T12:25:34.267674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":58,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346549+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:25:52.346549+0000","last_peered":"2024-09-05T12:25:52.346549+0000","last_clean":"2024-09-05T12:25:52.346549+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:25:52.346549+0000","last_undegraded":"2024-09-05T12:25:52.346549+0000","last_fullsized":"2024-09-05T12:25:52.346549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267415+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:25:34.267415+0000","last_peered":"2024-09-05T12:25:34.267415+0000","last_clean":"2024-09-05T12:25:34.267415+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:25:34.267415+0000","last_undegraded":"2024-09-05T12:25:34.267415+0000","last_fullsized":"2024-09-05T12:25:34.267415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269430+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:25:34.269430+0000","last_peered":"2024-09-05T12:25:34.269430+0000","last_clean":"2024-09-05T12:25:34.269430+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:25:34.269430+0000","last_undegraded":"2024-09-05T12:25:34.269430+0000","last_fullsized":"2024-09-05T12:25:34.269430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":59,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346062+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:25:52.346062+0000","last_peered":"2024-09-05T12:25:52.346062+0000","last_clean":"2024-09-05T12:25:52.346062+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:25:52.346062+0000","last_undegraded":"2024-09-05T12:25:52.346062+0000","last_fullsized":"2024-09-05T12:25:52.346062+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271660+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:25:34.271660+0000","last_peered":"2024-09-05T12:25:34.271660+0000","last_clean":"2024-09-05T12:25:34.271660+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:25:34.271660+0000","last_undegraded":"2024-09-05T12:25:34.271660+0000","last_fullsized":"2024-09-05T12:25:34.271660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266417+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:25:34.266417+0000","last_peered":"2024-09-05T12:25:34.266417+0000","last_clean":"2024-09-05T12:25:34.266417+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:25:34.266417+0000","last_undegraded":"2024-09-05T12:25:34.266417+0000","last_fullsized":"2024-09-05T12:25:34.266417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.299617+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:25:36.299617+0000","last_peered":"2024-09-05T12:25:36.299617+0000","last_clean":"2024-09-05T12:25:36.299617+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:25:36.299617+0000","last_undegraded":"2024-09-05T12:25:36.299617+0000","last_fullsized":"2024-09-05T12:25:36.299617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272425+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:25:34.272425+0000","last_peered":"2024-09-05T12:25:34.272425+0000","last_clean":"2024-09-05T12:25:34.272425+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:25:34.272425+0000","last_undegraded":"2024-09-05T12:25:34.272425+0000","last_fullsized":"2024-09-05T12:25:34.272425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263271+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:25:34.263271+0000","last_peered":"2024-09-05T12:25:34.263271+0000","last_clean":"2024-09-05T12:25:34.263271+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:25:34.263271+0000","last_undegraded":"2024-09-05T12:25:34.263271+0000","last_fullsized":"2024-09-05T12:25:34.263271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.325619+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:25:39.325619+0000","last_peered":"2024-09-05T12:25:39.325619+0000","last_clean":"2024-09-05T12:25:39.325619+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:25:39.325619+0000","last_undegraded":"2024-09-05T12:25:39.325619+0000","last_fullsized":"2024-09-05T12:25:39.325619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269719+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:25:34.269719+0000","last_peered":"2024-09-05T12:25:34.269719+0000","last_clean":"2024-09-05T12:25:34.269719+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:25:34.269719+0000","last_undegraded":"2024-09-05T12:25:34.269719+0000","last_fullsized":"2024-09-05T12:25:34.269719+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268797+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:25:34.268797+0000","last_peered":"2024-09-05T12:25:34.268797+0000","last_clean":"2024-09-05T12:25:34.268797+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:25:34.268797+0000","last_undegraded":"2024-09-05T12:25:34.268797+0000","last_fullsized":"2024-09-05T12:25:34.268797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269019+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:25:34.269019+0000","last_peered":"2024-09-05T12:25:34.269019+0000","last_clean":"2024-09-05T12:25:34.269019+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:25:34.269019+0000","last_undegraded":"2024-09-05T12:25:34.269019+0000","last_fullsized":"2024-09-05T12:25:34.269019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'6","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:56.352438+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:25:56.352438+0000","last_peered":"2024-09-05T12:25:56.352438+0000","last_clean":"2024-09-05T12:25:56.352438+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:25:56.352438+0000","last_undegraded":"2024-09-05T12:25:56.352438+0000","last_fullsized":"2024-09-05T12:25:56.352438+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404323+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:25:34.404323+0000","last_peered":"2024-09-05T12:25:34.404323+0000","last_clean":"2024-09-05T12:25:34.404323+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:25:34.404323+0000","last_undegraded":"2024-09-05T12:25:34.404323+0000","last_fullsized":"2024-09-05T12:25:34.404323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269554+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:25:34.269554+0000","last_peered":"2024-09-05T12:25:34.269554+0000","last_clean":"2024-09-05T12:25:34.269554+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:25:34.269554+0000","last_undegraded":"2024-09-05T12:25:34.269554+0000","last_fullsized":"2024-09-05T12:25:34.269554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321412+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:25:38.321412+0000","last_peered":"2024-09-05T12:25:38.321412+0000","last_clean":"2024-09-05T12:25:38.321412+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:25:38.321412+0000","last_undegraded":"2024-09-05T12:25:38.321412+0000","last_fullsized":"2024-09-05T12:25:38.321412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267091+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:25:34.267091+0000","last_peered":"2024-09-05T12:25:34.267091+0000","last_clean":"2024-09-05T12:25:34.267091+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:25:34.267091+0000","last_undegraded":"2024-09-05T12:25:34.267091+0000","last_fullsized":"2024-09-05T12:25:34.267091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268658+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:25:34.268658+0000","last_peered":"2024-09-05T12:25:34.268658+0000","last_clean":"2024-09-05T12:25:34.268658+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:25:34.268658+0000","last_undegraded":"2024-09-05T12:25:34.268658+0000","last_fullsized":"2024-09-05T12:25:34.268658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323388+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:25:38.323388+0000","last_peered":"2024-09-05T12:25:38.323388+0000","last_clean":"2024-09-05T12:25:38.323388+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:25:38.323388+0000","last_undegraded":"2024-09-05T12:25:38.323388+0000","last_fullsized":"2024-09-05T12:25:38.323388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269057+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:25:34.269057+0000","last_peered":"2024-09-05T12:25:34.269057+0000","last_clean":"2024-09-05T12:25:34.269057+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:25:34.269057+0000","last_undegraded":"2024-09-05T12:25:34.269057+0000","last_fullsized":"2024-09-05T12:25:34.269057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266821+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:25:34.266821+0000","last_peered":"2024-09-05T12:25:34.266821+0000","last_clean":"2024-09-05T12:25:34.266821+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:25:34.266821+0000","last_undegraded":"2024-09-05T12:25:34.266821+0000","last_fullsized":"2024-09-05T12:25:34.266821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.296703+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:25:38.296703+0000","last_peered":"2024-09-05T12:25:38.296703+0000","last_clean":"2024-09-05T12:25:38.296703+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:25:38.296703+0000","last_undegraded":"2024-09-05T12:25:38.296703+0000","last_fullsized":"2024-09-05T12:25:38.296703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270206+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:25:34.270206+0000","last_peered":"2024-09-05T12:25:34.270206+0000","last_clean":"2024-09-05T12:25:34.270206+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:25:34.270206+0000","last_undegraded":"2024-09-05T12:25:34.270206+0000","last_fullsized":"2024-09-05T12:25:34.270206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270383+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:25:34.270383+0000","last_peered":"2024-09-05T12:25:34.270383+0000","last_clean":"2024-09-05T12:25:34.270383+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:25:34.270383+0000","last_undegraded":"2024-09-05T12:25:34.270383+0000","last_fullsized":"2024-09-05T12:25:34.270383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321005+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:25:38.321005+0000","last_peered":"2024-09-05T12:25:38.321005+0000","last_clean":"2024-09-05T12:25:38.321005+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:25:38.321005+0000","last_undegraded":"2024-09-05T12:25:38.321005+0000","last_fullsized":"2024-09-05T12:25:38.321005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267124+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:25:34.267124+0000","last_peered":"2024-09-05T12:25:34.267124+0000","last_clean":"2024-09-05T12:25:34.267124+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:25:34.267124+0000","last_undegraded":"2024-09-05T12:25:34.267124+0000","last_fullsized":"2024-09-05T12:25:34.267124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331448+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:25:34.331448+0000","last_peered":"2024-09-05T12:25:34.331448+0000","last_clean":"2024-09-05T12:25:34.331448+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:25:34.331448+0000","last_undegraded":"2024-09-05T12:25:34.331448+0000","last_fullsized":"2024-09-05T12:25:34.331448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":59,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:52.346562+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:25:52.346562+0000","last_peered":"2024-09-05T12:25:52.346562+0000","last_clean":"2024-09-05T12:25:52.346562+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:25:52.346562+0000","last_undegraded":"2024-09-05T12:25:52.346562+0000","last_fullsized":"2024-09-05T12:25:52.346562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330666+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:25:34.330666+0000","last_peered":"2024-09-05T12:25:34.330666+0000","last_clean":"2024-09-05T12:25:34.330666+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:25:34.330666+0000","last_undegraded":"2024-09-05T12:25:34.330666+0000","last_fullsized":"2024-09-05T12:25:34.330666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268645+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:25:34.268645+0000","last_peered":"2024-09-05T12:25:34.268645+0000","last_clean":"2024-09-05T12:25:34.268645+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:25:34.268645+0000","last_undegraded":"2024-09-05T12:25:34.268645+0000","last_fullsized":"2024-09-05T12:25:34.268645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322440+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:25:38.322440+0000","last_peered":"2024-09-05T12:25:38.322440+0000","last_clean":"2024-09-05T12:25:38.322440+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:25:38.322440+0000","last_undegraded":"2024-09-05T12:25:38.322440+0000","last_fullsized":"2024-09-05T12:25:38.322440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271674+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:25:34.271674+0000","last_peered":"2024-09-05T12:25:34.271674+0000","last_clean":"2024-09-05T12:25:34.271674+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:25:34.271674+0000","last_undegraded":"2024-09-05T12:25:34.271674+0000","last_fullsized":"2024-09-05T12:25:34.271674+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266874+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:25:34.266874+0000","last_peered":"2024-09-05T12:25:34.266874+0000","last_clean":"2024-09-05T12:25:34.266874+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:25:34.266874+0000","last_undegraded":"2024-09-05T12:25:34.266874+0000","last_fullsized":"2024-09-05T12:25:34.266874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.326756+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:25:39.326756+0000","last_peered":"2024-09-05T12:25:39.326756+0000","last_clean":"2024-09-05T12:25:39.326756+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:25:39.326756+0000","last_undegraded":"2024-09-05T12:25:39.326756+0000","last_fullsized":"2024-09-05T12:25:39.326756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269780+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:25:34.269780+0000","last_peered":"2024-09-05T12:25:34.269780+0000","last_clean":"2024-09-05T12:25:34.269780+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:25:34.269780+0000","last_undegraded":"2024-09-05T12:25:34.269780+0000","last_fullsized":"2024-09-05T12:25:34.269780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266924+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:25:34.266924+0000","last_peered":"2024-09-05T12:25:34.266924+0000","last_clean":"2024-09-05T12:25:34.266924+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:25:34.266924+0000","last_undegraded":"2024-09-05T12:25:34.266924+0000","last_fullsized":"2024-09-05T12:25:34.266924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'32","reported_seq":117,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:58.355912+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:25:58.355912+0000","last_peered":"2024-09-05T12:25:58.355912+0000","last_clean":"2024-09-05T12:25:58.355912+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:25:58.355912+0000","last_undegraded":"2024-09-05T12:25:58.355912+0000","last_fullsized":"2024-09-05T12:25:58.355912+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56678,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":79,"num_write":34,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330067+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:25:34.330067+0000","last_peered":"2024-09-05T12:25:34.330067+0000","last_clean":"2024-09-05T12:25:34.330067+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:25:34.330067+0000","last_undegraded":"2024-09-05T12:25:34.330067+0000","last_fullsized":"2024-09-05T12:25:34.330067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404529+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:25:34.404529+0000","last_peered":"2024-09-05T12:25:34.404529+0000","last_clean":"2024-09-05T12:25:34.404529+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:25:34.404529+0000","last_undegraded":"2024-09-05T12:25:34.404529+0000","last_fullsized":"2024-09-05T12:25:34.404529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313398+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:25:37.313398+0000","last_peered":"2024-09-05T12:25:37.313398+0000","last_clean":"2024-09-05T12:25:37.313398+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:25:37.313398+0000","last_undegraded":"2024-09-05T12:25:37.313398+0000","last_fullsized":"2024-09-05T12:25:37.313398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264502+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:25:34.264502+0000","last_peered":"2024-09-05T12:25:34.264502+0000","last_clean":"2024-09-05T12:25:34.264502+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:25:34.264502+0000","last_undegraded":"2024-09-05T12:25:34.264502+0000","last_fullsized":"2024-09-05T12:25:34.264502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268248+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:25:34.268248+0000","last_peered":"2024-09-05T12:25:34.268248+0000","last_clean":"2024-09-05T12:25:34.268248+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:25:34.268248+0000","last_undegraded":"2024-09-05T12:25:34.268248+0000","last_fullsized":"2024-09-05T12:25:34.268248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321510+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:25:38.321510+0000","last_peered":"2024-09-05T12:25:38.321510+0000","last_clean":"2024-09-05T12:25:38.321510+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:25:38.321510+0000","last_undegraded":"2024-09-05T12:25:38.321510+0000","last_fullsized":"2024-09-05T12:25:38.321510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272597+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:25:34.272597+0000","last_peered":"2024-09-05T12:25:34.272597+0000","last_clean":"2024-09-05T12:25:34.272597+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:25:34.272597+0000","last_undegraded":"2024-09-05T12:25:34.272597+0000","last_fullsized":"2024-09-05T12:25:34.272597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404720+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:25:34.404720+0000","last_peered":"2024-09-05T12:25:34.404720+0000","last_clean":"2024-09-05T12:25:34.404720+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:25:34.404720+0000","last_undegraded":"2024-09-05T12:25:34.404720+0000","last_fullsized":"2024-09-05T12:25:34.404720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":81,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:58.354362+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:25:58.354362+0000","last_peered":"2024-09-05T12:25:58.354362+0000","last_clean":"2024-09-05T12:25:58.354362+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:25:58.354362+0000","last_undegraded":"2024-09-05T12:25:58.354362+0000","last_fullsized":"2024-09-05T12:25:58.354362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263017+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:25:34.263017+0000","last_peered":"2024-09-05T12:25:34.263017+0000","last_clean":"2024-09-05T12:25:34.263017+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:25:34.263017+0000","last_undegraded":"2024-09-05T12:25:34.263017+0000","last_fullsized":"2024-09-05T12:25:34.263017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268073+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:25:34.268073+0000","last_peered":"2024-09-05T12:25:34.268073+0000","last_clean":"2024-09-05T12:25:34.268073+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:25:34.268073+0000","last_undegraded":"2024-09-05T12:25:34.268073+0000","last_fullsized":"2024-09-05T12:25:34.268073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.329887+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:25:34.329887+0000","last_peered":"2024-09-05T12:25:34.329887+0000","last_clean":"2024-09-05T12:25:34.329887+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:25:34.329887+0000","last_undegraded":"2024-09-05T12:25:34.329887+0000","last_fullsized":"2024-09-05T12:25:34.329887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322579+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:25:38.322579+0000","last_peered":"2024-09-05T12:25:38.322579+0000","last_clean":"2024-09-05T12:25:38.322579+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:25:38.322579+0000","last_undegraded":"2024-09-05T12:25:38.322579+0000","last_fullsized":"2024-09-05T12:25:38.322579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267082+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:25:34.267082+0000","last_peered":"2024-09-05T12:25:34.267082+0000","last_clean":"2024-09-05T12:25:34.267082+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:25:34.267082+0000","last_undegraded":"2024-09-05T12:25:34.267082+0000","last_fullsized":"2024-09-05T12:25:34.267082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263148+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:25:34.263148+0000","last_peered":"2024-09-05T12:25:34.263148+0000","last_clean":"2024-09-05T12:25:34.263148+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:25:34.263148+0000","last_undegraded":"2024-09-05T12:25:34.263148+0000","last_fullsized":"2024-09-05T12:25:34.263148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.302852+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:25:36.302852+0000","last_peered":"2024-09-05T12:25:36.302852+0000","last_clean":"2024-09-05T12:25:36.302852+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:25:36.302852+0000","last_undegraded":"2024-09-05T12:25:36.302852+0000","last_fullsized":"2024-09-05T12:25:36.302852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313373+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:25:37.313373+0000","last_peered":"2024-09-05T12:25:37.313373+0000","last_clean":"2024-09-05T12:25:37.313373+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:25:37.313373+0000","last_undegraded":"2024-09-05T12:25:37.313373+0000","last_fullsized":"2024-09-05T12:25:37.313373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269899+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:25:34.269899+0000","last_peered":"2024-09-05T12:25:34.269899+0000","last_clean":"2024-09-05T12:25:34.269899+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:25:34.269899+0000","last_undegraded":"2024-09-05T12:25:34.269899+0000","last_fullsized":"2024-09-05T12:25:34.269899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264384+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:25:34.264384+0000","last_peered":"2024-09-05T12:25:34.264384+0000","last_clean":"2024-09-05T12:25:34.264384+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:25:34.264384+0000","last_undegraded":"2024-09-05T12:25:34.264384+0000","last_fullsized":"2024-09-05T12:25:34.264384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:55.350848+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:25:55.350848+0000","last_peered":"2024-09-05T12:25:55.350848+0000","last_clean":"2024-09-05T12:25:55.350848+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:25:55.350848+0000","last_undegraded":"2024-09-05T12:25:55.350848+0000","last_fullsized":"2024-09-05T12:25:55.350848+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266460+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:25:34.266460+0000","last_peered":"2024-09-05T12:25:34.266460+0000","last_clean":"2024-09-05T12:25:34.266460+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:25:34.266460+0000","last_undegraded":"2024-09-05T12:25:34.266460+0000","last_fullsized":"2024-09-05T12:25:34.266460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264349+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:25:34.264349+0000","last_peered":"2024-09-05T12:25:34.264349+0000","last_clean":"2024-09-05T12:25:34.264349+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:25:34.264349+0000","last_undegraded":"2024-09-05T12:25:34.264349+0000","last_fullsized":"2024-09-05T12:25:34.264349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265242+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:25:34.265242+0000","last_peered":"2024-09-05T12:25:34.265242+0000","last_clean":"2024-09-05T12:25:34.265242+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:25:34.265242+0000","last_undegraded":"2024-09-05T12:25:34.265242+0000","last_fullsized":"2024-09-05T12:25:34.265242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264119+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:25:34.264119+0000","last_peered":"2024-09-05T12:25:34.264119+0000","last_clean":"2024-09-05T12:25:34.264119+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:25:34.264119+0000","last_undegraded":"2024-09-05T12:25:34.264119+0000","last_fullsized":"2024-09-05T12:25:34.264119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263751+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:25:34.263751+0000","last_peered":"2024-09-05T12:25:34.263751+0000","last_clean":"2024-09-05T12:25:34.263751+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:25:34.263751+0000","last_undegraded":"2024-09-05T12:25:34.263751+0000","last_fullsized":"2024-09-05T12:25:34.263751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270400+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:25:34.270400+0000","last_peered":"2024-09-05T12:25:34.270400+0000","last_clean":"2024-09-05T12:25:34.270400+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:25:34.270400+0000","last_undegraded":"2024-09-05T12:25:34.270400+0000","last_fullsized":"2024-09-05T12:25:34.270400+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331799+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:25:34.331799+0000","last_peered":"2024-09-05T12:25:34.331799+0000","last_clean":"2024-09-05T12:25:34.331799+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:25:34.331799+0000","last_undegraded":"2024-09-05T12:25:34.331799+0000","last_fullsized":"2024-09-05T12:25:34.331799+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266950+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:25:34.266950+0000","last_peered":"2024-09-05T12:25:34.266950+0000","last_clean":"2024-09-05T12:25:34.266950+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:25:34.266950+0000","last_undegraded":"2024-09-05T12:25:34.266950+0000","last_fullsized":"2024-09-05T12:25:34.266950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264875+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:25:34.264875+0000","last_peered":"2024-09-05T12:25:34.264875+0000","last_clean":"2024-09-05T12:25:34.264875+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:25:34.264875+0000","last_undegraded":"2024-09-05T12:25:34.264875+0000","last_fullsized":"2024-09-05T12:25:34.264875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263394+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:25:34.263394+0000","last_peered":"2024-09-05T12:25:34.263394+0000","last_clean":"2024-09-05T12:25:34.263394+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:25:34.263394+0000","last_undegraded":"2024-09-05T12:25:34.263394+0000","last_fullsized":"2024-09-05T12:25:34.263394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269818+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:25:34.269818+0000","last_peered":"2024-09-05T12:25:34.269818+0000","last_clean":"2024-09-05T12:25:34.269818+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:25:34.269818+0000","last_undegraded":"2024-09-05T12:25:34.269818+0000","last_fullsized":"2024-09-05T12:25:34.269818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263113+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:25:34.263113+0000","last_peered":"2024-09-05T12:25:34.263113+0000","last_clean":"2024-09-05T12:25:34.263113+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:25:34.263113+0000","last_undegraded":"2024-09-05T12:25:34.263113+0000","last_fullsized":"2024-09-05T12:25:34.263113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404441+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:25:34.404441+0000","last_peered":"2024-09-05T12:25:34.404441+0000","last_clean":"2024-09-05T12:25:34.404441+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:25:34.404441+0000","last_undegraded":"2024-09-05T12:25:34.404441+0000","last_fullsized":"2024-09-05T12:25:34.404441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267531+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:25:34.267531+0000","last_peered":"2024-09-05T12:25:34.267531+0000","last_clean":"2024-09-05T12:25:34.267531+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:25:34.267531+0000","last_undegraded":"2024-09-05T12:25:34.267531+0000","last_fullsized":"2024-09-05T12:25:34.267531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266222+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:25:34.266222+0000","last_peered":"2024-09-05T12:25:34.266222+0000","last_clean":"2024-09-05T12:25:34.266222+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:25:34.266222+0000","last_undegraded":"2024-09-05T12:25:34.266222+0000","last_fullsized":"2024-09-05T12:25:34.266222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264463+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:25:34.264463+0000","last_peered":"2024-09-05T12:25:34.264463+0000","last_clean":"2024-09-05T12:25:34.264463+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:25:34.264463+0000","last_undegraded":"2024-09-05T12:25:34.264463+0000","last_fullsized":"2024-09-05T12:25:34.264463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330539+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:25:34.330539+0000","last_peered":"2024-09-05T12:25:34.330539+0000","last_clean":"2024-09-05T12:25:34.330539+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:25:34.330539+0000","last_undegraded":"2024-09-05T12:25:34.330539+0000","last_fullsized":"2024-09-05T12:25:34.330539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267312+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:25:34.267312+0000","last_peered":"2024-09-05T12:25:34.267312+0000","last_clean":"2024-09-05T12:25:34.267312+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:25:34.267312+0000","last_undegraded":"2024-09-05T12:25:34.267312+0000","last_fullsized":"2024-09-05T12:25:34.267312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404202+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:25:34.404202+0000","last_peered":"2024-09-05T12:25:34.404202+0000","last_clean":"2024-09-05T12:25:34.404202+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:25:34.404202+0000","last_undegraded":"2024-09-05T12:25:34.404202+0000","last_fullsized":"2024-09-05T12:25:34.404202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266403+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:25:34.266403+0000","last_peered":"2024-09-05T12:25:34.266403+0000","last_clean":"2024-09-05T12:25:34.266403+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:25:34.266403+0000","last_undegraded":"2024-09-05T12:25:34.266403+0000","last_fullsized":"2024-09-05T12:25:34.266403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266842+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:25:34.266842+0000","last_peered":"2024-09-05T12:25:34.266842+0000","last_clean":"2024-09-05T12:25:34.266842+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:25:34.266842+0000","last_undegraded":"2024-09-05T12:25:34.266842+0000","last_fullsized":"2024-09-05T12:25:34.266842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267435+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:25:34.267435+0000","last_peered":"2024-09-05T12:25:34.267435+0000","last_clean":"2024-09-05T12:25:34.267435+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:25:34.267435+0000","last_undegraded":"2024-09-05T12:25:34.267435+0000","last_fullsized":"2024-09-05T12:25:34.267435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265181+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:25:34.265181+0000","last_peered":"2024-09-05T12:25:34.265181+0000","last_clean":"2024-09-05T12:25:34.265181+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:25:34.265181+0000","last_undegraded":"2024-09-05T12:25:34.265181+0000","last_fullsized":"2024-09-05T12:25:34.265181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268910+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:25:34.268910+0000","last_peered":"2024-09-05T12:25:34.268910+0000","last_clean":"2024-09-05T12:25:34.268910+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:25:34.268910+0000","last_undegraded":"2024-09-05T12:25:34.268910+0000","last_fullsized":"2024-09-05T12:25:34.268910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331174+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:25:34.331174+0000","last_peered":"2024-09-05T12:25:34.331174+0000","last_clean":"2024-09-05T12:25:34.331174+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:25:34.331174+0000","last_undegraded":"2024-09-05T12:25:34.331174+0000","last_fullsized":"2024-09-05T12:25:34.331174+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404780+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:25:34.404780+0000","last_peered":"2024-09-05T12:25:34.404780+0000","last_clean":"2024-09-05T12:25:34.404780+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:25:34.404780+0000","last_undegraded":"2024-09-05T12:25:34.404780+0000","last_fullsized":"2024-09-05T12:25:34.404780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263044+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:25:34.263044+0000","last_peered":"2024-09-05T12:25:34.263044+0000","last_clean":"2024-09-05T12:25:34.263044+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:25:34.263044+0000","last_undegraded":"2024-09-05T12:25:34.263044+0000","last_fullsized":"2024-09-05T12:25:34.263044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268375+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:25:34.268375+0000","last_peered":"2024-09-05T12:25:34.268375+0000","last_clean":"2024-09-05T12:25:34.268375+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:25:34.268375+0000","last_undegraded":"2024-09-05T12:25:34.268375+0000","last_fullsized":"2024-09-05T12:25:34.268375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267499+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:25:34.267499+0000","last_peered":"2024-09-05T12:25:34.267499+0000","last_clean":"2024-09-05T12:25:34.267499+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:25:34.267499+0000","last_undegraded":"2024-09-05T12:25:34.267499+0000","last_fullsized":"2024-09-05T12:25:34.267499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265767+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:25:34.265767+0000","last_peered":"2024-09-05T12:25:34.265767+0000","last_clean":"2024-09-05T12:25:34.265767+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:25:34.265767+0000","last_undegraded":"2024-09-05T12:25:34.265767+0000","last_fullsized":"2024-09-05T12:25:34.265767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265061+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:25:34.265061+0000","last_peered":"2024-09-05T12:25:34.265061+0000","last_clean":"2024-09-05T12:25:34.265061+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:25:34.265061+0000","last_undegraded":"2024-09-05T12:25:34.265061+0000","last_fullsized":"2024-09-05T12:25:34.265061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267876+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:25:34.267876+0000","last_peered":"2024-09-05T12:25:34.267876+0000","last_clean":"2024-09-05T12:25:34.267876+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:25:34.267876+0000","last_undegraded":"2024-09-05T12:25:34.267876+0000","last_fullsized":"2024-09-05T12:25:34.267876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268171+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:25:34.268171+0000","last_peered":"2024-09-05T12:25:34.268171+0000","last_clean":"2024-09-05T12:25:34.268171+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:25:34.268171+0000","last_undegraded":"2024-09-05T12:25:34.268171+0000","last_fullsized":"2024-09-05T12:25:34.268171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267268+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:25:34.267268+0000","last_peered":"2024-09-05T12:25:34.267268+0000","last_clean":"2024-09-05T12:25:34.267268+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:25:34.267268+0000","last_undegraded":"2024-09-05T12:25:34.267268+0000","last_fullsized":"2024-09-05T12:25:34.267268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63493,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":221,"num_read_kb":185,"num_write":244,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":186214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449115,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":533938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678048,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28756,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716396,"statfs":{"total":95995035648,"available":95965589504,"internally_reserved":0,"allocated":1773568,"data_stored":1181113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841573,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":533453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103210,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":533809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1699840,"data_stored":1129937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":528957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1708032,"data_stored":1125519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":533031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":528665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1163264,"data_stored":587278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36456,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708696,"statfs":{"total":95995035648,"available":95957704704,"internally_reserved":0,"allocated":1138688,"data_stored":582400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":530806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":59043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":58414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:26:00.255 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T12:26:00.255 INFO:tasks.cephadm:Setting up client nodes... 2024-09-05T12:26:00.255 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-05T12:26:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:00 smithi174 ceph-mon[34313]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-09-05T12:26:00.624 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:00 smithi071 ceph-mon[29845]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-09-05T12:26:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:00 smithi123 ceph-mon[34321]: pgmap v337: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-09-05T12:26:01.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 -- 172.21.15.71:0/3595339233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c103e00 msgr2=0x7f0d9c104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 --2- 172.21.15.71:0/3595339233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c103e00 0x7f0d9c104200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009900 tx=0x7f0d8c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 -- 172.21.15.71:0/3595339233 shutdown_connections 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 --2- 172.21.15.71:0/3595339233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d9c105920 0x7f0d9c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 --2- 172.21.15.71:0/3595339233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d9c104f60 0x7f0d9c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 --2- 172.21.15.71:0/3595339233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c103e00 0x7f0d9c104200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 -- 172.21.15.71:0/3595339233 >> 172.21.15.71:0/3595339233 conn(0x7f0d9c0ff5b0 msgr2=0x7f0d9c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 -- 172.21.15.71:0/3595339233 shutdown_connections 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.231+0000 7f0da2a00640 1 -- 172.21.15.71:0/3595339233 wait complete. 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 Processor -- start 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 -- start start 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d9c103e00 0x7f0d9c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d9c104f60 0x7f0d9c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 0x7f0d9c1a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:01.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10fab0 con 0x7f0d9c105920 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10f930 con 0x7f0d9c103e00 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.232+0000 7f0da2a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c10fc30 con 0x7f0d9c104f60 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 0x7f0d9c1a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 0x7f0d9c1a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41262/0 (socket says 172.21.15.71:41262) 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 -- 172.21.15.71:0/1087842349 learned_addr learned my addr 172.21.15.71:0/1087842349 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0d9b400640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d9c104f60 0x7f0d9c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0d9be00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d9c103e00 0x7f0d9c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 -- 172.21.15.71:0/1087842349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d9c104f60 msgr2=0x7f0d9c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d9c104f60 0x7f0d9c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 -- 172.21.15.71:0/1087842349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d9c103e00 msgr2=0x7f0d9c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d9c103e00 0x7f0d9c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.233+0000 7f0da0a00640 1 -- 172.21.15.71:0/1087842349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c1a1d10 con 0x7f0d9c105920 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.234+0000 7f0da0a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 0x7f0d9c1a16a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400eb10 tx=0x7f0d9400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.234+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94004b20 con 0x7f0d9c105920 2024-09-05T12:26:01.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.234+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d94004cc0 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.234+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d94010760 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.234+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c1a1fa0 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.234+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c1a2500 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.235+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c10b1a0 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.237+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0d94005020 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.238+0000 7f0d98c00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0d800a2cc0 0x7f0d800a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.238+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f0d940cf990 con 0x7f0d9c105920 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.238+0000 7f0d9be00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0d800a2cc0 0x7f0d800a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.239+0000 7f0d9be00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0d800a2cc0 0x7f0d800a5180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0314e0 tx=0x7f0d8c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:01.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.242+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d9409c3d0 con 0x7f0d9c105920 2024-09-05T12:26:01.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:01 smithi071 ceph-mon[29845]: from='client.24743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:01.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.487+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f0d9c02d7c0 con 0x7f0d9c105920 2024-09-05T12:26:01.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.489+0000 7f0d98c00640 1 -- 172.21.15.71:0/1087842349 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f0d9409bcd0 con 0x7f0d9c105920 2024-09-05T12:26:01.491 INFO:teuthology.orchestra.run.smithi071.stdout:[client.0] 2024-09-05T12:26:01.491 INFO:teuthology.orchestra.run.smithi071.stdout: key = AQBZo9lm5j4rHRAAlJmE2a9t+TcHBwKrSSeqdA== 2024-09-05T12:26:01.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0d800a2cc0 msgr2=0x7f0d800a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:01.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0d800a2cc0 0x7f0d800a5180 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0314e0 tx=0x7f0d8c000bc0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 msgr2=0x7f0d9c1a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:01.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 0x7f0d9c1a16a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400eb10 tx=0x7f0d9400c2d0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 shutdown_connections 2024-09-05T12:26:01.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f0d800a2cc0 0x7f0d800a5180 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d9c105920 0x7f0d9c1a16a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d9c104f60 0x7f0d9c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 --2- 172.21.15.71:0/1087842349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d9c103e00 0x7f0d9c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 >> 172.21.15.71:0/1087842349 conn(0x7f0d9c0ff5b0 msgr2=0x7f0d9c108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 shutdown_connections 2024-09-05T12:26:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:01.493+0000 7f0da2a00640 1 -- 172.21.15.71:0/1087842349 wait complete. 2024-09-05T12:26:01.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:01 smithi174 ceph-mon[34313]: from='client.24743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:01.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:01 smithi123 ceph-mon[34321]: from='client.24743 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:02.296 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:26:02.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-05T12:26:02.296 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-05T12:26:02.376 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-05T12:26:02.376 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-05T12:26:02.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph mgr dump --format=json 2024-09-05T12:26:02.464 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:02 smithi071 ceph-mon[29845]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-05T12:26:02.464 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:02 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1087842349' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T12:26:02.464 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:02 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1087842349' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T12:26:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:02 smithi174 ceph-mon[34313]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-05T12:26:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:02 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1087842349' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T12:26:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:02 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1087842349' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T12:26:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:02 smithi123 ceph-mon[34321]: pgmap v338: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-09-05T12:26:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:02 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1087842349' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-05T12:26:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:02 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1087842349' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-05T12:26:02.789 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:03.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 -- 172.21.15.71:0/950132527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0077040 msgr2=0x7f2bb0112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 --2- 172.21.15.71:0/950132527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0077040 0x7f2bb0112330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2b98009900 tx=0x7f2b9802a410 comp rx=0 tx=0).stop 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 -- 172.21.15.71:0/950132527 shutdown_connections 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 --2- 172.21.15.71:0/950132527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0112870 0x7f2bb0114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 --2- 172.21.15.71:0/950132527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0077040 0x7f2bb0112330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 --2- 172.21.15.71:0/950132527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0076720 0x7f2bb0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 -- 172.21.15.71:0/950132527 >> 172.21.15.71:0/950132527 conn(0x7f2bb00ff8d0 msgr2=0x7f2bb0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 -- 172.21.15.71:0/950132527 shutdown_connections 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.380+0000 7f2bb6200640 1 -- 172.21.15.71:0/950132527 wait complete. 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 Processor -- start 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 -- start start 2024-09-05T12:26:03.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 0x7f2bb01a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0077040 0x7f2bb01a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0112870 0x7f2bb019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb0118430 con 0x7f2bb0076720 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01182b0 con 0x7f2bb0112870 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.381+0000 7f2bb6200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb01185b0 con 0x7f2bb0077040 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 0x7f2bb01a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 0x7f2bb01a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34556/0 (socket says 172.21.15.71:34556) 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 -- 172.21.15.71:0/3609072760 learned_addr learned my addr 172.21.15.71:0/3609072760 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2ba7e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0077040 0x7f2bb01a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb5800640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0112870 0x7f2bb019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 -- 172.21.15.71:0/3609072760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0077040 msgr2=0x7f2bb01a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:03.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0077040 0x7f2bb01a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 -- 172.21.15.71:0/3609072760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0112870 msgr2=0x7f2bb019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0112870 0x7f2bb019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2bb4e00640 1 -- 172.21.15.71:0/3609072760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb019ef90 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.382+0000 7f2ba7e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0077040 0x7f2bb01a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.383+0000 7f2bb4e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 0x7f2bb01a4650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000e9f0 tx=0x7f2ba000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.383+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba000cd80 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.383+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb019f280 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.383+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ba0010910 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.383+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00188f0 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.383+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb01ab5d0 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.384+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb0077e60 con 0x7f2bb0076720 2024-09-05T12:26:03.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.386+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2ba0010ac0 con 0x7f2bb0076720 2024-09-05T12:26:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.387+0000 7f2bae000640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b940a2c10 0x7f2b940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.387+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f2ba00cfd20 con 0x7f2bb0076720 2024-09-05T12:26:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.387+0000 7f2ba7e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b940a2c10 0x7f2b940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.388+0000 7f2ba7e00640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b940a2c10 0x7f2b940a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b98000c00 tx=0x7f2b98031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:03.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.389+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ba009c110 con 0x7f2bb0076720 2024-09-05T12:26:03.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.608+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2bb010da00 con 0x7f2bb0076720 2024-09-05T12:26:03.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.615+0000 7f2bae000640 1 -- 172.21.15.71:0/3609072760 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263200 (secure 0 0 0) 0x7f2ba00a0410 con 0x7f2bb0076720 2024-09-05T12:26:03.623 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:03.625 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b940a2c10 msgr2=0x7f2b940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b940a2c10 0x7f2b940a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2b98000c00 tx=0x7f2b98031510 comp rx=0 tx=0).stop 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 msgr2=0x7f2bb01a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 0x7f2bb01a4650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000e9f0 tx=0x7f2ba000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 shutdown_connections 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2b940a2c10 0x7f2b940a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb0112870 0x7f2bb019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb0077040 0x7f2bb01a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 --2- 172.21.15.71:0/3609072760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb0076720 0x7f2bb01a4650 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:03.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 >> 172.21.15.71:0/3609072760 conn(0x7f2bb00ff8d0 msgr2=0x7f2bb0113470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:03.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.619+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 shutdown_connections 2024-09-05T12:26:03.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:03.620+0000 7f2bb6200640 1 -- 172.21.15.71:0/3609072760 wait complete. 2024-09-05T12:26:04.479 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:04 smithi071 ceph-mon[29845]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-05T12:26:04.479 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:04 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3609072760' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T12:26:04.481 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2050574497},{"type":"v1","addr":"172.21.15.71:6801","nonce":2050574497}]},"active_addr":"172.21.15.71:6801/2050574497","active_change":"2024-09-05T12:17:33.045968+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.71:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2607784636}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":4208428214}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":372010080}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":3957197147}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.71:0","nonce":2730905603}]}]} 2024-09-05T12:26:04.487 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-05T12:26:04.487 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-05T12:26:04.487 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:26:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:04 smithi174 ceph-mon[34313]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-05T12:26:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:04 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3609072760' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T12:26:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:04 smithi123 ceph-mon[34321]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-05T12:26:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:04 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3609072760' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-05T12:26:04.854 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:05.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 -- 172.21.15.71:0/1564672350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c074f40 msgr2=0x7fa98c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:05.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 --2- 172.21.15.71:0/1564672350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c074f40 0x7fa98c10de80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa978009900 tx=0x7fa97802a480 comp rx=0 tx=0).stop 2024-09-05T12:26:05.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 -- 172.21.15.71:0/1564672350 shutdown_connections 2024-09-05T12:26:05.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 --2- 172.21.15.71:0/1564672350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa98c10e3c0 0x7fa98c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 --2- 172.21.15.71:0/1564672350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c074f40 0x7fa98c10de80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 --2- 172.21.15.71:0/1564672350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074620 0x7fa98c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 -- 172.21.15.71:0/1564672350 >> 172.21.15.71:0/1564672350 conn(0x7fa98c0fd8c0 msgr2=0x7fa98c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 -- 172.21.15.71:0/1564672350 shutdown_connections 2024-09-05T12:26:05.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.425+0000 7fa990a00640 1 -- 172.21.15.71:0/1564672350 wait complete. 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 Processor -- start 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 -- start start 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa98c074620 0x7fa98c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074f40 0x7fa98c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 0x7fa98c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c114160 con 0x7fa98c10e3c0 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c113fe0 con 0x7fa98c074f40 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.426+0000 7fa990a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa98c1142e0 con 0x7fa98c074620 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 0x7fa98c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 0x7fa98c19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34574/0 (socket says 172.21.15.71:34574) 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 -- 172.21.15.71:0/1945545911 learned_addr learned my addr 172.21.15.71:0/1945545911 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:05.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa989400640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074f40 0x7fa98c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa989e00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa98c074620 0x7fa98c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 -- 172.21.15.71:0/1945545911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa98c074620 msgr2=0x7fa98c19a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa98c074620 0x7fa98c19a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 -- 172.21.15.71:0/1945545911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074f40 msgr2=0x7fa98c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074f40 0x7fa98c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa98a800640 1 -- 172.21.15.71:0/1945545911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa98c19f630 con 0x7fa98c10e3c0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.427+0000 7fa989400640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074f40 0x7fa98c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.428+0000 7fa98a800640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 0x7fa98c19ef50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa98400eb10 tx=0x7fa98400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.428+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984004b20 con 0x7fa98c10e3c0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.428+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa984004cc0 con 0x7fa98c10e3c0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.428+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa98c19f920 con 0x7fa98c10e3c0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.428+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa984010760 con 0x7fa98c10e3c0 2024-09-05T12:26:05.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.428+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa98c1a7200 con 0x7fa98c10e3c0 2024-09-05T12:26:05.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.429+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa98c074a00 con 0x7fa98c10e3c0 2024-09-05T12:26:05.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.431+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa984005020 con 0x7fa98c10e3c0 2024-09-05T12:26:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.432+0000 7fa982a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9700a2c90 0x7fa9700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.432+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa9840cfb40 con 0x7fa98c10e3c0 2024-09-05T12:26:05.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.433+0000 7fa989e00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9700a2c90 0x7fa9700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:05.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.433+0000 7fa989e00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9700a2c90 0x7fa9700a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0095f0 tx=0x7fa97c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:05.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.437+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa98409c580 con 0x7fa98c10e3c0 2024-09-05T12:26:05.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.617+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa98c02d7c0 con 0x7fa98c10e3c0 2024-09-05T12:26:05.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.619+0000 7fa982a00640 1 -- 172.21.15.71:0/1945545911 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24741 (secure 0 0 0) 0x7fa98409be80 con 0x7fa98c10e3c0 2024-09-05T12:26:05.620 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:05.620 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:34.259275+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:26:05.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9700a2c90 msgr2=0x7fa9700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:05.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9700a2c90 0x7fa9700a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0095f0 tx=0x7fa97c005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:05.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 msgr2=0x7fa98c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:05.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 0x7fa98c19ef50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa98400eb10 tx=0x7fa98400c2d0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 shutdown_connections 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa9700a2c90 0x7fa9700a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa98c10e3c0 0x7fa98c19ef50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa98c074f40 0x7fa98c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 --2- 172.21.15.71:0/1945545911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa98c074620 0x7fa98c19a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.621+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 >> 172.21.15.71:0/1945545911 conn(0x7fa98c0fd8c0 msgr2=0x7fa98c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.622+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 shutdown_connections 2024-09-05T12:26:05.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:05.622+0000 7fa990a00640 1 -- 172.21.15.71:0/1945545911 wait complete. 2024-09-05T12:26:06.404 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:06 smithi071 ceph-mon[29845]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-09-05T12:26:06.404 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:06 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1945545911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:06.441 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-05T12:26:06.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:26:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:06 smithi174 ceph-mon[34313]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-09-05T12:26:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:06 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1945545911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:06 smithi123 ceph-mon[34321]: pgmap v340: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.5 KiB/s wr, 14 op/s 2024-09-05T12:26:06.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:06 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1945545911' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:06.810 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:07.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 -- 172.21.15.71:0/1577530896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 msgr2=0x7f998c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 --2- 172.21.15.71:0/1577530896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c112330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9978009900 tx=0x7f997802a410 comp rx=0 tx=0).stop 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 -- 172.21.15.71:0/1577530896 shutdown_connections 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 --2- 172.21.15.71:0/1577530896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f998c112870 0x7f998c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 --2- 172.21.15.71:0/1577530896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c112330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 --2- 172.21.15.71:0/1577530896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f998c076720 0x7f998c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 -- 172.21.15.71:0/1577530896 >> 172.21.15.71:0/1577530896 conn(0x7f998c0ff910 msgr2=0x7f998c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 -- 172.21.15.71:0/1577530896 shutdown_connections 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.383+0000 7f9992c00640 1 -- 172.21.15.71:0/1577530896 wait complete. 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.384+0000 7f9992c00640 1 Processor -- start 2024-09-05T12:26:07.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.384+0000 7f9992c00640 1 -- start start 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f998c076720 0x7f998c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f998c112870 0x7f998c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f998c1183f0 con 0x7f998c077040 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f998c118270 con 0x7f998c112870 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f998c118570 con 0x7f998c076720 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c1a4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34598/0 (socket says 172.21.15.71:34598) 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 -- 172.21.15.71:0/3030244427 learned_addr learned my addr 172.21.15.71:0/3030244427 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9991800640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f998c076720 0x7f998c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9992200640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f998c112870 0x7f998c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 -- 172.21.15.71:0/3030244427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f998c076720 msgr2=0x7f998c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f998c076720 0x7f998c1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.408 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 -- 172.21.15.71:0/3030244427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f998c112870 msgr2=0x7f998c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f998c112870 0x7f998c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.385+0000 7f9990e00640 1 -- 172.21.15.71:0/3030244427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c19ef70 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.386+0000 7f9990e00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c1a4b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9978006010 tx=0x7f9978004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.386+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997803f030 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.386+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9978004440 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.386+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9978035070 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.386+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998c19f200 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.386+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f998c1ab3e0 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.387+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c077e60 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.389+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9978042030 con 0x7f998c077040 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.392+0000 7f998a000640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f99740a2c10 0x7f99740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:07.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.392+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f99780ef990 con 0x7f998c077040 2024-09-05T12:26:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.392+0000 7f9991800640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f99740a2c10 0x7f99740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.393+0000 7f9991800640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f99740a2c10 0x7f99740a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9980009bd0 tx=0x7f9980005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:07.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.396+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99780bc3d0 con 0x7f998c077040 2024-09-05T12:26:07.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.571+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f998c076b00 con 0x7f998c077040 2024-09-05T12:26:07.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.573+0000 7f998a000640 1 -- 172.21.15.71:0/3030244427 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24741 (secure 0 0 0) 0x7f99780bbcd0 con 0x7f998c077040 2024-09-05T12:26:07.575 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:07.575 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:34.259275+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f99740a2c10 msgr2=0x7f99740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f99740a2c10 0x7f99740a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9980009bd0 tx=0x7f9980005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 msgr2=0x7f998c1a4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c1a4b70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9978006010 tx=0x7f9978004040 comp rx=0 tx=0).stop 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 shutdown_connections 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f99740a2c10 0x7f99740a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f998c112870 0x7f998c19e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f998c077040 0x7f998c1a4b70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 --2- 172.21.15.71:0/3030244427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f998c076720 0x7f998c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 >> 172.21.15.71:0/3030244427 conn(0x7f998c0ff910 msgr2=0x7f998c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:07.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 shutdown_connections 2024-09-05T12:26:07.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:07.576+0000 7f9992c00640 1 -- 172.21.15.71:0/3030244427 wait complete. 2024-09-05T12:26:08.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:08 smithi071 ceph-mon[29845]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-05T12:26:08.337 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:08 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3030244427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:08.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-05T12:26:08.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-05T12:26:08.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-05T12:26:08.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-05T12:26:08.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-05T12:26:08.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-05T12:26:08.377 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-05T12:26:08.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-05T12:26:08.378 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-05T12:26:08.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-05T12:26:08.379 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-05T12:26:08.380 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-05T12:26:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:08 smithi174 ceph-mon[34313]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-05T12:26:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:08 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3030244427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:08 smithi123 ceph-mon[34321]: pgmap v341: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-05T12:26:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:08 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3030244427' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:09.350 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:09.374 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:09.377 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:09.381 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:09.480 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:10.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 -- 172.21.15.71:0/1892128639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 msgr2=0x7f87ec110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 --2- 172.21.15.71:0/1892128639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 0x7f87ec110ca0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009ca0 tx=0x7f87e402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 -- 172.21.15.71:0/1892128639 shutdown_connections 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 --2- 172.21.15.71:0/1892128639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 0x7f87ec110ca0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 --2- 172.21.15.71:0/1892128639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87ec1049a0 0x7f87ec1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 --2- 172.21.15.71:0/1892128639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec103ff0 0x7f87ec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 -- 172.21.15.71:0/1892128639 >> 172.21.15.71:0/1892128639 conn(0x7f87ec0ffa90 msgr2=0x7f87ec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 -- 172.21.15.71:0/1892128639 shutdown_connections 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.146+0000 7f87f3c00640 1 -- 172.21.15.71:0/1892128639 wait complete. 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.147+0000 7f87f3c00640 1 Processor -- start 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.147+0000 7f87f3c00640 1 -- start start 2024-09-05T12:26:10.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f3c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87ec103ff0 0x7f87ec19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f3c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec1049a0 0x7f87ec19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f3c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 0x7f87ec19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f3c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec1141a0 con 0x7f87ec109120 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f3c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec114020 con 0x7f87ec103ff0 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f3c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87ec114320 con 0x7f87ec1049a0 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec1049a0 0x7f87ec19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec1049a0 0x7f87ec19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45452/0 (socket says 172.21.15.71:45452) 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 -- 172.21.15.71:0/799750356 learned_addr learned my addr 172.21.15.71:0/799750356 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f1c00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 0x7f87ec19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87f1200640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87ec103ff0 0x7f87ec19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 -- 172.21.15.71:0/799750356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87ec103ff0 msgr2=0x7f87ec19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:10.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87ec103ff0 0x7f87ec19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 -- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 msgr2=0x7f87ec19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87ec109120 0x7f87ec19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.148+0000 7f87ebe00640 1 -- 172.21.15.71:0/799750356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ec19f590 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.149+0000 7f87ebe00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec1049a0 0x7f87ec19ab20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87d800ea60 tx=0x7f87d800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.149+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d800cde0 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.149+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ec19f880 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.149+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87d80035f0 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.149+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8010670 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.149+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ec1a7160 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.150+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87ec101020 con 0x7f87ec1049a0 2024-09-05T12:26:10.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.152+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f87d80037a0 con 0x7f87ec1049a0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.153+0000 7f87e9600640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f87d40a2c10 0x7f87d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f87d80cfb00 con 0x7f87ec1049a0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87f1200640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f87d40a2c10 0x7f87d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87e9600640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] conn(0x7f87d40adde0 0x7f87d40b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 --> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87d8023e50 con 0x7f87d40adde0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f87d80cfeb0 con 0x7f87ec1049a0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87f1c00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] conn(0x7f87d40adde0 0x7f87d40b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.154+0000 7f87f1200640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f87d40a2c10 0x7f87d40a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009bd0 tx=0x7f87e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.155+0000 7f87f1c00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] conn(0x7f87d40adde0 0x7f87d40b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:10.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.156+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== osd.2 v2:172.21.15.174:6800/1307000891 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f87d8023e50 con 0x7f87d40adde0 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.170+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 --> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f87ec1054d0 con 0x7f87d40adde0 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.170+0000 7f87e9600640 1 -- 172.21.15.71:0/799750356 <== osd.2 v2:172.21.15.174:6800/1307000891 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f87ec1054d0 con 0x7f87d40adde0 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] conn(0x7f87d40adde0 msgr2=0x7f87d40b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] conn(0x7f87d40adde0 0x7f87d40b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f87d40a2c10 msgr2=0x7f87d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f87d40a2c10 0x7f87d40a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f87e0009bd0 tx=0x7f87e0005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec1049a0 msgr2=0x7f87ec19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 --2- 172.21.15.71:0/799750356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87ec1049a0 0x7f87ec19ab20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f87d800ea60 tx=0x7f87d800ef20 comp rx=0 tx=0).stop 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f1c00640 1 -- 172.21.15.71:0/799750356 reap_dead start 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 shutdown_connections 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 >> 172.21.15.71:0/799750356 conn(0x7f87ec0ffa90 msgr2=0x7f87ec1005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:10.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 shutdown_connections 2024-09-05T12:26:10.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.171+0000 7f87f3c00640 1 -- 172.21.15.71:0/799750356 wait complete. 2024-09-05T12:26:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:10 smithi071 ceph-mon[29845]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-05T12:26:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:10 smithi174 ceph-mon[34313]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-05T12:26:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:10 smithi123 ceph-mon[34321]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 -- 172.21.15.71:0/3484881160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac101de0 msgr2=0x7febac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 --2- 172.21.15.71:0/3484881160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac101de0 0x7febac1021c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009900 tx=0x7feb9c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 -- 172.21.15.71:0/3484881160 shutdown_connections 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 --2- 172.21.15.71:0/3484881160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac106f10 0x7febac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 --2- 172.21.15.71:0/3484881160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac102790 0x7febac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 --2- 172.21.15.71:0/3484881160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac101de0 0x7febac1021c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 -- 172.21.15.71:0/3484881160 >> 172.21.15.71:0/3484881160 conn(0x7febac0fd8c0 msgr2=0x7febac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 -- 172.21.15.71:0/3484881160 shutdown_connections 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.968+0000 7febb2000640 1 -- 172.21.15.71:0/3484881160 wait complete. 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 Processor -- start 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 -- start start 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac101de0 0x7febac19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac102790 0x7febac19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac106f10 0x7febac19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febac111f70 con 0x7febac106f10 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febac111df0 con 0x7febac101de0 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.969+0000 7febb2000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febac1120f0 con 0x7febac102790 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac106f10 0x7febac19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac106f10 0x7febac19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34638/0 (socket says 172.21.15.71:34638) 2024-09-05T12:26:12.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 -- 172.21.15.71:0/958604860 learned_addr learned my addr 172.21.15.71:0/958604860 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febab400640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac101de0 0x7febac19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7feba2a00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac102790 0x7febac19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 -- 172.21.15.71:0/958604860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac102790 msgr2=0x7febac19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febac102790 0x7febac19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 -- 172.21.15.71:0/958604860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac101de0 msgr2=0x7febac19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febac101de0 0x7febac19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 -- 172.21.15.71:0/958604860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febac19f660 con 0x7febac106f10 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.970+0000 7febabe00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac106f10 0x7febac19ef80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feba400eb10 tx=0x7feba400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:12.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.971+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4004b20 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.971+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feba4004cc0 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.971+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febac19f950 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.971+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba4010760 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.971+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febac1a7230 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.971+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febac0fee90 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.974+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7feba4005020 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.975+0000 7feba8c00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feb900a2c10 0x7feb900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.975+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7feba40cfeb0 con 0x7febac106f10 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.975+0000 7febab400640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feb900a2c10 0x7feb900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.975+0000 7feba8c00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] conn(0x7feb900adde0 0x7feb900b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.975+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 --> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb900b0950 con 0x7feb900adde0 2024-09-05T12:26:12.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.975+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7feba40d0260 con 0x7febac106f10 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.976+0000 7feba2a00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] conn(0x7feb900adde0 0x7feb900b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.976+0000 7febab400640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feb900a2c10 0x7feb900a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009230 tx=0x7feb9c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.976+0000 7feba2a00640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] conn(0x7feb900adde0 0x7feb900b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.977+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== osd.3 v2:172.21.15.71:6810/3336671100 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb900b0950 con 0x7feb900adde0 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.991+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 --> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febac1032c0 con 0x7feb900adde0 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.991+0000 7feba8c00640 1 -- 172.21.15.71:0/958604860 <== osd.3 v2:172.21.15.71:6810/3336671100 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7febac1032c0 con 0x7feb900adde0 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] conn(0x7feb900adde0 msgr2=0x7feb900b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] conn(0x7feb900adde0 0x7feb900b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feb900a2c10 msgr2=0x7feb900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7feb900a2c10 0x7feb900a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009230 tx=0x7feb9c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac106f10 msgr2=0x7febac19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:12.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 --2- 172.21.15.71:0/958604860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac106f10 0x7febac19ef80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feba400eb10 tx=0x7feba400c2d0 comp rx=0 tx=0).stop 2024-09-05T12:26:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febabe00640 1 -- 172.21.15.71:0/958604860 reap_dead start 2024-09-05T12:26:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 shutdown_connections 2024-09-05T12:26:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 >> 172.21.15.71:0/958604860 conn(0x7febac0fd8c0 msgr2=0x7febac0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 shutdown_connections 2024-09-05T12:26:12.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:10.992+0000 7febb2000640 1 -- 172.21.15.71:0/958604860 wait complete. 2024-09-05T12:26:12.469 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:12 smithi071 ceph-mon[29845]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T12:26:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:12 smithi174 ceph-mon[34313]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T12:26:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:12 smithi123 ceph-mon[34321]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-05T12:26:13.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.974+0000 7f4857e00640 1 -- 172.21.15.71:0/1789143963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 msgr2=0x7f4858104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.974+0000 7f4857e00640 1 --2- 172.21.15.71:0/1789143963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f4858104210 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4848009900 tx=0x7f484802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 -- 172.21.15.71:0/1789143963 shutdown_connections 2024-09-05T12:26:13.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 --2- 172.21.15.71:0/1789143963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858108f60 0x7f4858110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 --2- 172.21.15.71:0/1789143963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48581047e0 0x7f4858108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 --2- 172.21.15.71:0/1789143963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f4858104210 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 -- 172.21.15.71:0/1789143963 >> 172.21.15.71:0/1789143963 conn(0x7f48580ff8d0 msgr2=0x7f4858101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 -- 172.21.15.71:0/1789143963 shutdown_connections 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.975+0000 7f4857e00640 1 -- 172.21.15.71:0/1789143963 wait complete. 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 Processor -- start 2024-09-05T12:26:13.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 -- start start 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f485819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48581047e0 0x7f485819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858108f60 0x7f485819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4858114010 con 0x7f4858103e30 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4858113e90 con 0x7f4858108f60 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4858114190 con 0x7f48581047e0 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4856a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f485819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4856a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f485819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34344/0 (socket says 172.21.15.71:34344) 2024-09-05T12:26:13.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4856a00640 1 -- 172.21.15.71:0/2276032855 learned_addr learned my addr 172.21.15.71:0/2276032855 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4856000640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48581047e0 0x7f485819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.976+0000 7f4857400640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858108f60 0x7f485819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f4856a00640 1 -- 172.21.15.71:0/2276032855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48581047e0 msgr2=0x7f485819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f4856a00640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48581047e0 0x7f485819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f4856a00640 1 -- 172.21.15.71:0/2276032855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858108f60 msgr2=0x7f485819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f4856a00640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4858108f60 0x7f485819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f4856a00640 1 -- 172.21.15.71:0/2276032855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485819f480 con 0x7f4858103e30 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f4856a00640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f485819a4d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4848004770 tx=0x7f4848003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.977+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f484803f070 con 0x7f4858103e30 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.978+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4848031ca0 con 0x7f4858103e30 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.978+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f485819f710 con 0x7f4858103e30 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.978+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848035030 con 0x7f4858103e30 2024-09-05T12:26:13.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.978+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48581a7050 con 0x7f4858103e30 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.979+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4858062940 con 0x7f4858103e30 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.980+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4848042070 con 0x7f4858103e30 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f484f400640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f483c0a2c90 0x7f483c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f48480f0310 con 0x7f4858103e30 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f4856000640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f483c0a2c90 0x7f483c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f484f400640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] conn(0x7f483c0ade60 0x7f483c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 --> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f484800b0b0 con 0x7f483c0ade60 2024-09-05T12:26:13.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f48480f06c0 con 0x7f4858103e30 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f4857400640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] conn(0x7f483c0ade60 0x7f483c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.982+0000 7f4856000640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f483c0a2c90 0x7f483c0a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48580ffcf0 tx=0x7f4844005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.983+0000 7f4857400640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] conn(0x7f483c0ade60 0x7f483c0b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.984+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== osd.0 v2:172.21.15.71:6802/245899056 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f484800b0b0 con 0x7f483c0ade60 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.998+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 --> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4858062d70 con 0x7f483c0ade60 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.998+0000 7f484f400640 1 -- 172.21.15.71:0/2276032855 <== osd.0 v2:172.21.15.71:6802/245899056 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4858062d70 con 0x7f483c0ade60 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] conn(0x7f483c0ade60 msgr2=0x7f483c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] conn(0x7f483c0ade60 0x7f483c0b0300 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f483c0a2c90 msgr2=0x7f483c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f483c0a2c90 0x7f483c0a5150 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48580ffcf0 tx=0x7f4844005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:13.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 msgr2=0x7f485819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 --2- 172.21.15.71:0/2276032855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4858103e30 0x7f485819a4d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4848004770 tx=0x7f4848003c10 comp rx=0 tx=0).stop 2024-09-05T12:26:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857400640 1 -- 172.21.15.71:0/2276032855 reap_dead start 2024-09-05T12:26:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 shutdown_connections 2024-09-05T12:26:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 >> 172.21.15.71:0/2276032855 conn(0x7f48580ff8d0 msgr2=0x7f4858100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 shutdown_connections 2024-09-05T12:26:13.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:12.999+0000 7f4857e00640 1 -- 172.21.15.71:0/2276032855 wait complete. 2024-09-05T12:26:13.431 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:13.441 INFO:teuthology.orchestra.run.smithi071.stdout:107374182472 2024-09-05T12:26:13.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-05T12:26:13.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 -- 172.21.15.71:0/950091848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c108f60 msgr2=0x7fc09c110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 --2- 172.21.15.71:0/950091848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c108f60 0x7fc09c110ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc094009ca0 tx=0x7fc09402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 -- 172.21.15.71:0/950091848 shutdown_connections 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 --2- 172.21.15.71:0/950091848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c108f60 0x7fc09c110ae0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 --2- 172.21.15.71:0/950091848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c1047e0 0x7fc09c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 --2- 172.21.15.71:0/950091848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc09c103e30 0x7fc09c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 -- 172.21.15.71:0/950091848 >> 172.21.15.71:0/950091848 conn(0x7fc09c0ff8d0 msgr2=0x7fc09c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 -- 172.21.15.71:0/950091848 shutdown_connections 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.796+0000 7fc0a2200640 1 -- 172.21.15.71:0/950091848 wait complete. 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 Processor -- start 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 -- start start 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c103e30 0x7fc09c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c1047e0 0x7fc09c19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc09c108f60 0x7fc09c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c113fc0 con 0x7fc09c103e30 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c113e40 con 0x7fc09c1047e0 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a2200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc09c114140 con 0x7fc09c108f60 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.797+0000 7fc0a0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c103e30 0x7fc09c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c103e30 0x7fc09c19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34668/0 (socket says 172.21.15.71:34668) 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 -- 172.21.15.71:0/3411996544 learned_addr learned my addr 172.21.15.71:0/3411996544 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc09be00640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c1047e0 0x7fc09c19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a1800640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc09c108f60 0x7fc09c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 -- 172.21.15.71:0/3411996544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc09c108f60 msgr2=0x7fc09c19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc09c108f60 0x7fc09c19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 -- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c1047e0 msgr2=0x7fc09c19a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09c1047e0 0x7fc09c19a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 -- 172.21.15.71:0/3411996544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09c19f380 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.798+0000 7fc0a0e00640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c103e30 0x7fc09c19a3d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc09000e9f0 tx=0x7fc09000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.799+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09000cd80 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.799+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090010910 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.799+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090018990 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.799+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09c19f670 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.799+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09c1a6f50 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.800+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc09c062980 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.802+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc090010ac0 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.803+0000 7fc099600640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc0840a2c90 0x7fc0840a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.803+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fc0900cfec0 con 0x7fc09c103e30 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.803+0000 7fc09be00640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc0840a2c90 0x7fc0840a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.803+0000 7fc099600640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] conn(0x7fc0840ade60 0x7fc0840b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:13.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.803+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 --> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc090019ec0 con 0x7fc0840ade60 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.803+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fc0900d0270 con 0x7fc09c103e30 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.804+0000 7fc0a1800640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] conn(0x7fc0840ade60 0x7fc0840b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.804+0000 7fc09be00640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc0840a2c90 0x7fc0840a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc09c10a8d0 tx=0x7fc08800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.804+0000 7fc0a1800640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] conn(0x7fc0840ade60 0x7fc0840b0300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.805+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== osd.4 v2:172.21.15.123:6808/1661996031 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc090019ec0 con 0x7fc0840ade60 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.819+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 --> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc09c062e50 con 0x7fc0840ade60 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.820+0000 7fc099600640 1 -- 172.21.15.71:0/3411996544 <== osd.4 v2:172.21.15.123:6808/1661996031 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc09c062e50 con 0x7fc0840ade60 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.820+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] conn(0x7fc0840ade60 msgr2=0x7fc0840b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.820+0000 7fc0a2200640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] conn(0x7fc0840ade60 0x7fc0840b0300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc0840a2c90 msgr2=0x7fc0840a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc0840a2c90 0x7fc0840a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc09c10a8d0 tx=0x7fc08800b410 comp rx=0 tx=0).stop 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c103e30 msgr2=0x7fc09c19a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 --2- 172.21.15.71:0/3411996544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc09c103e30 0x7fc09c19a3d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc09000e9f0 tx=0x7fc09000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a1800640 1 -- 172.21.15.71:0/3411996544 reap_dead start 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 shutdown_connections 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 >> 172.21.15.71:0/3411996544 conn(0x7fc09c0ff8d0 msgr2=0x7fc09c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 shutdown_connections 2024-09-05T12:26:13.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:11.821+0000 7fc0a2200640 1 -- 172.21.15.71:0/3411996544 wait complete. 2024-09-05T12:26:14.030 INFO:teuthology.orchestra.run.smithi071.stdout:77309411405 2024-09-05T12:26:14.030 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-05T12:26:14.469 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:14 smithi071 ceph-mon[29845]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-05T12:26:14.508 INFO:teuthology.orchestra.run.smithi071.stdout:137438953539 2024-09-05T12:26:14.509 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-05T12:26:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:14 smithi174 ceph-mon[34313]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-05T12:26:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:14 smithi123 ceph-mon[34321]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-05T12:26:14.682 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:14.683 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:14.683 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:14.686 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:14.689 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:14.690 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:15.121 INFO:teuthology.orchestra.run.smithi071.stdout:34359738457 2024-09-05T12:26:15.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-05T12:26:16.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:16 smithi071 ceph-mon[29845]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-05T12:26:16.482 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:16 smithi123 ceph-mon[34321]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-05T12:26:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:16 smithi174 ceph-mon[34313]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-05T12:26:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 -- 172.21.15.71:0/196305464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5701079c0 msgr2=0x7ff570107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 --2- 172.21.15.71:0/196305464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5701079c0 0x7ff570107da0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff560009930 tx=0x7ff56002a420 comp rx=0 tx=0).stop 2024-09-05T12:26:17.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 -- 172.21.15.71:0/196305464 shutdown_connections 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 --2- 172.21.15.71:0/196305464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff570102360 0x7ff57010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 --2- 172.21.15.71:0/196305464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5701019c0 0x7ff570101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 --2- 172.21.15.71:0/196305464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5701079c0 0x7ff570107da0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 -- 172.21.15.71:0/196305464 >> 172.21.15.71:0/196305464 conn(0x7ff5700fd740 msgr2=0x7ff5700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 -- 172.21.15.71:0/196305464 shutdown_connections 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.356+0000 7ff575400640 1 -- 172.21.15.71:0/196305464 wait complete. 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 Processor -- start 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 -- start start 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5701019c0 0x7ff5700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff570102360 0x7ff570074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5701079c0 0x7ff570074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff570111e10 con 0x7ff570102360 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff570111c90 con 0x7ff5701019c0 2024-09-05T12:26:17.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.357+0000 7ff575400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff570111f90 con 0x7ff5701079c0 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff570102360 0x7ff570074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff570102360 0x7ff570074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34400/0 (socket says 172.21.15.71:34400) 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 -- 172.21.15.71:0/3904795880 learned_addr learned my addr 172.21.15.71:0/3904795880 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56fe00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5701019c0 0x7ff5700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff574a00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5701079c0 0x7ff570074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 -- 172.21.15.71:0/3904795880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5701079c0 msgr2=0x7ff570074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5701079c0 0x7ff570074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 -- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5701019c0 msgr2=0x7ff5700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff5701019c0 0x7ff5700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.358+0000 7ff56f400640 1 -- 172.21.15.71:0/3904795880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5701a67f0 con 0x7ff570102360 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.359+0000 7ff56f400640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff570102360 0x7ff570074a30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00d770 tx=0x7ff55c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.359+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c017070 con 0x7ff570102360 2024-09-05T12:26:17.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.359+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff55c005500 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.359+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5701a6ae0 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.359+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c004e00 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.359+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5701a6fd0 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.360+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff570062900 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.365+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff55c004fa0 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.366+0000 7ff56cc00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff5540a2c10 0x7ff5540a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.366+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff55c0cf2f0 con 0x7ff570102360 2024-09-05T12:26:17.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.366+0000 7ff56fe00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff5540a2c10 0x7ff5540a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.366+0000 7ff56cc00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] conn(0x7ff5540adde0 0x7ff5540b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.366+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 --> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff55c001d30 con 0x7ff5540adde0 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.366+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff55c0cf730 con 0x7ff570102360 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.367+0000 7ff574a00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] conn(0x7ff5540adde0 0x7ff5540b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.367+0000 7ff56fe00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff5540a2c10 0x7ff5540a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff560006010 tx=0x7ff560000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.367+0000 7ff574a00640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] conn(0x7ff5540adde0 0x7ff5540b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:17.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.368+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== osd.10 v2:172.21.15.123:6824/256837648 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff55c001d30 con 0x7ff5540adde0 2024-09-05T12:26:17.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.383+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 --> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff570062d80 con 0x7ff5540adde0 2024-09-05T12:26:17.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff56cc00640 1 -- 172.21.15.71:0/3904795880 <== osd.10 v2:172.21.15.123:6824/256837648 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff570062d80 con 0x7ff5540adde0 2024-09-05T12:26:17.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] conn(0x7ff5540adde0 msgr2=0x7ff5540b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff575400640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] conn(0x7ff5540adde0 0x7ff5540b0280 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff5540a2c10 msgr2=0x7ff5540a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff575400640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff5540a2c10 0x7ff5540a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff560006010 tx=0x7ff560000bc0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff570102360 msgr2=0x7ff570074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff575400640 1 --2- 172.21.15.71:0/3904795880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff570102360 0x7ff570074a30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00d770 tx=0x7ff55c00dc30 comp rx=0 tx=0).stop 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.384+0000 7ff574a00640 1 -- 172.21.15.71:0/3904795880 reap_dead start 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.385+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 shutdown_connections 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.385+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 >> 172.21.15.71:0/3904795880 conn(0x7ff5700fd740 msgr2=0x7ff5700fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.385+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 shutdown_connections 2024-09-05T12:26:17.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:17.385+0000 7ff575400640 1 -- 172.21.15.71:0/3904795880 wait complete. 2024-09-05T12:26:17.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 -- 172.21.15.71:0/3135807298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5601007a0 msgr2=0x7fe560100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 --2- 172.21.15.71:0/3135807298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5601007a0 0x7fe560100b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe550009900 tx=0x7fe55002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 -- 172.21.15.71:0/3135807298 shutdown_connections 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 --2- 172.21.15.71:0/3135807298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56010e350 0x7fe560110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 --2- 172.21.15.71:0/3135807298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe560101150 0x7fe56010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 --2- 172.21.15.71:0/3135807298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5601007a0 0x7fe560100b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.258+0000 7fe564e00640 1 -- 172.21.15.71:0/3135807298 >> 172.21.15.71:0/3135807298 conn(0x7fe5600fc610 msgr2=0x7fe5600fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.259+0000 7fe564e00640 1 -- 172.21.15.71:0/3135807298 shutdown_connections 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.259+0000 7fe564e00640 1 -- 172.21.15.71:0/3135807298 wait complete. 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 Processor -- start 2024-09-05T12:26:17.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 -- start start 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5601007a0 0x7fe56019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe560101150 0x7fe56019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe56010e350 0x7fe56019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe560113fd0 con 0x7fe560101150 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe560113e50 con 0x7fe5601007a0 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.260+0000 7fe564e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe560114150 con 0x7fe56010e350 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe560101150 0x7fe56019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe560101150 0x7fe56019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34360/0 (socket says 172.21.15.71:34360) 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 -- 172.21.15.71:0/1507499162 learned_addr learned my addr 172.21.15.71:0/1507499162 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55f400640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5601007a0 0x7fe56019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55fe00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe56010e350 0x7fe56019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 -- 172.21.15.71:0/1507499162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe56010e350 msgr2=0x7fe56019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe56010e350 0x7fe56019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 -- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5601007a0 msgr2=0x7fe56019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5601007a0 0x7fe56019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 -- 172.21.15.71:0/1507499162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56019f490 con 0x7fe560101150 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.261+0000 7fe55ea00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe560101150 0x7fe56019aa20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00ea60 tx=0x7fe54c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.262+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00cde0 con 0x7fe560101150 2024-09-05T12:26:17.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.262+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c0035f0 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.262+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56019f780 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.262+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c010670 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.262+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5601a7060 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.263+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe560062980 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.265+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe54c0037a0 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.267+0000 7fe557e00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe5440a2c90 0x7fe5440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.267+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe54c0cfb60 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.267+0000 7fe55f400640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe5440a2c90 0x7fe5440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.267+0000 7fe557e00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] conn(0x7fe5440ade60 0x7fe5440b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.267+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 --> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe54c023e50 con 0x7fe5440ade60 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.267+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fe54c0cff10 con 0x7fe560101150 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.268+0000 7fe55fe00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] conn(0x7fe5440ade60 0x7fe5440b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:17.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.268+0000 7fe55f400640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe5440a2c90 0x7fe5440a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe550009230 tx=0x7fe5500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.268+0000 7fe55fe00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] conn(0x7fe5440ade60 0x7fe5440b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.269+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== osd.7 v2:172.21.15.123:6816/83143340 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe54c023e50 con 0x7fe5440ade60 2024-09-05T12:26:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.283+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 --> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe560062d80 con 0x7fe5440ade60 2024-09-05T12:26:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe557e00640 1 -- 172.21.15.71:0/1507499162 <== osd.7 v2:172.21.15.123:6816/83143340 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe560062d80 con 0x7fe5440ade60 2024-09-05T12:26:17.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] conn(0x7fe5440ade60 msgr2=0x7fe5440b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe564e00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] conn(0x7fe5440ade60 0x7fe5440b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe5440a2c90 msgr2=0x7fe5440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe564e00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe5440a2c90 0x7fe5440a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe550009230 tx=0x7fe5500058d0 comp rx=0 tx=0).stop 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe560101150 msgr2=0x7fe56019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.284+0000 7fe564e00640 1 --2- 172.21.15.71:0/1507499162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe560101150 0x7fe56019aa20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00ea60 tx=0x7fe54c00ef20 comp rx=0 tx=0).stop 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.285+0000 7fe55fe00640 1 -- 172.21.15.71:0/1507499162 reap_dead start 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.285+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 shutdown_connections 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.285+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 >> 172.21.15.71:0/1507499162 conn(0x7fe5600fc610 msgr2=0x7fe5600fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.285+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 shutdown_connections 2024-09-05T12:26:17.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.285+0000 7fe564e00640 1 -- 172.21.15.71:0/1507499162 wait complete. 2024-09-05T12:26:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 -- 172.21.15.71:0/1866101326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b98074ee0 msgr2=0x7f8b9810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 --2- 172.21.15.71:0/1866101326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b98074ee0 0x7f8b9810dd20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009860 tx=0x7f8b8402a3d0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 -- 172.21.15.71:0/1866101326 shutdown_connections 2024-09-05T12:26:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 --2- 172.21.15.71:0/1866101326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b9810e260 0x7f8b98110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 --2- 172.21.15.71:0/1866101326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b98074ee0 0x7f8b9810dd20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 --2- 172.21.15.71:0/1866101326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b980745c0 0x7f8b980749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 -- 172.21.15.71:0/1866101326 >> 172.21.15.71:0/1866101326 conn(0x7f8b980fd740 msgr2=0x7f8b980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 -- 172.21.15.71:0/1866101326 shutdown_connections 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.938+0000 7f8b97200640 1 -- 172.21.15.71:0/1866101326 wait complete. 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.939+0000 7f8b97200640 1 Processor -- start 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.939+0000 7f8b97200640 1 -- start start 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b97200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b980745c0 0x7f8b9819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b97200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b98074ee0 0x7f8b9819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b97200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b9810e260 0x7f8b9819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b97200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98114000 con 0x7f8b9810e260 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b97200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98113e80 con 0x7f8b980745c0 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b97200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b98114180 con 0x7f8b98074ee0 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b96800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b9810e260 0x7f8b9819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:18.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b96800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b9810e260 0x7f8b9819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34382/0 (socket says 172.21.15.71:34382) 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b96800640 1 -- 172.21.15.71:0/2415394076 learned_addr learned my addr 172.21.15.71:0/2415394076 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b95400640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b98074ee0 0x7f8b9819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.940+0000 7f8b95e00640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b980745c0 0x7f8b9819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b96800640 1 -- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b98074ee0 msgr2=0x7f8b9819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b96800640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b98074ee0 0x7f8b9819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b96800640 1 -- 172.21.15.71:0/2415394076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b980745c0 msgr2=0x7f8b9819a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b96800640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b980745c0 0x7f8b9819a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b96800640 1 -- 172.21.15.71:0/2415394076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9819f470 con 0x7f8b9810e260 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b96800640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b9810e260 0x7f8b9819ed90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000eb40 tx=0x7f8b9000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9000ce40 con 0x7f8b9810e260 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b90005020 con 0x7f8b9810e260 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9819f760 con 0x7f8b9810e260 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b90010690 con 0x7f8b9810e260 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.941+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b981a7040 con 0x7f8b9810e260 2024-09-05T12:26:18.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.942+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b98062980 con 0x7f8b9810e260 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.946+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b900051d0 con 0x7f8b9810e260 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.947+0000 7f8b8ea00640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b7c0a2c90 0x7f8b7c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.947+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f8b900cfc50 con 0x7f8b9810e260 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.947+0000 7f8b95e00640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b7c0a2c90 0x7f8b7c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.947+0000 7f8b8ea00640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] conn(0x7f8b7c0ade60 0x7f8b7c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.947+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 --> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b90016e50 con 0x7f8b7c0ade60 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.947+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f8b900d2270 con 0x7f8b9810e260 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.948+0000 7f8b95400640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] conn(0x7f8b7c0ade60 0x7f8b7c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.948+0000 7f8b95e00640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b7c0a2c90 0x7f8b7c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005fd0 tx=0x7f8b88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:18.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.948+0000 7f8b95400640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] conn(0x7f8b7c0ade60 0x7f8b7c0b0300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.949+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== osd.8 v2:172.21.15.174:6816/2584468270 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b90016e50 con 0x7f8b7c0ade60 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.967+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 --> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b98062e00 con 0x7f8b7c0ade60 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b8ea00640 1 -- 172.21.15.71:0/2415394076 <== osd.8 v2:172.21.15.174:6816/2584468270 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b98062e00 con 0x7f8b7c0ade60 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] conn(0x7f8b7c0ade60 msgr2=0x7f8b7c0b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] conn(0x7f8b7c0ade60 0x7f8b7c0b0300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b7c0a2c90 msgr2=0x7f8b7c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b7c0a2c90 0x7f8b7c0a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8b88005fd0 tx=0x7f8b88009260 comp rx=0 tx=0).stop 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b9810e260 msgr2=0x7f8b9819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:18.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 --2- 172.21.15.71:0/2415394076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b9810e260 0x7f8b9819ed90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8b9000eb40 tx=0x7f8b9000ef40 comp rx=0 tx=0).stop 2024-09-05T12:26:18.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b96800640 1 -- 172.21.15.71:0/2415394076 reap_dead start 2024-09-05T12:26:18.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 shutdown_connections 2024-09-05T12:26:18.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.968+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 >> 172.21.15.71:0/2415394076 conn(0x7f8b980fd740 msgr2=0x7f8b980fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:18.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.969+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 shutdown_connections 2024-09-05T12:26:18.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:16.969+0000 7f8b97200640 1 -- 172.21.15.71:0/2415394076 wait complete. 2024-09-05T12:26:18.325 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:18 smithi071 ceph-mon[29845]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:18 smithi174 ceph-mon[34313]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:18 smithi123 ceph-mon[34321]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:19.186 INFO:teuthology.orchestra.run.smithi071.stdout:249108103215 2024-09-05T12:26:19.187 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-05T12:26:19.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 -- 172.21.15.71:0/795590596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d3010e420 msgr2=0x7f2d30110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:19.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 --2- 172.21.15.71:0/795590596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d3010e420 0x7f2d30110810 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009be0 tx=0x7f2d2802a750 comp rx=0 tx=0).stop 2024-09-05T12:26:19.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 -- 172.21.15.71:0/795590596 shutdown_connections 2024-09-05T12:26:19.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 --2- 172.21.15.71:0/795590596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d3010e420 0x7f2d30110810 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 --2- 172.21.15.71:0/795590596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d30074ee0 0x7f2d3010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 --2- 172.21.15.71:0/795590596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d300745c0 0x7f2d300749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 -- 172.21.15.71:0/795590596 >> 172.21.15.71:0/795590596 conn(0x7f2d300fd8c0 msgr2=0x7f2d300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 -- 172.21.15.71:0/795590596 shutdown_connections 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.421+0000 7f2d37600640 1 -- 172.21.15.71:0/795590596 wait complete. 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.422+0000 7f2d37600640 1 Processor -- start 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.422+0000 7f2d37600640 1 -- start start 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d37600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d300745c0 0x7f2d3019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d37600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d30074ee0 0x7f2d3019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d37600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d3010e420 0x7f2d3019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d37600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30114150 con 0x7f2d30074ee0 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d37600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30113fd0 con 0x7f2d3010e420 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d37600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301142d0 con 0x7f2d300745c0 2024-09-05T12:26:19.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d2fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d30074ee0 0x7f2d3019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d34c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d300745c0 0x7f2d3019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d35600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d3010e420 0x7f2d3019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d34c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d300745c0 0x7f2d3019a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39220/0 (socket says 172.21.15.71:39220) 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d34c00640 1 -- 172.21.15.71:0/198183931 learned_addr learned my addr 172.21.15.71:0/198183931 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.423+0000 7f2d2fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d30074ee0 0x7f2d3019ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34438/0 (socket says 172.21.15.71:34438) 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d35600640 1 -- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d300745c0 msgr2=0x7f2d3019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d35600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d300745c0 0x7f2d3019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d35600640 1 -- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d30074ee0 msgr2=0x7f2d3019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d35600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2d30074ee0 0x7f2d3019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d35600640 1 -- 172.21.15.71:0/198183931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3019f680 con 0x7f2d3010e420 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d35600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d3010e420 0x7f2d3019efa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d28034040 tx=0x7f2d280305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28041020 con 0x7f2d3010e420 2024-09-05T12:26:19.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d28003e50 con 0x7f2d3010e420 2024-09-05T12:26:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3019f910 con 0x7f2d3010e420 2024-09-05T12:26:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28036040 con 0x7f2d3010e420 2024-09-05T12:26:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.424+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d3019fdc0 con 0x7f2d3010e420 2024-09-05T12:26:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.425+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2d30062900 con 0x7f2d3010e420 2024-09-05T12:26:19.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.428+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d28044020 con 0x7f2d3010e420 2024-09-05T12:26:19.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.430+0000 7f2d2d600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2d180a2c10 0x7f2d180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:19.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.430+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f2d280f03e0 con 0x7f2d3010e420 2024-09-05T12:26:19.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.430+0000 7f2d34c00640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2d180a2c10 0x7f2d180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:19.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.430+0000 7f2d2d600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] conn(0x7f2d180adde0 0x7f2d180b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:19.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.430+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 --> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d28035e50 con 0x7f2d180adde0 2024-09-05T12:26:19.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.430+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f2d280f0790 con 0x7f2d3010e420 2024-09-05T12:26:19.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.431+0000 7f2d2fe00640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] conn(0x7f2d180adde0 0x7f2d180b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:19.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.431+0000 7f2d34c00640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2d180a2c10 0x7f2d180a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009c90 tx=0x7f2d24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:19.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.431+0000 7f2d2fe00640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] conn(0x7f2d180adde0 0x7f2d180b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:19.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.432+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== osd.11 v2:172.21.15.174:6824/1335368098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2d28035e50 con 0x7f2d180adde0 2024-09-05T12:26:19.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.448+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 --> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2d30062d80 con 0x7f2d180adde0 2024-09-05T12:26:19.449 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.448+0000 7f2d2d600640 1 -- 172.21.15.71:0/198183931 <== osd.11 v2:172.21.15.174:6824/1335368098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2d30062d80 con 0x7f2d180adde0 2024-09-05T12:26:19.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.449+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] conn(0x7f2d180adde0 msgr2=0x7f2d180b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:19.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.449+0000 7f2d37600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] conn(0x7f2d180adde0 0x7f2d180b0280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2d180a2c10 msgr2=0x7f2d180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:19.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f2d180a2c10 0x7f2d180a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009c90 tx=0x7f2d24005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:19.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d3010e420 msgr2=0x7f2d3019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:19.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 --2- 172.21.15.71:0/198183931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d3010e420 0x7f2d3019efa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2d28034040 tx=0x7f2d280305d0 comp rx=0 tx=0).stop 2024-09-05T12:26:19.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d35600640 1 -- 172.21.15.71:0/198183931 reap_dead start 2024-09-05T12:26:19.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 shutdown_connections 2024-09-05T12:26:19.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 >> 172.21.15.71:0/198183931 conn(0x7f2d300fd8c0 msgr2=0x7f2d300fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:19.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.450+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 shutdown_connections 2024-09-05T12:26:19.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.451+0000 7f2d37600640 1 -- 172.21.15.71:0/198183931 wait complete. 2024-09-05T12:26:20.128 INFO:teuthology.orchestra.run.smithi071.stdout:304942678052 2024-09-05T12:26:20.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-05T12:26:20.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:20 smithi071 ceph-mon[29845]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:20.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 -- 172.21.15.71:0/1521434152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 msgr2=0x7f46001008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 --2- 172.21.15.71:0/1521434152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f46001008e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009900 tx=0x7f45f402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 -- 172.21.15.71:0/1521434152 shutdown_connections 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 --2- 172.21.15.71:0/1521434152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f70 0x7f460010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 --2- 172.21.15.71:0/1521434152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4600100eb0 0x7f4600106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 --2- 172.21.15.71:0/1521434152 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f46001008e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 -- 172.21.15.71:0/1521434152 >> 172.21.15.71:0/1521434152 conn(0x7f46000fc5c0 msgr2=0x7f46000fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 -- 172.21.15.71:0/1521434152 shutdown_connections 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.898+0000 7f4607600640 1 -- 172.21.15.71:0/1521434152 wait complete. 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 Processor -- start 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 -- start start 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f4600101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600100eb0 0x7f46001019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4600106f70 0x7f4600101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4600111fe0 con 0x7f4600100500 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4600111e60 con 0x7f4600106f70 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4607600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4600112160 con 0x7f4600100eb0 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4604c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f4600101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4604c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f4600101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34420/0 (socket says 172.21.15.71:34420) 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f4604c00640 1 -- 172.21.15.71:0/2334003830 learned_addr learned my addr 172.21.15.71:0/2334003830 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.899+0000 7f45ffe00640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600100eb0 0x7f46001019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4605600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4600106f70 0x7f4600101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4604c00640 1 -- 172.21.15.71:0/2334003830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600100eb0 msgr2=0x7f46001019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4604c00640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600100eb0 0x7f46001019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4604c00640 1 -- 172.21.15.71:0/2334003830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4600106f70 msgr2=0x7f4600101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4604c00640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4600106f70 0x7f4600101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4604c00640 1 -- 172.21.15.71:0/2334003830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001a6940 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4604c00640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f4600101480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45f402a910 tx=0x7f45f4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f403f070 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f4031c90 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f4035030 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.900+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001a6bd0 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.901+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a7060 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.901+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4600062880 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.904+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f45f4042070 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.905+0000 7f45fd600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e80a2c10 0x7f45e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.905+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f45f40effe0 con 0x7f4600100500 2024-09-05T12:26:20.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.905+0000 7f45ffe00640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e80a2c10 0x7f45e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.905+0000 7f45fd600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] conn(0x7f45e80adde0 0x7f45e80b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.905+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 --> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45f4038e90 con 0x7f45e80adde0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.905+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f45f40f0410 con 0x7f4600100500 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.906+0000 7f4605600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] conn(0x7f45e80adde0 0x7f45e80b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.906+0000 7f45ffe00640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e80a2c10 0x7f45e80a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4600102f10 tx=0x7f45ec00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.906+0000 7f4605600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] conn(0x7f45e80adde0 0x7f45e80b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.907+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== osd.9 v2:172.21.15.71:6826/4063696611 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45f4038e90 con 0x7f45e80adde0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.922+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 --> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4600062c80 con 0x7f45e80adde0 2024-09-05T12:26:20.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.922+0000 7f45fd600640 1 -- 172.21.15.71:0/2334003830 <== osd.9 v2:172.21.15.71:6826/4063696611 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4600062c80 con 0x7f45e80adde0 2024-09-05T12:26:20.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] conn(0x7f45e80adde0 msgr2=0x7f45e80b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] conn(0x7f45e80adde0 0x7f45e80b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e80a2c10 msgr2=0x7f45e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e80a2c10 0x7f45e80a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4600102f10 tx=0x7f45ec00b410 comp rx=0 tx=0).stop 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 msgr2=0x7f4600101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 --2- 172.21.15.71:0/2334003830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600100500 0x7f4600101480 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f45f402a910 tx=0x7f45f4003b90 comp rx=0 tx=0).stop 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4605600640 1 -- 172.21.15.71:0/2334003830 reap_dead start 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 shutdown_connections 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 >> 172.21.15.71:0/2334003830 conn(0x7f46000fc5c0 msgr2=0x7f46001051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 shutdown_connections 2024-09-05T12:26:20.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:18.923+0000 7f4607600640 1 -- 172.21.15.71:0/2334003830 wait complete. 2024-09-05T12:26:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:20 smithi174 ceph-mon[34313]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:20 smithi123 ceph-mon[34321]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:20.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 -- 172.21.15.71:0/3218000513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68109140 msgr2=0x7f8b68110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/3218000513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68109140 0x7f8b68110cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009ca0 tx=0x7f8b6002a8b0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 -- 172.21.15.71:0/3218000513 shutdown_connections 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/3218000513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68109140 0x7f8b68110cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/3218000513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b681049c0 0x7f8b68108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/3218000513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b68104010 0x7f8b681043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 -- 172.21.15.71:0/3218000513 >> 172.21.15.71:0/3218000513 conn(0x7f8b680ffa50 msgr2=0x7f8b68101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 -- 172.21.15.71:0/3218000513 shutdown_connections 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.858+0000 7f8b6ec00640 1 -- 172.21.15.71:0/3218000513 wait complete. 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 Processor -- start 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 -- start start 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68104010 0x7f8b6819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b681049c0 0x7f8b6819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b68109140 0x7f8b6819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68114170 con 0x7f8b68104010 2024-09-05T12:26:20.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b68113ff0 con 0x7f8b681049c0 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.859+0000 7f8b6ec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b681142f0 con 0x7f8b68109140 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68104010 0x7f8b6819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68104010 0x7f8b6819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34458/0 (socket says 172.21.15.71:34458) 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 -- 172.21.15.71:0/247105319 learned_addr learned my addr 172.21.15.71:0/247105319 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b5fe00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b681049c0 0x7f8b6819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b6cc00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b68109140 0x7f8b6819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 -- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b68109140 msgr2=0x7f8b6819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b68109140 0x7f8b6819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 -- 172.21.15.71:0/247105319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b681049c0 msgr2=0x7f8b6819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b681049c0 0x7f8b6819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 -- 172.21.15.71:0/247105319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6819f5c0 con 0x7f8b68104010 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.860+0000 7f8b67e00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68104010 0x7f8b6819a610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800e9f0 tx=0x7f8b5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.861+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800cd80 con 0x7f8b68104010 2024-09-05T12:26:20.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.861+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58005500 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.861+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6819f8b0 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.861+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58010690 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.861+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a7190 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.862+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8b68062980 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.863+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8b580046c0 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.865+0000 7f8b65600640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b4c0a2c10 0x7f8b4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.865+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f8b580cfa20 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.865+0000 7f8b5fe00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b4c0a2c10 0x7f8b4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.865+0000 7f8b65600640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] conn(0x7f8b4c0adde0 0x7f8b4c0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.865+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 --> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b4c0b0950 con 0x7f8b4c0adde0 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.865+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f8b580d42a0 con 0x7f8b68104010 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.866+0000 7f8b6cc00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] conn(0x7f8b4c0adde0 0x7f8b4c0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.866+0000 7f8b5fe00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b4c0a2c10 0x7f8b4c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b500095f0 tx=0x7f8b50005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:20.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.866+0000 7f8b6cc00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] conn(0x7f8b4c0adde0 0x7f8b4c0b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.867+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== osd.5 v2:172.21.15.174:6808/2379389937 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8b4c0b0950 con 0x7f8b4c0adde0 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.881+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 --> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8b68062e00 con 0x7f8b4c0adde0 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b65600640 1 -- 172.21.15.71:0/247105319 <== osd.5 v2:172.21.15.174:6808/2379389937 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8b68062e00 con 0x7f8b4c0adde0 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] conn(0x7f8b4c0adde0 msgr2=0x7f8b4c0b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] conn(0x7f8b4c0adde0 0x7f8b4c0b0280 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b4c0a2c10 msgr2=0x7f8b4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f8b4c0a2c10 0x7f8b4c0a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8b500095f0 tx=0x7f8b50005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68104010 msgr2=0x7f8b6819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 --2- 172.21.15.71:0/247105319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b68104010 0x7f8b6819a610 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800e9f0 tx=0x7f8b5800edf0 comp rx=0 tx=0).stop 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6cc00640 1 -- 172.21.15.71:0/247105319 reap_dead start 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 shutdown_connections 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.882+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 >> 172.21.15.71:0/247105319 conn(0x7f8b680ffa50 msgr2=0x7f8b68100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.883+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 shutdown_connections 2024-09-05T12:26:20.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:19.883+0000 7f8b6ec00640 1 -- 172.21.15.71:0/247105319 wait complete. 2024-09-05T12:26:21.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 -- 172.21.15.71:0/3548225949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c074eb0 msgr2=0x7f5e6c1124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:21.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 --2- 172.21.15.71:0/3548225949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c074eb0 0x7f5e6c1124a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802a480 comp rx=0 tx=0).stop 2024-09-05T12:26:21.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 -- 172.21.15.71:0/3548225949 shutdown_connections 2024-09-05T12:26:21.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 --2- 172.21.15.71:0/3548225949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e6c1129e0 0x7f5e6c114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 --2- 172.21.15.71:0/3548225949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c074eb0 0x7f5e6c1124a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 --2- 172.21.15.71:0/3548225949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 0x7f5e6c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 -- 172.21.15.71:0/3548225949 >> 172.21.15.71:0/3548225949 conn(0x7f5e6c0fd710 msgr2=0x7f5e6c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 -- 172.21.15.71:0/3548225949 shutdown_connections 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.432+0000 7f5e72000640 1 -- 172.21.15.71:0/3548225949 wait complete. 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.433+0000 7f5e72000640 1 Processor -- start 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.433+0000 7f5e72000640 1 -- start start 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e72000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 0x7f5e6c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e72000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e6c074eb0 0x7f5e6c1a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e72000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c1129e0 0x7f5e6c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e72000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c1183e0 con 0x7f5e6c074eb0 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e72000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c118260 con 0x7f5e6c1129e0 2024-09-05T12:26:21.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e72000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6c118560 con 0x7f5e6c074590 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e70c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 0x7f5e6c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e70c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 0x7f5e6c1a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39256/0 (socket says 172.21.15.71:39256) 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e70c00640 1 -- 172.21.15.71:0/2990373658 learned_addr learned my addr 172.21.15.71:0/2990373658 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.434+0000 7f5e71600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c1129e0 0x7f5e6c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e70c00640 1 -- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c1129e0 msgr2=0x7f5e6c19e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e70c00640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e6c1129e0 0x7f5e6c19e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e70c00640 1 -- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e6c074eb0 msgr2=0x7f5e6c1a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e70c00640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e6c074eb0 0x7f5e6c1a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e70c00640 1 -- 172.21.15.71:0/2990373658 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e6c19ef70 con 0x7f5e6c074590 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e70c00640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 0x7f5e6c1a4630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000e9f0 tx=0x7f5e6000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e6000cd80 con 0x7f5e6c074590 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e60010910 con 0x7f5e6c074590 2024-09-05T12:26:21.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60018990 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.435+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e6c19f260 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.436+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e6c1ab3e0 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.437+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e6c0ff930 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.439+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5e60010ac0 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.440+0000 7f5e69600640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5e540a2c90 0x7f5e540a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.440+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f5e600cff70 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.440+0000 7f5e6be00640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5e540a2c90 0x7f5e540a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.440+0000 7f5e69600640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] conn(0x7f5e540ade60 0x7f5e540b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.440+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 --> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e60019ec0 con 0x7f5e540ade60 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.440+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f5e600d0320 con 0x7f5e6c074590 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.441+0000 7f5e71600640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] conn(0x7f5e540ade60 0x7f5e540b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.441+0000 7f5e6be00640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5e540a2c90 0x7f5e540a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5e580096c0 tx=0x7f5e58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.441+0000 7f5e71600640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] conn(0x7f5e540ade60 0x7f5e540b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:21.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.442+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== osd.1 v2:172.21.15.123:6800/1018377338 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e60019ec0 con 0x7f5e540ade60 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.456+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 --> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e6c075cd0 con 0x7f5e540ade60 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e69600640 1 -- 172.21.15.71:0/2990373658 <== osd.1 v2:172.21.15.123:6800/1018377338 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e6c075cd0 con 0x7f5e540ade60 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] conn(0x7f5e540ade60 msgr2=0x7f5e540b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e72000640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] conn(0x7f5e540ade60 0x7f5e540b0300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5e540a2c90 msgr2=0x7f5e540a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e72000640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f5e540a2c90 0x7f5e540a5150 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5e580096c0 tx=0x7f5e58000bc0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 msgr2=0x7f5e6c1a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:21.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e72000640 1 --2- 172.21.15.71:0/2990373658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e6c074590 0x7f5e6c1a4630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000e9f0 tx=0x7f5e6000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:26:21.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.457+0000 7f5e71600640 1 -- 172.21.15.71:0/2990373658 reap_dead start 2024-09-05T12:26:21.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.458+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 shutdown_connections 2024-09-05T12:26:21.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.458+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 >> 172.21.15.71:0/2990373658 conn(0x7f5e6c0fd710 msgr2=0x7f5e6c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:21.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.458+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 shutdown_connections 2024-09-05T12:26:21.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:20.458+0000 7f5e72000640 1 -- 172.21.15.71:0/2990373658 wait complete. 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 -- 172.21.15.71:0/1775286442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c109120 msgr2=0x7f1b5c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1775286442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c109120 0x7f1b5c110ca0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009ca0 tx=0x7f1b5402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 -- 172.21.15.71:0/1775286442 shutdown_connections 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1775286442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c109120 0x7f1b5c110ca0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1775286442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5c1049a0 0x7f1b5c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1775286442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 0x7f1b5c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.367+0000 7f1b60e00640 1 -- 172.21.15.71:0/1775286442 >> 172.21.15.71:0/1775286442 conn(0x7f1b5c0ffa90 msgr2=0x7f1b5c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.368+0000 7f1b60e00640 1 -- 172.21.15.71:0/1775286442 shutdown_connections 2024-09-05T12:26:22.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.368+0000 7f1b60e00640 1 -- 172.21.15.71:0/1775286442 wait complete. 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.369+0000 7f1b60e00640 1 Processor -- start 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.369+0000 7f1b60e00640 1 -- start start 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.369+0000 7f1b60e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 0x7f1b5c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.369+0000 7f1b60e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c1049a0 0x7f1b5c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b60e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5c109120 0x7f1b5c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b60e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c114150 con 0x7f1b5c1049a0 2024-09-05T12:26:22.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b60e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c113fd0 con 0x7f1b5c109120 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b60e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c1142d0 con 0x7f1b5c103ff0 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b51400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c1049a0 0x7f1b5c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 0x7f1b5c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 0x7f1b5c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39264/0 (socket says 172.21.15.71:39264) 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 -- 172.21.15.71:0/1517512691 learned_addr learned my addr 172.21.15.71:0/1517512691 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b5a800640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5c109120 0x7f1b5c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 -- 172.21.15.71:0/1517512691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5c109120 msgr2=0x7f1b5c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5c109120 0x7f1b5c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 -- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c1049a0 msgr2=0x7f1b5c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c1049a0 0x7f1b5c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b59e00640 1 -- 172.21.15.71:0/1517512691 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c19f5c0 con 0x7f1b5c103ff0 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b5a800640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5c109120 0x7f1b5c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:26:22.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.370+0000 7f1b51400640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b5c1049a0 0x7f1b5c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:22.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.371+0000 7f1b59e00640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 0x7f1b5c19a610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00e9f0 tx=0x7f1b4c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:22.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.371+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c00cd80 con 0x7f1b5c103ff0 2024-09-05T12:26:22.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.371+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b4c005500 con 0x7f1b5c103ff0 2024-09-05T12:26:22.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.371+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c19f8b0 con 0x7f1b5c103ff0 2024-09-05T12:26:22.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.371+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c010690 con 0x7f1b5c103ff0 2024-09-05T12:26:22.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.371+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c1a7190 con 0x7f1b5c103ff0 2024-09-05T12:26:22.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.372+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1b5c062980 con 0x7f1b5c103ff0 2024-09-05T12:26:22.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.374+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1b4c0046c0 con 0x7f1b5c103ff0 2024-09-05T12:26:22.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.375+0000 7f1b53400640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b400a2c90 0x7f1b400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:22.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f1b4c0cfe80 con 0x7f1b5c103ff0 2024-09-05T12:26:22.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b51400640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b400a2c90 0x7f1b400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:22.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b53400640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] conn(0x7f1b400ade60 0x7f1b400b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:22.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 --> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1b4c01aec0 con 0x7f1b400ade60 2024-09-05T12:26:22.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f1b4c0d0230 con 0x7f1b5c103ff0 2024-09-05T12:26:22.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b5a800640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] conn(0x7f1b400ade60 0x7f1b400b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:22.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.376+0000 7f1b51400640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b400a2c90 0x7f1b400a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0ffeb0 tx=0x7f1b44005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:22.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.377+0000 7f1b5a800640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] conn(0x7f1b400ade60 0x7f1b400b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:22.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.378+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== osd.6 v2:172.21.15.71:6818/1490057676 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1b4c01aec0 con 0x7f1b400ade60 2024-09-05T12:26:22.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.395+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 --> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1b5c062d80 con 0x7f1b400ade60 2024-09-05T12:26:22.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b53400640 1 -- 172.21.15.71:0/1517512691 <== osd.6 v2:172.21.15.71:6818/1490057676 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1b5c062d80 con 0x7f1b400ade60 2024-09-05T12:26:22.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] conn(0x7f1b400ade60 msgr2=0x7f1b400b0300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:22.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] conn(0x7f1b400ade60 0x7f1b400b0300 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b400a2c90 msgr2=0x7f1b400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:22.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b400a2c90 0x7f1b400a5150 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c0ffeb0 tx=0x7f1b44005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:22.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 msgr2=0x7f1b5c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 --2- 172.21.15.71:0/1517512691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b5c103ff0 0x7f1b5c19a610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c00e9f0 tx=0x7f1b4c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:26:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b5a800640 1 -- 172.21.15.71:0/1517512691 reap_dead start 2024-09-05T12:26:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 shutdown_connections 2024-09-05T12:26:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 >> 172.21.15.71:0/1517512691 conn(0x7f1b5c0ffa90 msgr2=0x7f1b5c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 shutdown_connections 2024-09-05T12:26:22.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:22.396+0000 7f1b60e00640 1 -- 172.21.15.71:0/1517512691 wait complete. 2024-09-05T12:26:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:22 smithi174 ceph-mon[34313]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:22.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:22 smithi071 ceph-mon[29845]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:22 smithi123 ceph-mon[34321]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:23.295 INFO:teuthology.orchestra.run.smithi071.stdout:335007449121 2024-09-05T12:26:23.295 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-05T12:26:24.092 INFO:teuthology.orchestra.run.smithi071.stdout:55834574934 2024-09-05T12:26:24.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-05T12:26:24.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:24 smithi071 ceph-mon[29845]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-05T12:26:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:24 smithi174 ceph-mon[34313]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-05T12:26:24.567 INFO:teuthology.orchestra.run.smithi071.stdout:214748364852 2024-09-05T12:26:24.568 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-05T12:26:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:24 smithi123 ceph-mon[34321]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-09-05T12:26:24.931 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:24.975 INFO:teuthology.orchestra.run.smithi071.stdout:283467841577 2024-09-05T12:26:24.975 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-05T12:26:25.577 INFO:teuthology.orchestra.run.smithi071.stdout:158913790016 2024-09-05T12:26:25.578 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-05T12:26:25.943 INFO:teuthology.orchestra.run.smithi071.stdout:193273528378 2024-09-05T12:26:25.943 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-05T12:26:26.002 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:26.022 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:26.023 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:26.023 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:26.506 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:26 smithi071 ceph-mon[29845]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T12:26:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:26 smithi174 ceph-mon[34313]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T12:26:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:26 smithi123 ceph-mon[34321]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.619+0000 7f3f43200640 1 -- 172.21.15.71:0/3657606470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 msgr2=0x7f3f44110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 --2- 172.21.15.71:0/3657606470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f44110670 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009ca0 tx=0x7f3f3c02a950 comp rx=0 tx=0).stop 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 -- 172.21.15.71:0/3657606470 shutdown_connections 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 --2- 172.21.15.71:0/3657606470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f44110670 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 --2- 172.21.15.71:0/3657606470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f44074e30 0x7f3f4410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 --2- 172.21.15.71:0/3657606470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074510 0x7f3f440748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 -- 172.21.15.71:0/3657606470 >> 172.21.15.71:0/3657606470 conn(0x7f3f440fd6e0 msgr2=0x7f3f440ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 -- 172.21.15.71:0/3657606470 shutdown_connections 2024-09-05T12:26:26.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.620+0000 7f3f43200640 1 -- 172.21.15.71:0/3657606470 wait complete. 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 Processor -- start 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 -- start start 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f44074510 0x7f3f4419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 0x7f3f4419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f4419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44113f90 con 0x7f3f4410e280 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44113e10 con 0x7f3f44074e30 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f43200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f44114110 con 0x7f3f44074510 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f41400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 0x7f3f4419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f42800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f4419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f41e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f44074510 0x7f3f4419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f41400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 0x7f3f4419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:38224/0 (socket says 172.21.15.71:38224) 2024-09-05T12:26:26.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.621+0000 7f3f41400640 1 -- 172.21.15.71:0/2490488902 learned_addr learned my addr 172.21.15.71:0/2490488902 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f41400640 1 -- 172.21.15.71:0/2490488902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f44074510 msgr2=0x7f3f4419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f41400640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f44074510 0x7f3f4419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f41400640 1 -- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 msgr2=0x7f3f4419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f41400640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f4419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f41400640 1 -- 172.21.15.71:0/2490488902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4419f400 con 0x7f3f44074e30 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f42800640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f4419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f41400640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 0x7f3f4419a990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000e9f0 tx=0x7f3f3000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f3000cd80 con 0x7f3f44074e30 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4419f6f0 con 0x7f3f44074e30 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f30005500 con 0x7f3f44074e30 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.622+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f30010690 con 0x7f3f44074e30 2024-09-05T12:26:26.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.623+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f441a6fd0 con 0x7f3f44074e30 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.624+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f440748f0 con 0x7f3f44074e30 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.626+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3f300046c0 con 0x7f3f44074e30 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.627+0000 7f3f3aa00640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3f280a2c10 0x7f3f280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.627+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f3f300cf950 con 0x7f3f44074e30 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.627+0000 7f3f41e00640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3f280a2c10 0x7f3f280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.628+0000 7f3f41e00640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3f280a2c10 0x7f3f280a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009bd0 tx=0x7f3f34005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.629+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f3009c390 con 0x7f3f44074e30 2024-09-05T12:26:26.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.816+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3f4419b7b0 con 0x7f3f44074e30 2024-09-05T12:26:26.971 INFO:teuthology.orchestra.run.smithi071.stdout:77309411408 2024-09-05T12:26:26.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.817+0000 7f3f3aa00640 1 -- 172.21.15.71:0/2490488902 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3f3009bc90 con 0x7f3f44074e30 2024-09-05T12:26:26.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3f280a2c10 msgr2=0x7f3f280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:26.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3f280a2c10 0x7f3f280a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009bd0 tx=0x7f3f34005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:26.971 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 msgr2=0x7f3f4419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 0x7f3f4419a990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000e9f0 tx=0x7f3f3000edf0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 shutdown_connections 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f3f280a2c10 0x7f3f280a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f4410e280 0x7f3f4419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f44074e30 0x7f3f4419a990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 --2- 172.21.15.71:0/2490488902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f44074510 0x7f3f4419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 >> 172.21.15.71:0/2490488902 conn(0x7f3f440fd6e0 msgr2=0x7f3f440fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 shutdown_connections 2024-09-05T12:26:26.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:26.819+0000 7f3f43200640 1 -- 172.21.15.71:0/2490488902 wait complete. 2024-09-05T12:26:27.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:27 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2490488902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T12:26:27.562 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:27 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2490488902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T12:26:27.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:27 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2490488902' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-05T12:26:27.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 -- 172.21.15.71:0/3245308114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f0103e30 msgr2=0x7f91f0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 --2- 172.21.15.71:0/3245308114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f0103e30 0x7f91f0104210 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009900 tx=0x7f91e002a410 comp rx=0 tx=0).stop 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 -- 172.21.15.71:0/3245308114 shutdown_connections 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 --2- 172.21.15.71:0/3245308114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0108f60 0x7f91f0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 --2- 172.21.15.71:0/3245308114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91f01047e0 0x7f91f0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 --2- 172.21.15.71:0/3245308114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f0103e30 0x7f91f0104210 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 -- 172.21.15.71:0/3245308114 >> 172.21.15.71:0/3245308114 conn(0x7f91f00ff8d0 msgr2=0x7f91f0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 -- 172.21.15.71:0/3245308114 shutdown_connections 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.671+0000 7f91ef200640 1 -- 172.21.15.71:0/3245308114 wait complete. 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.672+0000 7f91ef200640 1 Processor -- start 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.672+0000 7f91ef200640 1 -- start start 2024-09-05T12:26:27.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.672+0000 7f91ef200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0103e30 0x7f91f019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.672+0000 7f91ef200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 0x7f91f019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ef200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91f0108f60 0x7f91f019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ef200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0114010 con 0x7f91f01047e0 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ef200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0113e90 con 0x7f91f0103e30 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ef200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91f0114190 con 0x7f91f0108f60 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 0x7f91f019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0103e30 0x7f91f019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 0x7f91f019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53258/0 (socket says 172.21.15.71:53258) 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ede00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0103e30 0x7f91f019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:38246/0 (socket says 172.21.15.71:38246) 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 -- 172.21.15.71:0/169016272 learned_addr learned my addr 172.21.15.71:0/169016272 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ee800640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91f0108f60 0x7f91f019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 -- 172.21.15.71:0/169016272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91f0108f60 msgr2=0x7f91f019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:27.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91f0108f60 0x7f91f019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 -- 172.21.15.71:0/169016272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0103e30 msgr2=0x7f91f019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0103e30 0x7f91f019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.673+0000 7f91ed400640 1 -- 172.21.15.71:0/169016272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f019f480 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.674+0000 7f91ed400640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 0x7f91f019aa10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00d950 tx=0x7f91dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.674+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc017070 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.674+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f019f770 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.674+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91dc009c30 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.674+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc00f6b0 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.674+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01a7050 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.675+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f0104210 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.677+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f91dc0109d0 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.679+0000 7f91e6a00640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f91d40a2c10 0x7f91d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.679+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f91dc0cf150 con 0x7f91f01047e0 2024-09-05T12:26:27.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.679+0000 7f91ede00640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f91d40a2c10 0x7f91d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.680+0000 7f91ede00640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f91d40a2c10 0x7f91d40a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f91e0000c00 tx=0x7f91e0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:27.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.683+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91dc009de0 con 0x7f91f01047e0 2024-09-05T12:26:27.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.862+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f91f010bbb0 con 0x7f91f01047e0 2024-09-05T12:26:27.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.862+0000 7f91e6a00640 1 -- 172.21.15.71:0/169016272 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f91dc09b540 con 0x7f91f01047e0 2024-09-05T12:26:27.863 INFO:teuthology.orchestra.run.smithi071.stdout:137438953542 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f91d40a2c10 msgr2=0x7f91d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f91d40a2c10 0x7f91d40a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f91e0000c00 tx=0x7f91e0005b50 comp rx=0 tx=0).stop 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 msgr2=0x7f91f019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 0x7f91f019aa10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f91dc00d950 tx=0x7f91dc00de10 comp rx=0 tx=0).stop 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 shutdown_connections 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f91d40a2c10 0x7f91d40a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91f0108f60 0x7f91f019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91f01047e0 0x7f91f019aa10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 --2- 172.21.15.71:0/169016272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91f0103e30 0x7f91f019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:27.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 >> 172.21.15.71:0/169016272 conn(0x7f91f00ff8d0 msgr2=0x7f91f0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:27.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 shutdown_connections 2024-09-05T12:26:27.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:27.865+0000 7f91ef200640 1 -- 172.21.15.71:0/169016272 wait complete. 2024-09-05T12:26:27.891 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:28.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:28 smithi071 ceph-mon[29845]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:28.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:28 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/169016272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T12:26:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:28 smithi174 ceph-mon[34313]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:28 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/169016272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T12:26:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 -- 172.21.15.71:0/1103208953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc103ff0 msgr2=0x7fb9fc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:28.614 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 --2- 172.21.15.71:0/1103208953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc1043d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec009900 tx=0x7fb9ec02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 -- 172.21.15.71:0/1103208953 shutdown_connections 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 --2- 172.21.15.71:0/1103208953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9fc109120 0x7fb9fc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 --2- 172.21.15.71:0/1103208953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc1049a0 0x7fb9fc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 --2- 172.21.15.71:0/1103208953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc1043d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 -- 172.21.15.71:0/1103208953 >> 172.21.15.71:0/1103208953 conn(0x7fb9fc0ffa90 msgr2=0x7fb9fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 -- 172.21.15.71:0/1103208953 shutdown_connections 2024-09-05T12:26:28.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.613+0000 7fba01a00640 1 -- 172.21.15.71:0/1103208953 wait complete. 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.614+0000 7fba01a00640 1 Processor -- start 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.614+0000 7fba01a00640 1 -- start start 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.614+0000 7fba01a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fba01a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc1049a0 0x7fb9fc19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fba01a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9fc109120 0x7fb9fc19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fba01a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc114170 con 0x7fb9fc103ff0 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fba01a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc113ff0 con 0x7fb9fc1049a0 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fba01a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9fc1142f0 con 0x7fb9fc109120 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53280/0 (socket says 172.21.15.71:53280) 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 -- 172.21.15.71:0/3488383061 learned_addr learned my addr 172.21.15.71:0/3488383061 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:28.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9f2000640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc1049a0 0x7fb9fc19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9fb400640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9fc109120 0x7fb9fc19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 -- 172.21.15.71:0/3488383061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9fc109120 msgr2=0x7fb9fc19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9fc109120 0x7fb9fc19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 -- 172.21.15.71:0/3488383061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc1049a0 msgr2=0x7fb9fc19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc1049a0 0x7fb9fc19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.615+0000 7fb9faa00640 1 -- 172.21.15.71:0/3488383061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9fc19f600 con 0x7fb9fc103ff0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.616+0000 7fb9faa00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc19a650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec004770 tx=0x7fb9ec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.616+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec03f070 con 0x7fb9fc103ff0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.616+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9ec031ca0 con 0x7fb9fc103ff0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.616+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9fc19f890 con 0x7fb9fc103ff0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.616+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec035030 con 0x7fb9fc103ff0 2024-09-05T12:26:28.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.616+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9fc1a71d0 con 0x7fb9fc103ff0 2024-09-05T12:26:28.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.617+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9fc1054d0 con 0x7fb9fc103ff0 2024-09-05T12:26:28.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.619+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb9ec042070 con 0x7fb9fc103ff0 2024-09-05T12:26:28.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.620+0000 7fb9f3e00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9e00a2c10 0x7fb9e00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:28.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.620+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fb9ec0efc30 con 0x7fb9fc103ff0 2024-09-05T12:26:28.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.621+0000 7fb9f2000640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9e00a2c10 0x7fb9e00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:28.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.621+0000 7fb9f2000640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9e00a2c10 0x7fb9e00a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40095f0 tx=0x7fb9e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:28.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.625+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9ec0bc020 con 0x7fb9fc103ff0 2024-09-05T12:26:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:28 smithi123 ceph-mon[34321]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:28 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/169016272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-05T12:26:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.808+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb9fc10c300 con 0x7fb9fc103ff0 2024-09-05T12:26:28.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.809+0000 7fb9f3e00640 1 -- 172.21.15.71:0/3488383061 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb9ec0c0320 con 0x7fb9fc103ff0 2024-09-05T12:26:28.810 INFO:teuthology.orchestra.run.smithi071.stdout:249108103217 2024-09-05T12:26:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9e00a2c10 msgr2=0x7fb9e00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9e00a2c10 0x7fb9e00a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40095f0 tx=0x7fb9e4005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 msgr2=0x7fb9fc19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:28.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc19a650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb9ec004770 tx=0x7fb9ec003c10 comp rx=0 tx=0).stop 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 shutdown_connections 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb9e00a2c10 0x7fb9e00a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9fc109120 0x7fb9fc19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9fc1049a0 0x7fb9fc19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 --2- 172.21.15.71:0/3488383061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9fc103ff0 0x7fb9fc19a650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 >> 172.21.15.71:0/3488383061 conn(0x7fb9fc0ffa90 msgr2=0x7fb9fc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:28.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 shutdown_connections 2024-09-05T12:26:28.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:28.811+0000 7fba01a00640 1 -- 172.21.15.71:0/3488383061 wait complete. 2024-09-05T12:26:29.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 -- 172.21.15.71:0/2504059592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 msgr2=0x7f46001021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:29.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 --2- 172.21.15.71:0/2504059592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f46001021d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009900 tx=0x7f45f002a410 comp rx=0 tx=0).stop 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 -- 172.21.15.71:0/2504059592 shutdown_connections 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 --2- 172.21.15.71:0/2504059592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 0x7f460010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 --2- 172.21.15.71:0/2504059592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 0x7f46001067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 --2- 172.21.15.71:0/2504059592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f46001021d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 -- 172.21.15.71:0/2504059592 >> 172.21.15.71:0/2504059592 conn(0x7f46000fd8b0 msgr2=0x7f46000ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 -- 172.21.15.71:0/2504059592 shutdown_connections 2024-09-05T12:26:29.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.469+0000 7f4604c00640 1 -- 172.21.15.71:0/2504059592 wait complete. 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.470+0000 7f4604c00640 1 Processor -- start 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 -- start start 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f460019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 0x7f460019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 0x7f460019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4600111fd0 con 0x7f4600101df0 2024-09-05T12:26:29.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4600111e50 con 0x7f46001027a0 2024-09-05T12:26:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f4604c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4600112150 con 0x7f4600106f20 2024-09-05T12:26:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f45fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f460019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f45fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f460019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53292/0 (socket says 172.21.15.71:53292) 2024-09-05T12:26:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f45fde00640 1 -- 172.21.15.71:0/2641939484 learned_addr learned my addr 172.21.15.71:0/2641939484 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f45fd400640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 0x7f460019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:29.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.471+0000 7f45fe800640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 0x7f460019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fde00640 1 -- 172.21.15.71:0/2641939484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 msgr2=0x7f460019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fde00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 0x7f460019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fde00640 1 -- 172.21.15.71:0/2641939484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 msgr2=0x7f460019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fde00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 0x7f460019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fde00640 1 -- 172.21.15.71:0/2641939484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460019f5d0 con 0x7f4600101df0 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fe800640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 0x7f460019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fd400640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 0x7f460019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45fde00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f460019a620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45f0004770 tx=0x7f45f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.472+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f003f070 con 0x7f4600101df0 2024-09-05T12:26:29.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.473+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f0031ca0 con 0x7f4600101df0 2024-09-05T12:26:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.473+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0035030 con 0x7f4600101df0 2024-09-05T12:26:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.473+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460019f860 con 0x7f4600101df0 2024-09-05T12:26:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.473+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001a71a0 con 0x7f4600101df0 2024-09-05T12:26:29.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.474+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46001021d0 con 0x7f4600101df0 2024-09-05T12:26:29.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.475+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f45f0042070 con 0x7f4600101df0 2024-09-05T12:26:29.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.477+0000 7f45f6a00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e40a2c90 0x7f45e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:29.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.477+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f45f00efa40 con 0x7f4600101df0 2024-09-05T12:26:29.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.477+0000 7f45fd400640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e40a2c90 0x7f45e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:29.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.478+0000 7f45fd400640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e40a2c90 0x7f45e40a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46000ff6b0 tx=0x7f45ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:29.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.481+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f45f00bbf50 con 0x7f4600101df0 2024-09-05T12:26:29.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:29 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3488383061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T12:26:29.602 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:29 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3488383061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T12:26:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.663+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f460019b980 con 0x7f4600101df0 2024-09-05T12:26:29.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.663+0000 7f45f6a00640 1 -- 172.21.15.71:0/2641939484 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f45f00c0250 con 0x7f4600101df0 2024-09-05T12:26:29.664 INFO:teuthology.orchestra.run.smithi071.stdout:214748364854 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e40a2c90 msgr2=0x7f45e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e40a2c90 0x7f45e40a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f46000ff6b0 tx=0x7f45ec005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 msgr2=0x7f460019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f460019a620 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45f0004770 tx=0x7f45f0003c10 comp rx=0 tx=0).stop 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 shutdown_connections 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f45e40a2c90 0x7f45e40a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4600106f20 0x7f460019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46001027a0 0x7f460019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 --2- 172.21.15.71:0/2641939484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4600101df0 0x7f460019a620 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 >> 172.21.15.71:0/2641939484 conn(0x7f46000fd8b0 msgr2=0x7f46000feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 shutdown_connections 2024-09-05T12:26:29.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:29.666+0000 7f4604c00640 1 -- 172.21.15.71:0/2641939484 wait complete. 2024-09-05T12:26:29.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:29 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3488383061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-05T12:26:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:30 smithi174 ceph-mon[34313]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:30 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2641939484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T12:26:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:30 smithi071 ceph-mon[29845]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:30 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2641939484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T12:26:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:30 smithi123 ceph-mon[34321]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:30 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2641939484' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-05T12:26:30.871 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:30.874 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:30.875 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:30.880 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:30.915 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411408 for osd.2 2024-09-05T12:26:30.916 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 -- 172.21.15.71:0/4030705268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 msgr2=0x7fc2b010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/4030705268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 0x7fc2b010ea90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009ca0 tx=0x7fc2a802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 -- 172.21.15.71:0/4030705268 shutdown_connections 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/4030705268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 0x7fc2b010ea90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/4030705268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2b0102790 0x7fc2b01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/4030705268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.152+0000 7fc2b7a00640 1 -- 172.21.15.71:0/4030705268 >> 172.21.15.71:0/4030705268 conn(0x7fc2b00fd8c0 msgr2=0x7fc2b00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:31.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.153+0000 7fc2b7a00640 1 -- 172.21.15.71:0/4030705268 shutdown_connections 2024-09-05T12:26:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.153+0000 7fc2b7a00640 1 -- 172.21.15.71:0/4030705268 wait complete. 2024-09-05T12:26:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 Processor -- start 2024-09-05T12:26:31.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 -- start start 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2b0102790 0x7fc2b019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 0x7fc2b019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0111f70 con 0x7fc2b0106f10 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0111df0 con 0x7fc2b0102790 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.154+0000 7fc2b7a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01120f0 con 0x7fc2b0101de0 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47244/0 (socket says 172.21.15.71:47244) 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 -- 172.21.15.71:0/882874554 learned_addr learned my addr 172.21.15.71:0/882874554 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2afe00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2b0102790 0x7fc2b019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 0x7fc2b019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 -- 172.21.15.71:0/882874554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2b0102790 msgr2=0x7fc2b019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2b0102790 0x7fc2b019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 -- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 msgr2=0x7fc2b019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 0x7fc2b019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.157 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.155+0000 7fc2b5000640 1 -- 172.21.15.71:0/882874554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b019f5a0 con 0x7fc2b0101de0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.156+0000 7fc2b5000640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b019a5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400e9f0 tx=0x7fc2a400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.156+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400cd80 con 0x7fc2b0101de0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.156+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b019f890 con 0x7fc2b0101de0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.156+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a4005500 con 0x7fc2b0101de0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.156+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4010690 con 0x7fc2b0101de0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.156+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b01a7170 con 0x7fc2b0101de0 2024-09-05T12:26:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.158+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc290005290 con 0x7fc2b0101de0 2024-09-05T12:26:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.160+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc2a40046c0 con 0x7fc2b0101de0 2024-09-05T12:26:31.162 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.162+0000 7fc2ad600640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc2980a2c90 0x7fc2980a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.162+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fc2a40cf870 con 0x7fc2b0101de0 2024-09-05T12:26:31.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.162+0000 7fc2afe00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc2980a2c90 0x7fc2980a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.163+0000 7fc2afe00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc2980a2c90 0x7fc2980a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2b010eeb0 tx=0x7fc29c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.165+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc2a409c310 con 0x7fc2b0101de0 2024-09-05T12:26:31.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.345+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc290005850 con 0x7fc2b0101de0 2024-09-05T12:26:31.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.346+0000 7fc2ad600640 1 -- 172.21.15.71:0/882874554 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc2a409bc10 con 0x7fc2b0101de0 2024-09-05T12:26:31.347 INFO:teuthology.orchestra.run.smithi071.stdout:34359738461 2024-09-05T12:26:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc2980a2c90 msgr2=0x7fc2980a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc2980a2c90 0x7fc2980a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2b010eeb0 tx=0x7fc29c00b390 comp rx=0 tx=0).stop 2024-09-05T12:26:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 msgr2=0x7fc2b019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b019a5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400e9f0 tx=0x7fc2a400edf0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 shutdown_connections 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fc2980a2c90 0x7fc2980a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc2b0106f10 0x7fc2b019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2b0102790 0x7fc2b019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 --2- 172.21.15.71:0/882874554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2b0101de0 0x7fc2b019a5f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.348+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 >> 172.21.15.71:0/882874554 conn(0x7fc2b00fd8c0 msgr2=0x7fc2b00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.349+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 shutdown_connections 2024-09-05T12:26:31.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.349+0000 7fc2b7a00640 1 -- 172.21.15.71:0/882874554 wait complete. 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 -- 172.21.15.71:0/105603637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630106f10 msgr2=0x7f863010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 --2- 172.21.15.71:0/105603637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630106f10 0x7f863010ea90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8628009900 tx=0x7f862802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 -- 172.21.15.71:0/105603637 shutdown_connections 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 --2- 172.21.15.71:0/105603637 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630106f10 0x7f863010ea90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 --2- 172.21.15.71:0/105603637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630102790 0x7f86301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 --2- 172.21.15.71:0/105603637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8630101de0 0x7f86301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 -- 172.21.15.71:0/105603637 >> 172.21.15.71:0/105603637 conn(0x7f86300fd8c0 msgr2=0x7f86300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 -- 172.21.15.71:0/105603637 shutdown_connections 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.907+0000 7f8635200640 1 -- 172.21.15.71:0/105603637 wait complete. 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.908+0000 7f8635200640 1 Processor -- start 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.908+0000 7f8635200640 1 -- start start 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8635200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 0x7f86300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8635200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630102790 0x7f8630074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8635200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8630106f10 0x7f8630074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8635200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8630111fc0 con 0x7f8630102790 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8635200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8630111e40 con 0x7f8630106f10 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8635200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8630112140 con 0x7f8630101de0 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f862de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 0x7f86300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f8625400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630102790 0x7f8630074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f862de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 0x7f86300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47248/0 (socket says 172.21.15.71:47248) 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f862de00640 1 -- 172.21.15.71:0/772888889 learned_addr learned my addr 172.21.15.71:0/772888889 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f862e800640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8630106f10 0x7f8630074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.909+0000 7f862de00640 1 -- 172.21.15.71:0/772888889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8630106f10 msgr2=0x7f8630074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f862de00640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8630106f10 0x7f8630074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f862de00640 1 -- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630102790 msgr2=0x7f8630074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f862de00640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630102790 0x7f8630074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f862de00640 1 -- 172.21.15.71:0/772888889 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86301a68a0 con 0x7f8630101de0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f862de00640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 0x7f86300744f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f862000ebc0 tx=0x7f862000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620010630 con 0x7f8630101de0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86301a6b90 con 0x7f8630101de0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8620010cb0 con 0x7f8630101de0 2024-09-05T12:26:31.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8620018bd0 con 0x7f8630101de0 2024-09-05T12:26:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.910+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86301a70d0 con 0x7f8630101de0 2024-09-05T12:26:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.911+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86301021c0 con 0x7f8630101de0 2024-09-05T12:26:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.913+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f86200107d0 con 0x7f8630101de0 2024-09-05T12:26:31.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.915+0000 7f8627400640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86140a2c10 0x7f86140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.915+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f86200cfe60 con 0x7f8630101de0 2024-09-05T12:26:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.915+0000 7f8625400640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86140a2c10 0x7f86140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.916+0000 7f8625400640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86140a2c10 0x7f86140a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8630075f80 tx=0x7f8618005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:31.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:31.917+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f862009c250 con 0x7f8630101de0 2024-09-05T12:26:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.097+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f8630075a20 con 0x7f8630101de0 2024-09-05T12:26:32.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.098+0000 7f8627400640 1 -- 172.21.15.71:0/772888889 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f8630075a20 con 0x7f8630101de0 2024-09-05T12:26:32.098 INFO:teuthology.orchestra.run.smithi071.stdout:304942678055 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86140a2c10 msgr2=0x7f86140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86140a2c10 0x7f86140a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8630075f80 tx=0x7f8618005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 msgr2=0x7f86300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 0x7f86300744f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f862000ebc0 tx=0x7f862000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 shutdown_connections 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f86140a2c10 0x7f86140a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8630106f10 0x7f8630074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8630102790 0x7f8630074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 --2- 172.21.15.71:0/772888889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8630101de0 0x7f86300744f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 >> 172.21.15.71:0/772888889 conn(0x7f86300fd8c0 msgr2=0x7f86300feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 shutdown_connections 2024-09-05T12:26:32.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.101+0000 7f8635200640 1 -- 172.21.15.71:0/772888889 wait complete. 2024-09-05T12:26:32.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:32 smithi071 ceph-mon[29845]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:32.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:32 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/882874554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T12:26:32.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:32 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/772888889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T12:26:32.456 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364852 got 214748364854 for osd.7 2024-09-05T12:26:32.456 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:32 smithi174 ceph-mon[34313]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:32 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/882874554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T12:26:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:32 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/772888889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.662+0000 7f8430600640 1 -- 172.21.15.71:0/325243727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 msgr2=0x7f8428110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.662+0000 7f8430600640 1 --2- 172.21.15.71:0/325243727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f8428110ca0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8420009ca0 tx=0x7f842002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 -- 172.21.15.71:0/325243727 shutdown_connections 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 --2- 172.21.15.71:0/325243727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f8428110ca0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 --2- 172.21.15.71:0/325243727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84281049a0 0x7f84281089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 --2- 172.21.15.71:0/325243727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8428103ff0 0x7f84281043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 -- 172.21.15.71:0/325243727 >> 172.21.15.71:0/325243727 conn(0x7f84280ffa90 msgr2=0x7f8428101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 -- 172.21.15.71:0/325243727 shutdown_connections 2024-09-05T12:26:32.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.663+0000 7f8430600640 1 -- 172.21.15.71:0/325243727 wait complete. 2024-09-05T12:26:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 Processor -- start 2024-09-05T12:26:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 -- start start 2024-09-05T12:26:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:32.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84281141a0 con 0x7f8428109120 2024-09-05T12:26:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8428114020 con 0x7f84281049a0 2024-09-05T12:26:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.664+0000 7f8430600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8428114320 con 0x7f8428103ff0 2024-09-05T12:26:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842dc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:32.666 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53332/0 (socket says 172.21.15.71:53332) 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842e600640 1 -- 172.21.15.71:0/3480078392 learned_addr learned my addr 172.21.15.71:0/3480078392 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842d200640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842dc00640 1 -- 172.21.15.71:0/3480078392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84281049a0 msgr2=0x7f842819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842dc00640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842dc00640 1 -- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 msgr2=0x7f842819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842dc00640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.665+0000 7f842dc00640 1 -- 172.21.15.71:0/3480078392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842819f5b0 con 0x7f8428103ff0 2024-09-05T12:26:32.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.666+0000 7f842dc00640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8428103ff0 0x7f842819a600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e9f0 tx=0x7f841c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.666+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c00ccc0 con 0x7f8428103ff0 2024-09-05T12:26:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.666+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f841c010910 con 0x7f8428103ff0 2024-09-05T12:26:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.666+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842819f8a0 con 0x7f8428103ff0 2024-09-05T12:26:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.666+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841c018960 con 0x7f8428103ff0 2024-09-05T12:26:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.666+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a7180 con 0x7f8428103ff0 2024-09-05T12:26:32.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.667+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84281043d0 con 0x7f8428103ff0 2024-09-05T12:26:32.672 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.671+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f841c010ac0 con 0x7f8428103ff0 2024-09-05T12:26:32.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.672+0000 7f8426000640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84100a2c10 0x7f84100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:32.673 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.673+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f841c0cf920 con 0x7f8428103ff0 2024-09-05T12:26:32.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.673+0000 7f842d200640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84100a2c10 0x7f84100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:32.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.674+0000 7f842d200640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84100a2c10 0x7f84100a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f842810ab20 tx=0x7f8414005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:32.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.676+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f841c09bd10 con 0x7f8428103ff0 2024-09-05T12:26:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:32 smithi123 ceph-mon[34321]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:32 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/882874554' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-05T12:26:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:32 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/772888889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-05T12:26:32.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.860+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f84281a7470 con 0x7f8428103ff0 2024-09-05T12:26:32.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.860+0000 7f8426000640 1 -- 172.21.15.71:0/3480078392 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f841c0a0010 con 0x7f8428103ff0 2024-09-05T12:26:32.861 INFO:teuthology.orchestra.run.smithi071.stdout:107374182476 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84100a2c10 msgr2=0x7f84100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84100a2c10 0x7f84100a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f842810ab20 tx=0x7f8414005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8428103ff0 msgr2=0x7f842819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8428103ff0 0x7f842819a600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f841c00e9f0 tx=0x7f841c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 shutdown_connections 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f84100a2c10 0x7f84100a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8428109120 0x7f842819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84281049a0 0x7f842819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 --2- 172.21.15.71:0/3480078392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8428103ff0 0x7f842819a600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:32.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 >> 172.21.15.71:0/3480078392 conn(0x7f84280ffa90 msgr2=0x7f8428101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:32.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 shutdown_connections 2024-09-05T12:26:32.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:32.863+0000 7f8430600640 1 -- 172.21.15.71:0/3480078392 wait complete. 2024-09-05T12:26:33.530 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:33 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3480078392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T12:26:33.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 -- 172.21.15.71:0/828645256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741079d0 msgr2=0x7f2974107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:33.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 --2- 172.21.15.71:0/828645256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741079d0 0x7f2974107db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2964009930 tx=0x7f296402a420 comp rx=0 tx=0).stop 2024-09-05T12:26:33.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 -- 172.21.15.71:0/828645256 shutdown_connections 2024-09-05T12:26:33.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 --2- 172.21.15.71:0/828645256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 0x7f297410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 --2- 172.21.15.71:0/828645256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741019d0 0x7f2974101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 --2- 172.21.15.71:0/828645256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741079d0 0x7f2974107db0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 -- 172.21.15.71:0/828645256 >> 172.21.15.71:0/828645256 conn(0x7f29740fd710 msgr2=0x7f29740ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 -- 172.21.15.71:0/828645256 shutdown_connections 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.535+0000 7f2973e00640 1 -- 172.21.15.71:0/828645256 wait complete. 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.536+0000 7f2973e00640 1 Processor -- start 2024-09-05T12:26:33.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.536+0000 7f2973e00640 1 -- start start 2024-09-05T12:26:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741019d0 0x7f2974198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 0x7f2974198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741079d0 0x7f297419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:33.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2974111dd0 con 0x7f29741019d0 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2974111c50 con 0x7f29741079d0 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2974111f50 con 0x7f2974102370 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2972a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741019d0 0x7f2974198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2972000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 0x7f2974198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2972a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741019d0 0x7f2974198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34062/0 (socket says 172.21.15.71:34062) 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2972a00640 1 -- 172.21.15.71:0/191582537 learned_addr learned my addr 172.21.15.71:0/191582537 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.537+0000 7f2973400640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741079d0 0x7f297419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973400640 1 -- 172.21.15.71:0/191582537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 msgr2=0x7f2974198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973400640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 0x7f2974198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973400640 1 -- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741019d0 msgr2=0x7f2974198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973400640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741019d0 0x7f2974198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973400640 1 -- 172.21.15.71:0/191582537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297419d200 con 0x7f29741079d0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2972000640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 0x7f2974198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973400640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741079d0 0x7f297419cb20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f296c00e940 tx=0x7f296c00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c004d20 con 0x7f29741079d0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297419d4f0 con 0x7f29741079d0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c005020 con 0x7f29741079d0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.538+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c0116b0 con 0x7f29741079d0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.539+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a4dd0 con 0x7f29741079d0 2024-09-05T12:26:33.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.540+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974101e30 con 0x7f29741079d0 2024-09-05T12:26:33.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.542+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f296c0054d0 con 0x7f29741079d0 2024-09-05T12:26:33.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.543+0000 7f296b400640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29580a2c90 0x7f29580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:33.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.543+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f296c0cf660 con 0x7f29741079d0 2024-09-05T12:26:33.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.543+0000 7f2972a00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29580a2c90 0x7f29580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:33.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.544+0000 7f2972a00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29580a2c90 0x7f29580a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2964000c00 tx=0x7f2964009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:33.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.545+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f296c016030 con 0x7f29741079d0 2024-09-05T12:26:33.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:33 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3480078392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T12:26:33.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.737+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f29741995b0 con 0x7f29741079d0 2024-09-05T12:26:33.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.738+0000 7f296b400640 1 -- 172.21.15.71:0/191582537 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f29741995b0 con 0x7f29741079d0 2024-09-05T12:26:33.739 INFO:teuthology.orchestra.run.smithi071.stdout:335007449123 2024-09-05T12:26:33.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29580a2c90 msgr2=0x7f29580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29580a2c90 0x7f29580a5150 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2964000c00 tx=0x7f2964009560 comp rx=0 tx=0).stop 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741079d0 msgr2=0x7f297419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741079d0 0x7f297419cb20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f296c00e940 tx=0x7f296c00ee00 comp rx=0 tx=0).stop 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 shutdown_connections 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f29580a2c90 0x7f29580a5150 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29741079d0 0x7f297419cb20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2974102370 0x7f2974198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 --2- 172.21.15.71:0/191582537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29741019d0 0x7f2974198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 >> 172.21.15.71:0/191582537 conn(0x7f29740fd710 msgr2=0x7f29740fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 shutdown_connections 2024-09-05T12:26:33.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.741+0000 7f2973e00640 1 -- 172.21.15.71:0/191582537 wait complete. 2024-09-05T12:26:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:33 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3480078392' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-05T12:26:34.532 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:34 smithi071 ceph-mon[29845]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:34.532 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:34 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1619260156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T12:26:34.532 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:34 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/191582537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T12:26:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:34 smithi123 ceph-mon[34321]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:34 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1619260156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T12:26:34.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:34 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/191582537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T12:26:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:34 smithi174 ceph-mon[34313]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:34 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1619260156' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-05T12:26:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:34 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/191582537' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-05T12:26:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1504000602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff0074ee0 msgr2=0x7f9ff010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1504000602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff0074ee0 0x7f9ff010dd20 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009900 tx=0x7f9fd802a480 comp rx=0 tx=0).stop 2024-09-05T12:26:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1504000602 shutdown_connections 2024-09-05T12:26:35.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1504000602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 0x7f9ff0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1504000602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff0074ee0 0x7f9ff010dd20 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1504000602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff00745c0 0x7f9ff00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1504000602 >> 172.21.15.71:0/1504000602 conn(0x7f9ff00fd740 msgr2=0x7f9ff00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1504000602 shutdown_connections 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.185+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1504000602 wait complete. 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 Processor -- start 2024-09-05T12:26:35.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 -- start start 2024-09-05T12:26:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 0x7f9ff019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff0074ee0 0x7f9ff019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 0x7f9ff019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0114000 con 0x7f9ff00745c0 2024-09-05T12:26:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0113e80 con 0x7f9ff010e260 2024-09-05T12:26:35.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.187+0000 7f9ff4a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0114180 con 0x7f9ff0074ee0 2024-09-05T12:26:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 0x7f9ff019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 0x7f9ff019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34084/0 (socket says 172.21.15.71:34084) 2024-09-05T12:26:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 -- 172.21.15.71:0/1932049629 learned_addr learned my addr 172.21.15.71:0/1932049629 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fe6a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff0074ee0 0x7f9ff019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:35.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fefe00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 0x7f9ff019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 -- 172.21.15.71:0/1932049629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff0074ee0 msgr2=0x7f9ff019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff0074ee0 0x7f9ff019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 -- 172.21.15.71:0/1932049629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 msgr2=0x7f9ff019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 0x7f9ff019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fef400640 1 -- 172.21.15.71:0/1932049629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff019f490 con 0x7f9ff00745c0 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.188+0000 7f9fefe00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 0x7f9ff019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:35.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.189+0000 7f9fef400640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 0x7f9ff019a4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000db80 tx=0x7f9fe000b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.189+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0017070 con 0x7f9ff00745c0 2024-09-05T12:26:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.189+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fe0003b70 con 0x7f9ff00745c0 2024-09-05T12:26:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.189+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00106c0 con 0x7f9ff00745c0 2024-09-05T12:26:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.189+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff019f780 con 0x7f9ff00745c0 2024-09-05T12:26:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.189+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a7060 con 0x7f9ff00745c0 2024-09-05T12:26:35.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.191+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff00749a0 con 0x7f9ff00745c0 2024-09-05T12:26:35.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.192+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9fe0004020 con 0x7f9ff00745c0 2024-09-05T12:26:35.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.194+0000 7f9fecc00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9fd40a2c90 0x7f9fd40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.194+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f9fe00cf270 con 0x7f9ff00745c0 2024-09-05T12:26:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.194+0000 7f9fe6a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9fd40a2c90 0x7f9fd40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:35.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.195+0000 7f9fe6a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9fd40a2c90 0x7f9fd40a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000c00 tx=0x7f9fd8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:35.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.198+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fe0020070 con 0x7f9ff00745c0 2024-09-05T12:26:35.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.383+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9ff0109410 con 0x7f9ff00745c0 2024-09-05T12:26:35.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.384+0000 7f9fecc00640 1 -- 172.21.15.71:0/1932049629 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9fe009bcb0 con 0x7f9ff00745c0 2024-09-05T12:26:35.386 INFO:teuthology.orchestra.run.smithi071.stdout:55834574937 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9fd40a2c90 msgr2=0x7f9fd40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9fd40a2c90 0x7f9fd40a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8000c00 tx=0x7f9fd8005b50 comp rx=0 tx=0).stop 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 msgr2=0x7f9ff019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 0x7f9ff019a4e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9fe000db80 tx=0x7f9fe000b630 comp rx=0 tx=0).stop 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 shutdown_connections 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f9fd40a2c90 0x7f9fd40a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff010e260 0x7f9ff019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ff0074ee0 0x7f9ff019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 --2- 172.21.15.71:0/1932049629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ff00745c0 0x7f9ff019a4e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:35.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 >> 172.21.15.71:0/1932049629 conn(0x7f9ff00fd740 msgr2=0x7f9ff00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 shutdown_connections 2024-09-05T12:26:35.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:35.386+0000 7f9ff4a00640 1 -- 172.21.15.71:0/1932049629 wait complete. 2024-09-05T12:26:35.783 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103215 got 249108103217 for osd.8 2024-09-05T12:26:35.783 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:36.136 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953539 got 137438953542 for osd.4 2024-09-05T12:26:36.136 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:36 smithi071 ceph-mon[29845]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-05T12:26:36.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:36 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1932049629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T12:26:36.459 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678052 got 304942678055 for osd.10 2024-09-05T12:26:36.460 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:36 smithi123 ceph-mon[34321]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-05T12:26:36.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:36 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1932049629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T12:26:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:36 smithi174 ceph-mon[34313]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 11 op/s 2024-09-05T12:26:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:36 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1932049629' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 -- 172.21.15.71:0/2455749479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac48074ee0 msgr2=0x7fac4810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 --2- 172.21.15.71:0/2455749479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac48074ee0 0x7fac4810dd20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac3002a480 comp rx=0 tx=0).stop 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 -- 172.21.15.71:0/2455749479 shutdown_connections 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 --2- 172.21.15.71:0/2455749479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac4810e260 0x7fac48110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 --2- 172.21.15.71:0/2455749479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac48074ee0 0x7fac4810dd20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 --2- 172.21.15.71:0/2455749479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac480745c0 0x7fac480749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 -- 172.21.15.71:0/2455749479 >> 172.21.15.71:0/2455749479 conn(0x7fac480fd740 msgr2=0x7fac480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 -- 172.21.15.71:0/2455749479 shutdown_connections 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.431+0000 7fac47200640 1 -- 172.21.15.71:0/2455749479 wait complete. 2024-09-05T12:26:36.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.432+0000 7fac47200640 1 Processor -- start 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.432+0000 7fac47200640 1 -- start start 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac47200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac480745c0 0x7fac4819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac47200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 0x7fac4819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac47200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 0x7fac4819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac47200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fac48113fb0 con 0x7fac4810e260 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac47200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac48113e30 con 0x7fac480745c0 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac47200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac48114130 con 0x7fac48074ee0 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac46800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 0x7fac4819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac3fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 0x7fac4819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac3fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 0x7fac4819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:56574/0 (socket says 172.21.15.71:56574) 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac3fe00640 1 -- 172.21.15.71:0/1619260156 learned_addr learned my addr 172.21.15.71:0/1619260156 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:36.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac46800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 0x7fac4819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34044/0 (socket says 172.21.15.71:34044) 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac45e00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac480745c0 0x7fac4819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.433+0000 7fac3fe00640 1 -- 172.21.15.71:0/1619260156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac480745c0 msgr2=0x7fac4819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3fe00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac480745c0 0x7fac4819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3fe00640 1 -- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 msgr2=0x7fac4819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3fe00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 0x7fac4819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3fe00640 1 -- 172.21.15.71:0/1619260156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4819f460 con 0x7fac48074ee0 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac46800640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 0x7fac4819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3fe00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 0x7fac4819a9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac30006010 tx=0x7fac30004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3003f030 con 0x7fac48074ee0 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac30004440 con 0x7fac48074ee0 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4819f6f0 con 0x7fac48074ee0 2024-09-05T12:26:36.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30035070 con 0x7fac48074ee0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.434+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481a7030 con 0x7fac48074ee0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.435+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac48075c70 con 0x7fac48074ee0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.437+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac30042030 con 0x7fac48074ee0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.438+0000 7fac3ea00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.439+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fac300efb00 con 0x7fac48074ee0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.439+0000 7fac45e00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.439+0000 7fac45e00640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac380095f0 tx=0x7fac38005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:36.917 INFO:teuthology.orchestra.run.smithi071.stdout:158913790019 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.441+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac300bbef0 con 0x7fac48074ee0 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.629+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fac481093e0 con 0x7fac48074ee0 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.630+0000 7fac3ea00640 1 -- 172.21.15.71:0/1619260156 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fac300c01f0 con 0x7fac48074ee0 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fac2c0a2c10 msgr2=0x7fac2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fac380095f0 tx=0x7fac38005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 msgr2=0x7fac4819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:36.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 0x7fac4819a9f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fac30006010 tx=0x7fac30004040 comp rx=0 tx=0).stop 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 shutdown_connections 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fac2c0a2c10 0x7fac2c0a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac4810e260 0x7fac4819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac48074ee0 0x7fac4819a9f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 --2- 172.21.15.71:0/1619260156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac480745c0 0x7fac4819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 >> 172.21.15.71:0/1619260156 conn(0x7fac480fd740 msgr2=0x7fac480fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 shutdown_connections 2024-09-05T12:26:36.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:33.632+0000 7fac47200640 1 -- 172.21.15.71:0/1619260156 wait complete. 2024-09-05T12:26:37.666 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182472 got 107374182476 for osd.3 2024-09-05T12:26:37.666 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:38.526 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:38 smithi071 ceph-mon[29845]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:38.528 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738461 for osd.0 2024-09-05T12:26:38.529 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:38 smithi123 ceph-mon[34321]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:38 smithi174 ceph-mon[34313]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:39.211 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:39.240 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574934 got 55834574937 for osd.1 2024-09-05T12:26:39.240 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:39.688 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790016 got 158913790019 for osd.5 2024-09-05T12:26:39.688 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:39.796 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:39.825 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449121 got 335007449123 for osd.11 2024-09-05T12:26:39.825 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 -- 172.21.15.71:0/3435363453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 msgr2=0x7f7a1c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 --2- 172.21.15.71:0/3435363453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c10ea90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009ca0 tx=0x7f7a1402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 -- 172.21.15.71:0/3435363453 shutdown_connections 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 --2- 172.21.15.71:0/3435363453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c10ea90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 --2- 172.21.15.71:0/3435363453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c102790 0x7f7a1c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 --2- 172.21.15.71:0/3435363453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c101de0 0x7f7a1c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 -- 172.21.15.71:0/3435363453 >> 172.21.15.71:0/3435363453 conn(0x7f7a1c0fd8c0 msgr2=0x7f7a1c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 -- 172.21.15.71:0/3435363453 shutdown_connections 2024-09-05T12:26:40.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.448+0000 7f7a20c00640 1 -- 172.21.15.71:0/3435363453 wait complete. 2024-09-05T12:26:40.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 Processor -- start 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 -- start start 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c101de0 0x7f7a1c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c102790 0x7f7a1c19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c19eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c111f70 con 0x7f7a1c106f10 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c111df0 con 0x7f7a1c102790 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.449+0000 7f7a20c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c1120f0 con 0x7f7a1c101de0 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c19eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a19e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c101de0 0x7f7a1c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c19eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34098/0 (socket says 172.21.15.71:34098) 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 -- 172.21.15.71:0/392200042 learned_addr learned my addr 172.21.15.71:0/392200042 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a11400640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c102790 0x7f7a1c19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:40.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 -- 172.21.15.71:0/392200042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c101de0 msgr2=0x7f7a1c19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c101de0 0x7f7a1c19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 -- 172.21.15.71:0/392200042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c102790 msgr2=0x7f7a1c19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c102790 0x7f7a1c19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a1a800640 1 -- 172.21.15.71:0/392200042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c19f580 con 0x7f7a1c106f10 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a19e00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c101de0 0x7f7a1c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.450+0000 7f7a11400640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c102790 0x7f7a1c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.451+0000 7f7a1a800640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c19eea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a14031a20 tx=0x7f7a14031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.451+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14003b70 con 0x7f7a1c106f10 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.451+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a14003d10 con 0x7f7a1c106f10 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.451+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1402fc40 con 0x7f7a1c106f10 2024-09-05T12:26:40.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.451+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c19f810 con 0x7f7a1c106f10 2024-09-05T12:26:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.451+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a7150 con 0x7f7a1c106f10 2024-09-05T12:26:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.452+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c1032c0 con 0x7f7a1c106f10 2024-09-05T12:26:40.455 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.454+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7a14004050 con 0x7f7a1c106f10 2024-09-05T12:26:40.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.455+0000 7f7a13400640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a000a2c10 0x7f7a000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:40.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.455+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7a140f0570 con 0x7f7a1c106f10 2024-09-05T12:26:40.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.455+0000 7f7a19e00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a000a2c10 0x7f7a000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:40.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.456+0000 7f7a19e00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a000a2c10 0x7f7a000a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0095f0 tx=0x7f7a0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:40.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.460+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a140bd150 con 0x7f7a1c106f10 2024-09-05T12:26:40.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:40 smithi071 ceph-mon[29845]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:40.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.646+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f7a1c1a7350 con 0x7f7a1c106f10 2024-09-05T12:26:40.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.646+0000 7f7a13400640 1 -- 172.21.15.71:0/392200042 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a140bca50 con 0x7f7a1c106f10 2024-09-05T12:26:40.648 INFO:teuthology.orchestra.run.smithi071.stdout:283467841582 2024-09-05T12:26:40.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a000a2c10 msgr2=0x7f7a000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:40.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a000a2c10 0x7f7a000a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0095f0 tx=0x7f7a0c005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:40.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 msgr2=0x7f7a1c19eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:40.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c19eea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7a14031a20 tx=0x7f7a14031a50 comp rx=0 tx=0).stop 2024-09-05T12:26:40.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 shutdown_connections 2024-09-05T12:26:40.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7a000a2c10 0x7f7a000a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7a1c106f10 0x7f7a1c19eea0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a1c102790 0x7f7a1c19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.648+0000 7f7a20c00640 1 --2- 172.21.15.71:0/392200042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a1c101de0 0x7f7a1c19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:40.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.649+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 >> 172.21.15.71:0/392200042 conn(0x7f7a1c0fd8c0 msgr2=0x7f7a1c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:40.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.649+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 shutdown_connections 2024-09-05T12:26:40.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:40.649+0000 7f7a20c00640 1 -- 172.21.15.71:0/392200042 wait complete. 2024-09-05T12:26:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:40 smithi123 ceph-mon[34321]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:40 smithi174 ceph-mon[34313]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.230+0000 7f04e7000640 1 -- 172.21.15.71:0/2259002746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0109120 msgr2=0x7f04e0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.230+0000 7f04e7000640 1 --2- 172.21.15.71:0/2259002746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0109120 0x7f04e0110ca0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04d8009ca0 tx=0x7f04d802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 -- 172.21.15.71:0/2259002746 shutdown_connections 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 --2- 172.21.15.71:0/2259002746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0109120 0x7f04e0110ca0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 --2- 172.21.15.71:0/2259002746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 0x7f04e01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 --2- 172.21.15.71:0/2259002746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e0103ff0 0x7f04e01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 -- 172.21.15.71:0/2259002746 >> 172.21.15.71:0/2259002746 conn(0x7f04e00ffa90 msgr2=0x7f04e0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:41.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 -- 172.21.15.71:0/2259002746 shutdown_connections 2024-09-05T12:26:41.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.231+0000 7f04e7000640 1 -- 172.21.15.71:0/2259002746 wait complete. 2024-09-05T12:26:41.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 Processor -- start 2024-09-05T12:26:41.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 -- start start 2024-09-05T12:26:41.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 0x7f04e019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:41.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 0x7f04e019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:41.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e0109120 0x7f04e019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:41.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114180 con 0x7f04e0103ff0 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114000 con 0x7f04e01049a0 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.232+0000 7f04e7000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0114300 con 0x7f04e0109120 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 0x7f04e019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04dfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 0x7f04e019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34104/0 (socket says 172.21.15.71:34104) 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04dfe00640 1 -- 172.21.15.71:0/3095721274 learned_addr learned my addr 172.21.15.71:0/3095721274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 0x7f04e019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04e5000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e0109120 0x7f04e019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 -- 172.21.15.71:0/3095721274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e0109120 msgr2=0x7f04e019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e0109120 0x7f04e019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 -- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 msgr2=0x7f04e019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 0x7f04e019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 -- 172.21.15.71:0/3095721274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e019f590 con 0x7f04e01049a0 2024-09-05T12:26:41.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04dfe00640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 0x7f04e019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.233+0000 7f04df400640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 0x7f04e019ab20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e9f0 tx=0x7f04cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.234+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc00ccc0 con 0x7f04e01049a0 2024-09-05T12:26:41.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.234+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04cc010910 con 0x7f04e01049a0 2024-09-05T12:26:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.234+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc018960 con 0x7f04e01049a0 2024-09-05T12:26:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.234+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e019f880 con 0x7f04e01049a0 2024-09-05T12:26:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.234+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e01a7160 con 0x7f04e01049a0 2024-09-05T12:26:41.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.235+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e01043d0 con 0x7f04e01049a0 2024-09-05T12:26:41.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.238+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f04cc010ac0 con 0x7f04e01049a0 2024-09-05T12:26:41.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.239+0000 7f04dcc00640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f04c40a2c90 0x7f04c40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:41.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.240+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f04cc0cff90 con 0x7f04e01049a0 2024-09-05T12:26:41.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.240+0000 7f04dfe00640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f04c40a2c90 0x7f04c40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:41.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.240+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04cc026070 con 0x7f04e01049a0 2024-09-05T12:26:41.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.241+0000 7f04dfe00640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f04c40a2c90 0x7f04c40a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04d00095f0 tx=0x7f04d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:41.373 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:41 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/392200042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T12:26:41.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.423+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f04e01a73c0 con 0x7f04e01049a0 2024-09-05T12:26:41.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.424+0000 7f04dcc00640 1 -- 172.21.15.71:0/3095721274 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f04cc09c380 con 0x7f04e01049a0 2024-09-05T12:26:41.426 INFO:teuthology.orchestra.run.smithi071.stdout:193273528382 2024-09-05T12:26:41.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.426+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f04c40a2c90 msgr2=0x7f04c40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.426+0000 7f04e7000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f04c40a2c90 0x7f04c40a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f04d00095f0 tx=0x7f04d0005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 msgr2=0x7f04e019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 0x7f04e019ab20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00e9f0 tx=0x7f04cc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 shutdown_connections 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f04c40a2c90 0x7f04c40a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04e0109120 0x7f04e019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04e01049a0 0x7f04e019ab20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 --2- 172.21.15.71:0/3095721274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04e0103ff0 0x7f04e019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 >> 172.21.15.71:0/3095721274 conn(0x7f04e00ffa90 msgr2=0x7f04e0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 shutdown_connections 2024-09-05T12:26:41.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:41.427+0000 7f04e7000640 1 -- 172.21.15.71:0/3095721274 wait complete. 2024-09-05T12:26:41.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:41 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/392200042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T12:26:41.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:41 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/392200042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-05T12:26:41.887 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841577 got 283467841582 for osd.9 2024-09-05T12:26:41.887 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:42.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:42 smithi071 ceph-mon[29845]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:42.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:42 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3095721274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T12:26:42.652 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528378 got 193273528382 for osd.6 2024-09-05T12:26:42.652 DEBUG:teuthology.parallel:result is None 2024-09-05T12:26:42.652 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T12:26:42.653 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:26:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:42 smithi123 ceph-mon[34321]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:42 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3095721274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T12:26:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:42 smithi174 ceph-mon[34313]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:42 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3095721274' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-05T12:26:43.021 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 -- 172.21.15.71:0/1301069226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0102790 msgr2=0x7f08d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 --2- 172.21.15.71:0/1301069226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0102790 0x7f08d01067e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f08bc009900 tx=0x7f08bc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 -- 172.21.15.71:0/1301069226 shutdown_connections 2024-09-05T12:26:43.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 --2- 172.21.15.71:0/1301069226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d0106f10 0x7f08d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 --2- 172.21.15.71:0/1301069226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0102790 0x7f08d01067e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 --2- 172.21.15.71:0/1301069226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0101de0 0x7f08d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 -- 172.21.15.71:0/1301069226 >> 172.21.15.71:0/1301069226 conn(0x7f08d00fd8c0 msgr2=0x7f08d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 -- 172.21.15.71:0/1301069226 shutdown_connections 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.610+0000 7f08d7200640 1 -- 172.21.15.71:0/1301069226 wait complete. 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.611+0000 7f08d7200640 1 Processor -- start 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 -- start start 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0101de0 0x7f08d019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 0x7f08d019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d0106f10 0x7f08d019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0111f70 con 0x7f08d0101de0 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08d0111df0 con 0x7f08d0102790 2024-09-05T12:26:43.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d7200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08d01120f0 con 0x7f08d0106f10 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 0x7f08d019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08cf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 0x7f08d019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45896/0 (socket says 172.21.15.71:45896) 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08cf400640 1 -- 172.21.15.71:0/2525742644 learned_addr learned my addr 172.21.15.71:0/2525742644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08cfe00640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0101de0 0x7f08d019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.612+0000 7f08d5200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d0106f10 0x7f08d019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cf400640 1 -- 172.21.15.71:0/2525742644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d0106f10 msgr2=0x7f08d019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cf400640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d0106f10 0x7f08d019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cf400640 1 -- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0101de0 msgr2=0x7f08d019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cf400640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0101de0 0x7f08d019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cf400640 1 -- 172.21.15.71:0/2525742644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d019f620 con 0x7f08d0102790 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cfe00640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0101de0 0x7f08d019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.613+0000 7f08cf400640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 0x7f08d019abb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08bc006010 tx=0x7f08bc004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:43.638 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.614+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc03f030 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.614+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08bc004440 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.614+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d019f8b0 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.614+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08bc035070 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.614+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d01a71f0 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.615+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d01021c0 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.617+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f08bc042030 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.618+0000 7f08ccc00640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f08b40a2c10 0x7f08b40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.619+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f08bc0efbf0 con 0x7f08d0102790 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.619+0000 7f08cfe00640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f08b40a2c10 0x7f08b40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.619+0000 7f08cfe00640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f08b40a2c10 0x7f08b40a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009fc0 tx=0x7f08c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:43.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.620+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f08bc0bc0a0 con 0x7f08d0102790 2024-09-05T12:26:43.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.790+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08d0075430 con 0x7f08b40a2c10 2024-09-05T12:26:43.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.799+0000 7f08ccc00640 1 -- 172.21.15.71:0/2525742644 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340422 (secure 0 0 0) 0x7f08d0075430 con 0x7f08b40a2c10 2024-09-05T12:26:43.805 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:43.807 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f08b40a2c10 msgr2=0x7f08b40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f08b40a2c10 0x7f08b40a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009fc0 tx=0x7f08c0005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 msgr2=0x7f08d019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 0x7f08d019abb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f08bc006010 tx=0x7f08bc004040 comp rx=0 tx=0).stop 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 shutdown_connections 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f08b40a2c10 0x7f08b40a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d0106f10 0x7f08d019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08d0102790 0x7f08d019abb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 --2- 172.21.15.71:0/2525742644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0101de0 0x7f08d019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:43.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.803+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 >> 172.21.15.71:0/2525742644 conn(0x7f08d00fd8c0 msgr2=0x7f08d00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:43.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.804+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 shutdown_connections 2024-09-05T12:26:43.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:43.804+0000 7f08d7200640 1 -- 172.21.15.71:0/2525742644 wait complete. 2024-09-05T12:26:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:44 smithi071 ceph-mon[29845]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:44 smithi123 ceph-mon[34321]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:44 smithi174 ceph-mon[34313]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:44.782 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-05T12:26:43.164273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660348,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":778,"num_read_kb":516,"num_write":430,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376544,"kb_used_data":14816,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565280,"statfs":{"total":1151940427776,"available":1151554846720,"internally_reserved":0,"allocated":15171584,"data_stored":8351930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":111,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:37.421323+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:26:37.421323+0000","last_peered":"2024-09-05T12:26:37.421323+0000","last_clean":"2024-09-05T12:26:37.421323+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:26:37.421323+0000","last_undegraded":"2024-09-05T12:26:37.421323+0000","last_fullsized":"2024-09-05T12:26:37.421323+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263356+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:25:34.263356+0000","last_peered":"2024-09-05T12:25:34.263356+0000","last_clean":"2024-09-05T12:25:34.263356+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:25:34.263356+0000","last_undegraded":"2024-09-05T12:25:34.263356+0000","last_fullsized":"2024-09-05T12:25:34.263356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263685+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:25:34.263685+0000","last_peered":"2024-09-05T12:25:34.263685+0000","last_clean":"2024-09-05T12:25:34.263685+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:25:34.263685+0000","last_undegraded":"2024-09-05T12:25:34.263685+0000","last_fullsized":"2024-09-05T12:25:34.263685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263635+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:25:34.263635+0000","last_peered":"2024-09-05T12:25:34.263635+0000","last_clean":"2024-09-05T12:25:34.263635+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:25:34.263635+0000","last_undegraded":"2024-09-05T12:25:34.263635+0000","last_fullsized":"2024-09-05T12:25:34.263635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268221+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:25:34.268221+0000","last_peered":"2024-09-05T12:25:34.268221+0000","last_clean":"2024-09-05T12:25:34.268221+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:25:34.268221+0000","last_undegraded":"2024-09-05T12:25:34.268221+0000","last_fullsized":"2024-09-05T12:25:34.268221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264135+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:25:34.264135+0000","last_peered":"2024-09-05T12:25:34.264135+0000","last_clean":"2024-09-05T12:25:34.264135+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:25:34.264135+0000","last_undegraded":"2024-09-05T12:25:34.264135+0000","last_fullsized":"2024-09-05T12:25:34.264135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.317765+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:25:39.317765+0000","last_peered":"2024-09-05T12:25:39.317765+0000","last_clean":"2024-09-05T12:25:39.317765+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:25:39.317765+0000","last_undegraded":"2024-09-05T12:25:39.317765+0000","last_fullsized":"2024-09-05T12:25:39.317765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404504+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:25:34.404504+0000","last_peered":"2024-09-05T12:25:34.404504+0000","last_clean":"2024-09-05T12:25:34.404504+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:25:34.404504+0000","last_undegraded":"2024-09-05T12:25:34.404504+0000","last_fullsized":"2024-09-05T12:25:34.404504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264558+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:25:34.264558+0000","last_peered":"2024-09-05T12:25:34.264558+0000","last_clean":"2024-09-05T12:25:34.264558+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:25:34.264558+0000","last_undegraded":"2024-09-05T12:25:34.264558+0000","last_fullsized":"2024-09-05T12:25:34.264558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322603+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:25:38.322603+0000","last_peered":"2024-09-05T12:25:38.322603+0000","last_clean":"2024-09-05T12:25:38.322603+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:25:38.322603+0000","last_undegraded":"2024-09-05T12:25:38.322603+0000","last_fullsized":"2024-09-05T12:25:38.322603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271656+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:25:34.271656+0000","last_peered":"2024-09-05T12:25:34.271656+0000","last_clean":"2024-09-05T12:25:34.271656+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:25:34.271656+0000","last_undegraded":"2024-09-05T12:25:34.271656+0000","last_fullsized":"2024-09-05T12:25:34.271656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264243+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:25:34.264243+0000","last_peered":"2024-09-05T12:25:34.264243+0000","last_clean":"2024-09-05T12:25:34.264243+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:25:34.264243+0000","last_undegraded":"2024-09-05T12:25:34.264243+0000","last_fullsized":"2024-09-05T12:25:34.264243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268524+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:25:34.268524+0000","last_peered":"2024-09-05T12:25:34.268524+0000","last_clean":"2024-09-05T12:25:34.268524+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:25:34.268524+0000","last_undegraded":"2024-09-05T12:25:34.268524+0000","last_fullsized":"2024-09-05T12:25:34.268524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.294392+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:25:35.294392+0000","last_peered":"2024-09-05T12:25:35.294392+0000","last_clean":"2024-09-05T12:25:35.294392+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:25:35.294392+0000","last_undegraded":"2024-09-05T12:25:35.294392+0000","last_fullsized":"2024-09-05T12:25:35.294392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264525+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:25:34.264525+0000","last_peered":"2024-09-05T12:25:34.264525+0000","last_clean":"2024-09-05T12:25:34.264525+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:25:34.264525+0000","last_undegraded":"2024-09-05T12:25:34.264525+0000","last_fullsized":"2024-09-05T12:25:34.264525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263138+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:25:34.263138+0000","last_peered":"2024-09-05T12:25:34.263138+0000","last_clean":"2024-09-05T12:25:34.263138+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:25:34.263138+0000","last_undegraded":"2024-09-05T12:25:34.263138+0000","last_fullsized":"2024-09-05T12:25:34.263138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313403+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:25:37.313403+0000","last_peered":"2024-09-05T12:25:37.313403+0000","last_clean":"2024-09-05T12:25:37.313403+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:25:37.313403+0000","last_undegraded":"2024-09-05T12:25:37.313403+0000","last_fullsized":"2024-09-05T12:25:37.313403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271242+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:25:34.271242+0000","last_peered":"2024-09-05T12:25:34.271242+0000","last_clean":"2024-09-05T12:25:34.271242+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:25:34.271242+0000","last_undegraded":"2024-09-05T12:25:34.271242+0000","last_fullsized":"2024-09-05T12:25:34.271242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265378+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:25:34.265378+0000","last_peered":"2024-09-05T12:25:34.265378+0000","last_clean":"2024-09-05T12:25:34.265378+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:25:34.265378+0000","last_undegraded":"2024-09-05T12:25:34.265378+0000","last_fullsized":"2024-09-05T12:25:34.265378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323200+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:25:38.323200+0000","last_peered":"2024-09-05T12:25:38.323200+0000","last_clean":"2024-09-05T12:25:38.323200+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:25:38.323200+0000","last_undegraded":"2024-09-05T12:25:38.323200+0000","last_fullsized":"2024-09-05T12:25:38.323200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330571+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:25:34.330571+0000","last_peered":"2024-09-05T12:25:34.330571+0000","last_clean":"2024-09-05T12:25:34.330571+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:25:34.330571+0000","last_undegraded":"2024-09-05T12:25:34.330571+0000","last_fullsized":"2024-09-05T12:25:34.330571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265675+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:25:34.265675+0000","last_peered":"2024-09-05T12:25:34.265675+0000","last_clean":"2024-09-05T12:25:34.265675+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:25:34.265675+0000","last_undegraded":"2024-09-05T12:25:34.265675+0000","last_fullsized":"2024-09-05T12:25:34.265675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.293537+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:25:35.293537+0000","last_peered":"2024-09-05T12:25:35.293537+0000","last_clean":"2024-09-05T12:25:35.293537+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:25:35.293537+0000","last_undegraded":"2024-09-05T12:25:35.293537+0000","last_fullsized":"2024-09-05T12:25:35.293537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264519+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:25:34.264519+0000","last_peered":"2024-09-05T12:25:34.264519+0000","last_clean":"2024-09-05T12:25:34.264519+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:25:34.264519+0000","last_undegraded":"2024-09-05T12:25:34.264519+0000","last_fullsized":"2024-09-05T12:25:34.264519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268273+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:25:34.268273+0000","last_peered":"2024-09-05T12:25:34.268273+0000","last_clean":"2024-09-05T12:25:34.268273+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:25:34.268273+0000","last_undegraded":"2024-09-05T12:25:34.268273+0000","last_fullsized":"2024-09-05T12:25:34.268273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323273+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:25:38.323273+0000","last_peered":"2024-09-05T12:25:38.323273+0000","last_clean":"2024-09-05T12:25:38.323273+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:25:38.323273+0000","last_undegraded":"2024-09-05T12:25:38.323273+0000","last_fullsized":"2024-09-05T12:25:38.323273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263934+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:25:34.263934+0000","last_peered":"2024-09-05T12:25:34.263934+0000","last_clean":"2024-09-05T12:25:34.263934+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:25:34.263934+0000","last_undegraded":"2024-09-05T12:25:34.263934+0000","last_fullsized":"2024-09-05T12:25:34.263934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263632+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:25:34.263632+0000","last_peered":"2024-09-05T12:25:34.263632+0000","last_clean":"2024-09-05T12:25:34.263632+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:25:34.263632+0000","last_undegraded":"2024-09-05T12:25:34.263632+0000","last_fullsized":"2024-09-05T12:25:34.263632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'14","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:54.522676+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:25:54.522676+0000","last_peered":"2024-09-05T12:25:54.522676+0000","last_clean":"2024-09-05T12:25:54.522676+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:25:54.522676+0000","last_undegraded":"2024-09-05T12:25:54.522676+0000","last_fullsized":"2024-09-05T12:25:54.522676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.262906+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:25:34.262906+0000","last_peered":"2024-09-05T12:25:34.262906+0000","last_clean":"2024-09-05T12:25:34.262906+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:25:34.262906+0000","last_undegraded":"2024-09-05T12:25:34.262906+0000","last_fullsized":"2024-09-05T12:25:34.262906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267595+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:25:34.267595+0000","last_peered":"2024-09-05T12:25:34.267595+0000","last_clean":"2024-09-05T12:25:34.267595+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:25:34.267595+0000","last_undegraded":"2024-09-05T12:25:34.267595+0000","last_fullsized":"2024-09-05T12:25:34.267595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323069+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:25:38.323069+0000","last_peered":"2024-09-05T12:25:38.323069+0000","last_clean":"2024-09-05T12:25:38.323069+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:25:38.323069+0000","last_undegraded":"2024-09-05T12:25:38.323069+0000","last_fullsized":"2024-09-05T12:25:38.323069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265409+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:25:34.265409+0000","last_peered":"2024-09-05T12:25:34.265409+0000","last_clean":"2024-09-05T12:25:34.265409+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:25:34.265409+0000","last_undegraded":"2024-09-05T12:25:34.265409+0000","last_fullsized":"2024-09-05T12:25:34.265409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267791+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:25:34.267791+0000","last_peered":"2024-09-05T12:25:34.267791+0000","last_clean":"2024-09-05T12:25:34.267791+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:25:34.267791+0000","last_undegraded":"2024-09-05T12:25:34.267791+0000","last_fullsized":"2024-09-05T12:25:34.267791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330392+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:25:34.330392+0000","last_peered":"2024-09-05T12:25:34.330392+0000","last_clean":"2024-09-05T12:25:34.330392+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:25:34.330392+0000","last_undegraded":"2024-09-05T12:25:34.330392+0000","last_fullsized":"2024-09-05T12:25:34.330392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266261+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:25:34.266261+0000","last_peered":"2024-09-05T12:25:34.266261+0000","last_clean":"2024-09-05T12:25:34.266261+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:25:34.266261+0000","last_undegraded":"2024-09-05T12:25:34.266261+0000","last_fullsized":"2024-09-05T12:25:34.266261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267674+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:25:34.267674+0000","last_peered":"2024-09-05T12:25:34.267674+0000","last_clean":"2024-09-05T12:25:34.267674+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:25:34.267674+0000","last_undegraded":"2024-09-05T12:25:34.267674+0000","last_fullsized":"2024-09-05T12:25:34.267674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":103,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:37.421803+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:26:37.421803+0000","last_peered":"2024-09-05T12:26:37.421803+0000","last_clean":"2024-09-05T12:26:37.421803+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:26:37.421803+0000","last_undegraded":"2024-09-05T12:26:37.421803+0000","last_fullsized":"2024-09-05T12:26:37.421803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":61,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267415+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:25:34.267415+0000","last_peered":"2024-09-05T12:25:34.267415+0000","last_clean":"2024-09-05T12:25:34.267415+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:25:34.267415+0000","last_undegraded":"2024-09-05T12:25:34.267415+0000","last_fullsized":"2024-09-05T12:25:34.267415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269430+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:25:34.269430+0000","last_peered":"2024-09-05T12:25:34.269430+0000","last_clean":"2024-09-05T12:25:34.269430+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:25:34.269430+0000","last_undegraded":"2024-09-05T12:25:34.269430+0000","last_fullsized":"2024-09-05T12:25:34.269430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:37.412639+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:26:37.412639+0000","last_peered":"2024-09-05T12:26:37.412639+0000","last_clean":"2024-09-05T12:26:37.412639+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:26:37.412639+0000","last_undegraded":"2024-09-05T12:26:37.412639+0000","last_fullsized":"2024-09-05T12:26:37.412639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271660+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:25:34.271660+0000","last_peered":"2024-09-05T12:25:34.271660+0000","last_clean":"2024-09-05T12:25:34.271660+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:25:34.271660+0000","last_undegraded":"2024-09-05T12:25:34.271660+0000","last_fullsized":"2024-09-05T12:25:34.271660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266417+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:25:34.266417+0000","last_peered":"2024-09-05T12:25:34.266417+0000","last_clean":"2024-09-05T12:25:34.266417+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:25:34.266417+0000","last_undegraded":"2024-09-05T12:25:34.266417+0000","last_fullsized":"2024-09-05T12:25:34.266417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.299617+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:25:36.299617+0000","last_peered":"2024-09-05T12:25:36.299617+0000","last_clean":"2024-09-05T12:25:36.299617+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:25:36.299617+0000","last_undegraded":"2024-09-05T12:25:36.299617+0000","last_fullsized":"2024-09-05T12:25:36.299617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272425+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:25:34.272425+0000","last_peered":"2024-09-05T12:25:34.272425+0000","last_clean":"2024-09-05T12:25:34.272425+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:25:34.272425+0000","last_undegraded":"2024-09-05T12:25:34.272425+0000","last_fullsized":"2024-09-05T12:25:34.272425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263271+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:25:34.263271+0000","last_peered":"2024-09-05T12:25:34.263271+0000","last_clean":"2024-09-05T12:25:34.263271+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:25:34.263271+0000","last_undegraded":"2024-09-05T12:25:34.263271+0000","last_fullsized":"2024-09-05T12:25:34.263271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.325619+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:25:39.325619+0000","last_peered":"2024-09-05T12:25:39.325619+0000","last_clean":"2024-09-05T12:25:39.325619+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:25:39.325619+0000","last_undegraded":"2024-09-05T12:25:39.325619+0000","last_fullsized":"2024-09-05T12:25:39.325619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269785+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:25:34.269785+0000","last_peered":"2024-09-05T12:25:34.269785+0000","last_clean":"2024-09-05T12:25:34.269785+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:25:34.269785+0000","last_undegraded":"2024-09-05T12:25:34.269785+0000","last_fullsized":"2024-09-05T12:25:34.269785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268797+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:25:34.268797+0000","last_peered":"2024-09-05T12:25:34.268797+0000","last_clean":"2024-09-05T12:25:34.268797+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:25:34.268797+0000","last_undegraded":"2024-09-05T12:25:34.268797+0000","last_fullsized":"2024-09-05T12:25:34.268797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269019+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:25:34.269019+0000","last_peered":"2024-09-05T12:25:34.269019+0000","last_clean":"2024-09-05T12:25:34.269019+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:25:34.269019+0000","last_undegraded":"2024-09-05T12:25:34.269019+0000","last_fullsized":"2024-09-05T12:25:34.269019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'6","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:36.409384+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:26:36.409384+0000","last_peered":"2024-09-05T12:26:36.409384+0000","last_clean":"2024-09-05T12:26:36.409384+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:26:36.409384+0000","last_undegraded":"2024-09-05T12:26:36.409384+0000","last_fullsized":"2024-09-05T12:26:36.409384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404323+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:25:34.404323+0000","last_peered":"2024-09-05T12:25:34.404323+0000","last_clean":"2024-09-05T12:25:34.404323+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:25:34.404323+0000","last_undegraded":"2024-09-05T12:25:34.404323+0000","last_fullsized":"2024-09-05T12:25:34.404323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269554+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:25:34.269554+0000","last_peered":"2024-09-05T12:25:34.269554+0000","last_clean":"2024-09-05T12:25:34.269554+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:25:34.269554+0000","last_undegraded":"2024-09-05T12:25:34.269554+0000","last_fullsized":"2024-09-05T12:25:34.269554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321412+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:25:38.321412+0000","last_peered":"2024-09-05T12:25:38.321412+0000","last_clean":"2024-09-05T12:25:38.321412+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:25:38.321412+0000","last_undegraded":"2024-09-05T12:25:38.321412+0000","last_fullsized":"2024-09-05T12:25:38.321412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267091+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:25:34.267091+0000","last_peered":"2024-09-05T12:25:34.267091+0000","last_clean":"2024-09-05T12:25:34.267091+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:25:34.267091+0000","last_undegraded":"2024-09-05T12:25:34.267091+0000","last_fullsized":"2024-09-05T12:25:34.267091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268658+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:25:34.268658+0000","last_peered":"2024-09-05T12:25:34.268658+0000","last_clean":"2024-09-05T12:25:34.268658+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:25:34.268658+0000","last_undegraded":"2024-09-05T12:25:34.268658+0000","last_fullsized":"2024-09-05T12:25:34.268658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323388+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:25:38.323388+0000","last_peered":"2024-09-05T12:25:38.323388+0000","last_clean":"2024-09-05T12:25:38.323388+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:25:38.323388+0000","last_undegraded":"2024-09-05T12:25:38.323388+0000","last_fullsized":"2024-09-05T12:25:38.323388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269057+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:25:34.269057+0000","last_peered":"2024-09-05T12:25:34.269057+0000","last_clean":"2024-09-05T12:25:34.269057+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:25:34.269057+0000","last_undegraded":"2024-09-05T12:25:34.269057+0000","last_fullsized":"2024-09-05T12:25:34.269057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266821+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:25:34.266821+0000","last_peered":"2024-09-05T12:25:34.266821+0000","last_clean":"2024-09-05T12:25:34.266821+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:25:34.266821+0000","last_undegraded":"2024-09-05T12:25:34.266821+0000","last_fullsized":"2024-09-05T12:25:34.266821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.296703+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:25:38.296703+0000","last_peered":"2024-09-05T12:25:38.296703+0000","last_clean":"2024-09-05T12:25:38.296703+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:25:38.296703+0000","last_undegraded":"2024-09-05T12:25:38.296703+0000","last_fullsized":"2024-09-05T12:25:38.296703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270206+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:25:34.270206+0000","last_peered":"2024-09-05T12:25:34.270206+0000","last_clean":"2024-09-05T12:25:34.270206+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:25:34.270206+0000","last_undegraded":"2024-09-05T12:25:34.270206+0000","last_fullsized":"2024-09-05T12:25:34.270206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270383+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:25:34.270383+0000","last_peered":"2024-09-05T12:25:34.270383+0000","last_clean":"2024-09-05T12:25:34.270383+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:25:34.270383+0000","last_undegraded":"2024-09-05T12:25:34.270383+0000","last_fullsized":"2024-09-05T12:25:34.270383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321005+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:25:38.321005+0000","last_peered":"2024-09-05T12:25:38.321005+0000","last_clean":"2024-09-05T12:25:38.321005+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:25:38.321005+0000","last_undegraded":"2024-09-05T12:25:38.321005+0000","last_fullsized":"2024-09-05T12:25:38.321005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267124+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:25:34.267124+0000","last_peered":"2024-09-05T12:25:34.267124+0000","last_clean":"2024-09-05T12:25:34.267124+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:25:34.267124+0000","last_undegraded":"2024-09-05T12:25:34.267124+0000","last_fullsized":"2024-09-05T12:25:34.267124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331448+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:25:34.331448+0000","last_peered":"2024-09-05T12:25:34.331448+0000","last_clean":"2024-09-05T12:25:34.331448+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:25:34.331448+0000","last_undegraded":"2024-09-05T12:25:34.331448+0000","last_fullsized":"2024-09-05T12:25:34.331448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":104,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:37.413181+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:26:37.413181+0000","last_peered":"2024-09-05T12:26:37.413181+0000","last_clean":"2024-09-05T12:26:37.413181+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:26:37.413181+0000","last_undegraded":"2024-09-05T12:26:37.413181+0000","last_fullsized":"2024-09-05T12:26:37.413181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330666+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:25:34.330666+0000","last_peered":"2024-09-05T12:25:34.330666+0000","last_clean":"2024-09-05T12:25:34.330666+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:25:34.330666+0000","last_undegraded":"2024-09-05T12:25:34.330666+0000","last_fullsized":"2024-09-05T12:25:34.330666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268645+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:25:34.268645+0000","last_peered":"2024-09-05T12:25:34.268645+0000","last_clean":"2024-09-05T12:25:34.268645+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:25:34.268645+0000","last_undegraded":"2024-09-05T12:25:34.268645+0000","last_fullsized":"2024-09-05T12:25:34.268645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322440+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:25:38.322440+0000","last_peered":"2024-09-05T12:25:38.322440+0000","last_clean":"2024-09-05T12:25:38.322440+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:25:38.322440+0000","last_undegraded":"2024-09-05T12:25:38.322440+0000","last_fullsized":"2024-09-05T12:25:38.322440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271674+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:25:34.271674+0000","last_peered":"2024-09-05T12:25:34.271674+0000","last_clean":"2024-09-05T12:25:34.271674+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:25:34.271674+0000","last_undegraded":"2024-09-05T12:25:34.271674+0000","last_fullsized":"2024-09-05T12:25:34.271674+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266874+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:25:34.266874+0000","last_peered":"2024-09-05T12:25:34.266874+0000","last_clean":"2024-09-05T12:25:34.266874+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:25:34.266874+0000","last_undegraded":"2024-09-05T12:25:34.266874+0000","last_fullsized":"2024-09-05T12:25:34.266874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.326756+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:25:39.326756+0000","last_peered":"2024-09-05T12:25:39.326756+0000","last_clean":"2024-09-05T12:25:39.326756+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:25:39.326756+0000","last_undegraded":"2024-09-05T12:25:39.326756+0000","last_fullsized":"2024-09-05T12:25:39.326756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269780+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:25:34.269780+0000","last_peered":"2024-09-05T12:25:34.269780+0000","last_clean":"2024-09-05T12:25:34.269780+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:25:34.269780+0000","last_undegraded":"2024-09-05T12:25:34.269780+0000","last_fullsized":"2024-09-05T12:25:34.269780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266924+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:25:34.266924+0000","last_peered":"2024-09-05T12:25:34.266924+0000","last_clean":"2024-09-05T12:25:34.266924+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:25:34.266924+0000","last_undegraded":"2024-09-05T12:25:34.266924+0000","last_fullsized":"2024-09-05T12:25:34.266924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":201,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:38.411649+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:26:38.411649+0000","last_peered":"2024-09-05T12:26:38.411649+0000","last_clean":"2024-09-05T12:26:38.411649+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:26:38.411649+0000","last_undegraded":"2024-09-05T12:26:38.411649+0000","last_fullsized":"2024-09-05T12:26:38.411649+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62931,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":126,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330067+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:25:34.330067+0000","last_peered":"2024-09-05T12:25:34.330067+0000","last_clean":"2024-09-05T12:25:34.330067+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:25:34.330067+0000","last_undegraded":"2024-09-05T12:25:34.330067+0000","last_fullsized":"2024-09-05T12:25:34.330067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404529+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:25:34.404529+0000","last_peered":"2024-09-05T12:25:34.404529+0000","last_clean":"2024-09-05T12:25:34.404529+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:25:34.404529+0000","last_undegraded":"2024-09-05T12:25:34.404529+0000","last_fullsized":"2024-09-05T12:25:34.404529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313398+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:25:37.313398+0000","last_peered":"2024-09-05T12:25:37.313398+0000","last_clean":"2024-09-05T12:25:37.313398+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:25:37.313398+0000","last_undegraded":"2024-09-05T12:25:37.313398+0000","last_fullsized":"2024-09-05T12:25:37.313398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264502+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:25:34.264502+0000","last_peered":"2024-09-05T12:25:34.264502+0000","last_clean":"2024-09-05T12:25:34.264502+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:25:34.264502+0000","last_undegraded":"2024-09-05T12:25:34.264502+0000","last_fullsized":"2024-09-05T12:25:34.264502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268248+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:25:34.268248+0000","last_peered":"2024-09-05T12:25:34.268248+0000","last_clean":"2024-09-05T12:25:34.268248+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:25:34.268248+0000","last_undegraded":"2024-09-05T12:25:34.268248+0000","last_fullsized":"2024-09-05T12:25:34.268248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321510+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:25:38.321510+0000","last_peered":"2024-09-05T12:25:38.321510+0000","last_clean":"2024-09-05T12:25:38.321510+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:25:38.321510+0000","last_undegraded":"2024-09-05T12:25:38.321510+0000","last_fullsized":"2024-09-05T12:25:38.321510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272597+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:25:34.272597+0000","last_peered":"2024-09-05T12:25:34.272597+0000","last_clean":"2024-09-05T12:25:34.272597+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:25:34.272597+0000","last_undegraded":"2024-09-05T12:25:34.272597+0000","last_fullsized":"2024-09-05T12:25:34.272597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404720+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:25:34.404720+0000","last_peered":"2024-09-05T12:25:34.404720+0000","last_clean":"2024-09-05T12:25:34.404720+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:25:34.404720+0000","last_undegraded":"2024-09-05T12:25:34.404720+0000","last_fullsized":"2024-09-05T12:25:34.404720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":161,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:38.411609+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:26:38.411609+0000","last_peered":"2024-09-05T12:26:38.411609+0000","last_clean":"2024-09-05T12:26:38.411609+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:26:38.411609+0000","last_undegraded":"2024-09-05T12:26:38.411609+0000","last_fullsized":"2024-09-05T12:26:38.411609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263017+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:25:34.263017+0000","last_peered":"2024-09-05T12:25:34.263017+0000","last_clean":"2024-09-05T12:25:34.263017+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:25:34.263017+0000","last_undegraded":"2024-09-05T12:25:34.263017+0000","last_fullsized":"2024-09-05T12:25:34.263017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268073+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:25:34.268073+0000","last_peered":"2024-09-05T12:25:34.268073+0000","last_clean":"2024-09-05T12:25:34.268073+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:25:34.268073+0000","last_undegraded":"2024-09-05T12:25:34.268073+0000","last_fullsized":"2024-09-05T12:25:34.268073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.329887+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:25:34.329887+0000","last_peered":"2024-09-05T12:25:34.329887+0000","last_clean":"2024-09-05T12:25:34.329887+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:25:34.329887+0000","last_undegraded":"2024-09-05T12:25:34.329887+0000","last_fullsized":"2024-09-05T12:25:34.329887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322579+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:25:38.322579+0000","last_peered":"2024-09-05T12:25:38.322579+0000","last_clean":"2024-09-05T12:25:38.322579+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:25:38.322579+0000","last_undegraded":"2024-09-05T12:25:38.322579+0000","last_fullsized":"2024-09-05T12:25:38.322579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267082+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:25:34.267082+0000","last_peered":"2024-09-05T12:25:34.267082+0000","last_clean":"2024-09-05T12:25:34.267082+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:25:34.267082+0000","last_undegraded":"2024-09-05T12:25:34.267082+0000","last_fullsized":"2024-09-05T12:25:34.267082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263148+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:25:34.263148+0000","last_peered":"2024-09-05T12:25:34.263148+0000","last_clean":"2024-09-05T12:25:34.263148+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:25:34.263148+0000","last_undegraded":"2024-09-05T12:25:34.263148+0000","last_fullsized":"2024-09-05T12:25:34.263148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.302852+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:25:36.302852+0000","last_peered":"2024-09-05T12:25:36.302852+0000","last_clean":"2024-09-05T12:25:36.302852+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:25:36.302852+0000","last_undegraded":"2024-09-05T12:25:36.302852+0000","last_fullsized":"2024-09-05T12:25:36.302852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313373+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:25:37.313373+0000","last_peered":"2024-09-05T12:25:37.313373+0000","last_clean":"2024-09-05T12:25:37.313373+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:25:37.313373+0000","last_undegraded":"2024-09-05T12:25:37.313373+0000","last_fullsized":"2024-09-05T12:25:37.313373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269899+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:25:34.269899+0000","last_peered":"2024-09-05T12:25:34.269899+0000","last_clean":"2024-09-05T12:25:34.269899+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:25:34.269899+0000","last_undegraded":"2024-09-05T12:25:34.269899+0000","last_fullsized":"2024-09-05T12:25:34.269899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264384+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:25:34.264384+0000","last_peered":"2024-09-05T12:25:34.264384+0000","last_clean":"2024-09-05T12:25:34.264384+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:25:34.264384+0000","last_undegraded":"2024-09-05T12:25:34.264384+0000","last_fullsized":"2024-09-05T12:25:34.264384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":108,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:40.414900+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:26:40.414900+0000","last_peered":"2024-09-05T12:26:40.414900+0000","last_clean":"2024-09-05T12:26:40.414900+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:26:40.414900+0000","last_undegraded":"2024-09-05T12:26:40.414900+0000","last_fullsized":"2024-09-05T12:26:40.414900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266460+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:25:34.266460+0000","last_peered":"2024-09-05T12:25:34.266460+0000","last_clean":"2024-09-05T12:25:34.266460+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:25:34.266460+0000","last_undegraded":"2024-09-05T12:25:34.266460+0000","last_fullsized":"2024-09-05T12:25:34.266460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264349+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:25:34.264349+0000","last_peered":"2024-09-05T12:25:34.264349+0000","last_clean":"2024-09-05T12:25:34.264349+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:25:34.264349+0000","last_undegraded":"2024-09-05T12:25:34.264349+0000","last_fullsized":"2024-09-05T12:25:34.264349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265242+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:25:34.265242+0000","last_peered":"2024-09-05T12:25:34.265242+0000","last_clean":"2024-09-05T12:25:34.265242+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:25:34.265242+0000","last_undegraded":"2024-09-05T12:25:34.265242+0000","last_fullsized":"2024-09-05T12:25:34.265242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264119+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:25:34.264119+0000","last_peered":"2024-09-05T12:25:34.264119+0000","last_clean":"2024-09-05T12:25:34.264119+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:25:34.264119+0000","last_undegraded":"2024-09-05T12:25:34.264119+0000","last_fullsized":"2024-09-05T12:25:34.264119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263751+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:25:34.263751+0000","last_peered":"2024-09-05T12:25:34.263751+0000","last_clean":"2024-09-05T12:25:34.263751+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:25:34.263751+0000","last_undegraded":"2024-09-05T12:25:34.263751+0000","last_fullsized":"2024-09-05T12:25:34.263751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270400+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:25:34.270400+0000","last_peered":"2024-09-05T12:25:34.270400+0000","last_clean":"2024-09-05T12:25:34.270400+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:25:34.270400+0000","last_undegraded":"2024-09-05T12:25:34.270400+0000","last_fullsized":"2024-09-05T12:25:34.270400+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331799+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:25:34.331799+0000","last_peered":"2024-09-05T12:25:34.331799+0000","last_clean":"2024-09-05T12:25:34.331799+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:25:34.331799+0000","last_undegraded":"2024-09-05T12:25:34.331799+0000","last_fullsized":"2024-09-05T12:25:34.331799+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266950+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:25:34.266950+0000","last_peered":"2024-09-05T12:25:34.266950+0000","last_clean":"2024-09-05T12:25:34.266950+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:25:34.266950+0000","last_undegraded":"2024-09-05T12:25:34.266950+0000","last_fullsized":"2024-09-05T12:25:34.266950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264875+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:25:34.264875+0000","last_peered":"2024-09-05T12:25:34.264875+0000","last_clean":"2024-09-05T12:25:34.264875+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:25:34.264875+0000","last_undegraded":"2024-09-05T12:25:34.264875+0000","last_fullsized":"2024-09-05T12:25:34.264875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263394+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:25:34.263394+0000","last_peered":"2024-09-05T12:25:34.263394+0000","last_clean":"2024-09-05T12:25:34.263394+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:25:34.263394+0000","last_undegraded":"2024-09-05T12:25:34.263394+0000","last_fullsized":"2024-09-05T12:25:34.263394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269818+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:25:34.269818+0000","last_peered":"2024-09-05T12:25:34.269818+0000","last_clean":"2024-09-05T12:25:34.269818+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:25:34.269818+0000","last_undegraded":"2024-09-05T12:25:34.269818+0000","last_fullsized":"2024-09-05T12:25:34.269818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263113+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:25:34.263113+0000","last_peered":"2024-09-05T12:25:34.263113+0000","last_clean":"2024-09-05T12:25:34.263113+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:25:34.263113+0000","last_undegraded":"2024-09-05T12:25:34.263113+0000","last_fullsized":"2024-09-05T12:25:34.263113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404441+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:25:34.404441+0000","last_peered":"2024-09-05T12:25:34.404441+0000","last_clean":"2024-09-05T12:25:34.404441+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:25:34.404441+0000","last_undegraded":"2024-09-05T12:25:34.404441+0000","last_fullsized":"2024-09-05T12:25:34.404441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267531+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:25:34.267531+0000","last_peered":"2024-09-05T12:25:34.267531+0000","last_clean":"2024-09-05T12:25:34.267531+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:25:34.267531+0000","last_undegraded":"2024-09-05T12:25:34.267531+0000","last_fullsized":"2024-09-05T12:25:34.267531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266222+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:25:34.266222+0000","last_peered":"2024-09-05T12:25:34.266222+0000","last_clean":"2024-09-05T12:25:34.266222+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:25:34.266222+0000","last_undegraded":"2024-09-05T12:25:34.266222+0000","last_fullsized":"2024-09-05T12:25:34.266222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264463+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:25:34.264463+0000","last_peered":"2024-09-05T12:25:34.264463+0000","last_clean":"2024-09-05T12:25:34.264463+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:25:34.264463+0000","last_undegraded":"2024-09-05T12:25:34.264463+0000","last_fullsized":"2024-09-05T12:25:34.264463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330539+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:25:34.330539+0000","last_peered":"2024-09-05T12:25:34.330539+0000","last_clean":"2024-09-05T12:25:34.330539+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:25:34.330539+0000","last_undegraded":"2024-09-05T12:25:34.330539+0000","last_fullsized":"2024-09-05T12:25:34.330539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267312+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:25:34.267312+0000","last_peered":"2024-09-05T12:25:34.267312+0000","last_clean":"2024-09-05T12:25:34.267312+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:25:34.267312+0000","last_undegraded":"2024-09-05T12:25:34.267312+0000","last_fullsized":"2024-09-05T12:25:34.267312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404202+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:25:34.404202+0000","last_peered":"2024-09-05T12:25:34.404202+0000","last_clean":"2024-09-05T12:25:34.404202+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:25:34.404202+0000","last_undegraded":"2024-09-05T12:25:34.404202+0000","last_fullsized":"2024-09-05T12:25:34.404202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266403+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:25:34.266403+0000","last_peered":"2024-09-05T12:25:34.266403+0000","last_clean":"2024-09-05T12:25:34.266403+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:25:34.266403+0000","last_undegraded":"2024-09-05T12:25:34.266403+0000","last_fullsized":"2024-09-05T12:25:34.266403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266842+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:25:34.266842+0000","last_peered":"2024-09-05T12:25:34.266842+0000","last_clean":"2024-09-05T12:25:34.266842+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:25:34.266842+0000","last_undegraded":"2024-09-05T12:25:34.266842+0000","last_fullsized":"2024-09-05T12:25:34.266842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267435+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:25:34.267435+0000","last_peered":"2024-09-05T12:25:34.267435+0000","last_clean":"2024-09-05T12:25:34.267435+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:25:34.267435+0000","last_undegraded":"2024-09-05T12:25:34.267435+0000","last_fullsized":"2024-09-05T12:25:34.267435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265181+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:25:34.265181+0000","last_peered":"2024-09-05T12:25:34.265181+0000","last_clean":"2024-09-05T12:25:34.265181+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:25:34.265181+0000","last_undegraded":"2024-09-05T12:25:34.265181+0000","last_fullsized":"2024-09-05T12:25:34.265181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268910+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:25:34.268910+0000","last_peered":"2024-09-05T12:25:34.268910+0000","last_clean":"2024-09-05T12:25:34.268910+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:25:34.268910+0000","last_undegraded":"2024-09-05T12:25:34.268910+0000","last_fullsized":"2024-09-05T12:25:34.268910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331174+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:25:34.331174+0000","last_peered":"2024-09-05T12:25:34.331174+0000","last_clean":"2024-09-05T12:25:34.331174+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:25:34.331174+0000","last_undegraded":"2024-09-05T12:25:34.331174+0000","last_fullsized":"2024-09-05T12:25:34.331174+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404780+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:25:34.404780+0000","last_peered":"2024-09-05T12:25:34.404780+0000","last_clean":"2024-09-05T12:25:34.404780+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:25:34.404780+0000","last_undegraded":"2024-09-05T12:25:34.404780+0000","last_fullsized":"2024-09-05T12:25:34.404780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263044+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:25:34.263044+0000","last_peered":"2024-09-05T12:25:34.263044+0000","last_clean":"2024-09-05T12:25:34.263044+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:25:34.263044+0000","last_undegraded":"2024-09-05T12:25:34.263044+0000","last_fullsized":"2024-09-05T12:25:34.263044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268375+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:25:34.268375+0000","last_peered":"2024-09-05T12:25:34.268375+0000","last_clean":"2024-09-05T12:25:34.268375+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:25:34.268375+0000","last_undegraded":"2024-09-05T12:25:34.268375+0000","last_fullsized":"2024-09-05T12:25:34.268375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267499+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:25:34.267499+0000","last_peered":"2024-09-05T12:25:34.267499+0000","last_clean":"2024-09-05T12:25:34.267499+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:25:34.267499+0000","last_undegraded":"2024-09-05T12:25:34.267499+0000","last_fullsized":"2024-09-05T12:25:34.267499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265767+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:25:34.265767+0000","last_peered":"2024-09-05T12:25:34.265767+0000","last_clean":"2024-09-05T12:25:34.265767+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:25:34.265767+0000","last_undegraded":"2024-09-05T12:25:34.265767+0000","last_fullsized":"2024-09-05T12:25:34.265767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265061+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:25:34.265061+0000","last_peered":"2024-09-05T12:25:34.265061+0000","last_clean":"2024-09-05T12:25:34.265061+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:25:34.265061+0000","last_undegraded":"2024-09-05T12:25:34.265061+0000","last_fullsized":"2024-09-05T12:25:34.265061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267876+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:25:34.267876+0000","last_peered":"2024-09-05T12:25:34.267876+0000","last_clean":"2024-09-05T12:25:34.267876+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:25:34.267876+0000","last_undegraded":"2024-09-05T12:25:34.267876+0000","last_fullsized":"2024-09-05T12:25:34.267876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268171+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:25:34.268171+0000","last_peered":"2024-09-05T12:25:34.268171+0000","last_clean":"2024-09-05T12:25:34.268171+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:25:34.268171+0000","last_undegraded":"2024-09-05T12:25:34.268171+0000","last_fullsized":"2024-09-05T12:25:34.268171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267268+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:25:34.267268+0000","last_peered":"2024-09-05T12:25:34.267268+0000","last_clean":"2024-09-05T12:25:34.267268+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:25:34.267268+0000","last_undegraded":"2024-09-05T12:25:34.267268+0000","last_fullsized":"2024-09-05T12:25:34.267268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69746,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":642,"num_read_kb":397,"num_write":247,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":209238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449125,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":533938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678057,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1781760,"data_stored":1187366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841582,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":533453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103220,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":533809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1699840,"data_stored":1129937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":528957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1708032,"data_stored":1125519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953545,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":533031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":528665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411411,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1171456,"data_stored":593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574938,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1150976,"data_stored":592918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":530806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:26:44.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:26:45.155 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:45.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:45 smithi071 ceph-mon[29845]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:45.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:45 smithi123 ceph-mon[34321]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:45.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:45 smithi174 ceph-mon[34313]: from='client.24917 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 -- 172.21.15.71:0/3305942288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 msgr2=0x7f06341124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 --2- 172.21.15.71:0/3305942288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341124f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0620009900 tx=0x7f062002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 -- 172.21.15.71:0/3305942288 shutdown_connections 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 --2- 172.21.15.71:0/3305942288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634112a30 0x7f0634114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 --2- 172.21.15.71:0/3305942288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341124f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 --2- 172.21.15.71:0/3305942288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634076720 0x7f0634076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 -- 172.21.15.71:0/3305942288 >> 172.21.15.71:0/3305942288 conn(0x7f06340ffa90 msgr2=0x7f0634101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 -- 172.21.15.71:0/3305942288 shutdown_connections 2024-09-05T12:26:45.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.787+0000 7f063b600640 1 -- 172.21.15.71:0/3305942288 wait complete. 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.788+0000 7f063b600640 1 Processor -- start 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.788+0000 7f063b600640 1 -- start start 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f063b600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 0x7f06341a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f063b600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f063b600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634112a30 0x7f063419e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f063b600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06341185a0 con 0x7f0634077040 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f063b600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0634118420 con 0x7f0634112a30 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f063b600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0634118720 con 0x7f0634076720 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f0633e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f0638c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 0x7f06341a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f0639600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634112a30 0x7f063419e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f0633e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341a4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50532/0 (socket says 172.21.15.71:50532) 2024-09-05T12:26:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f0638c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 0x7f06341a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46412/0 (socket says 172.21.15.71:46412) 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.789+0000 7f0633e00640 1 -- 172.21.15.71:0/3420383489 learned_addr learned my addr 172.21.15.71:0/3420383489 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0639600640 1 -- 172.21.15.71:0/3420383489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 msgr2=0x7f06341a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0639600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 0x7f06341a47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0639600640 1 -- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 msgr2=0x7f06341a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0639600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0639600640 1 -- 172.21.15.71:0/3420383489 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f063419f0c0 con 0x7f0634112a30 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0638c00640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 0x7f06341a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0639600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634112a30 0x7f063419e990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f063407aa10 tx=0x7f062c00dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c00bd40 con 0x7f0634112a30 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f062c003e50 con 0x7f0634112a30 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f063419f3b0 con 0x7f0634112a30 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062c0106c0 con 0x7f0634112a30 2024-09-05T12:26:45.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.790+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341ab6a0 con 0x7f0634112a30 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.791+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0634076b00 con 0x7f0634112a30 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.793+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f062c003970 con 0x7f0634112a30 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.795+0000 7f0631600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f061c0a2c90 0x7f061c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.795+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f062c0cfa90 con 0x7f0634112a30 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.795+0000 7f0638c00640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f061c0a2c90 0x7f061c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.796+0000 7f0638c00640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f061c0a2c90 0x7f061c0a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f063419fad0 tx=0x7f0628005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:45.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.797+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f062c09c5b0 con 0x7f0634112a30 2024-09-05T12:26:45.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.970+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0634062630 con 0x7f061c0a2c90 2024-09-05T12:26:45.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.979+0000 7f0631600640 1 -- 172.21.15.71:0/3420383489 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340422 (secure 0 0 0) 0x7f0634062630 con 0x7f061c0a2c90 2024-09-05T12:26:45.985 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f061c0a2c90 msgr2=0x7f061c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f061c0a2c90 0x7f061c0a5150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f063419fad0 tx=0x7f0628005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634112a30 msgr2=0x7f063419e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634112a30 0x7f063419e990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f063407aa10 tx=0x7f062c00dea0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 shutdown_connections 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f061c0a2c90 0x7f061c0a5150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0634112a30 0x7f063419e990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0634077040 0x7f06341a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 --2- 172.21.15.71:0/3420383489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0634076720 0x7f06341a47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 >> 172.21.15.71:0/3420383489 conn(0x7f06340ffa90 msgr2=0x7f0634101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 shutdown_connections 2024-09-05T12:26:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:45.984+0000 7f063b600640 1 -- 172.21.15.71:0/3420383489 wait complete. 2024-09-05T12:26:46.581 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:46 smithi123 ceph-mon[34321]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:26:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:46 smithi071 ceph-mon[29845]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:26:46.661 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:46 smithi174 ceph-mon[34313]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:26:46.820 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-05T12:26:45.164742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660348,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":823,"num_read_kb":541,"num_write":430,"num_write_kb":3081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":351,"ondisk_log_size":351,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376544,"kb_used_data":14816,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565280,"statfs":{"total":1151940427776,"available":1151554846720,"internally_reserved":0,"allocated":15171584,"data_stored":8351930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":75,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002883"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":116,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:42.427958+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:26:42.427958+0000","last_peered":"2024-09-05T12:26:42.427958+0000","last_clean":"2024-09-05T12:26:42.427958+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:26:42.427958+0000","last_undegraded":"2024-09-05T12:26:42.427958+0000","last_fullsized":"2024-09-05T12:26:42.427958+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263356+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:25:34.263356+0000","last_peered":"2024-09-05T12:25:34.263356+0000","last_clean":"2024-09-05T12:25:34.263356+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:25:34.263356+0000","last_undegraded":"2024-09-05T12:25:34.263356+0000","last_fullsized":"2024-09-05T12:25:34.263356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263685+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:25:34.263685+0000","last_peered":"2024-09-05T12:25:34.263685+0000","last_clean":"2024-09-05T12:25:34.263685+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:25:34.263685+0000","last_undegraded":"2024-09-05T12:25:34.263685+0000","last_fullsized":"2024-09-05T12:25:34.263685+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263635+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:25:34.263635+0000","last_peered":"2024-09-05T12:25:34.263635+0000","last_clean":"2024-09-05T12:25:34.263635+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:25:34.263635+0000","last_undegraded":"2024-09-05T12:25:34.263635+0000","last_fullsized":"2024-09-05T12:25:34.263635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268221+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:25:34.268221+0000","last_peered":"2024-09-05T12:25:34.268221+0000","last_clean":"2024-09-05T12:25:34.268221+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:25:34.268221+0000","last_undegraded":"2024-09-05T12:25:34.268221+0000","last_fullsized":"2024-09-05T12:25:34.268221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264135+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:25:34.264135+0000","last_peered":"2024-09-05T12:25:34.264135+0000","last_clean":"2024-09-05T12:25:34.264135+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:25:34.264135+0000","last_undegraded":"2024-09-05T12:25:34.264135+0000","last_fullsized":"2024-09-05T12:25:34.264135+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.317765+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:25:39.317765+0000","last_peered":"2024-09-05T12:25:39.317765+0000","last_clean":"2024-09-05T12:25:39.317765+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:25:39.317765+0000","last_undegraded":"2024-09-05T12:25:39.317765+0000","last_fullsized":"2024-09-05T12:25:39.317765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404504+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:25:34.404504+0000","last_peered":"2024-09-05T12:25:34.404504+0000","last_clean":"2024-09-05T12:25:34.404504+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:25:34.404504+0000","last_undegraded":"2024-09-05T12:25:34.404504+0000","last_fullsized":"2024-09-05T12:25:34.404504+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264558+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:25:34.264558+0000","last_peered":"2024-09-05T12:25:34.264558+0000","last_clean":"2024-09-05T12:25:34.264558+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:25:34.264558+0000","last_undegraded":"2024-09-05T12:25:34.264558+0000","last_fullsized":"2024-09-05T12:25:34.264558+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322603+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:25:38.322603+0000","last_peered":"2024-09-05T12:25:38.322603+0000","last_clean":"2024-09-05T12:25:38.322603+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:25:38.322603+0000","last_undegraded":"2024-09-05T12:25:38.322603+0000","last_fullsized":"2024-09-05T12:25:38.322603+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271656+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:25:34.271656+0000","last_peered":"2024-09-05T12:25:34.271656+0000","last_clean":"2024-09-05T12:25:34.271656+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:25:34.271656+0000","last_undegraded":"2024-09-05T12:25:34.271656+0000","last_fullsized":"2024-09-05T12:25:34.271656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264243+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:25:34.264243+0000","last_peered":"2024-09-05T12:25:34.264243+0000","last_clean":"2024-09-05T12:25:34.264243+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:25:34.264243+0000","last_undegraded":"2024-09-05T12:25:34.264243+0000","last_fullsized":"2024-09-05T12:25:34.264243+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268524+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:25:34.268524+0000","last_peered":"2024-09-05T12:25:34.268524+0000","last_clean":"2024-09-05T12:25:34.268524+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:25:34.268524+0000","last_undegraded":"2024-09-05T12:25:34.268524+0000","last_fullsized":"2024-09-05T12:25:34.268524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.294392+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:25:35.294392+0000","last_peered":"2024-09-05T12:25:35.294392+0000","last_clean":"2024-09-05T12:25:35.294392+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:25:35.294392+0000","last_undegraded":"2024-09-05T12:25:35.294392+0000","last_fullsized":"2024-09-05T12:25:35.294392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264525+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:25:34.264525+0000","last_peered":"2024-09-05T12:25:34.264525+0000","last_clean":"2024-09-05T12:25:34.264525+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:25:34.264525+0000","last_undegraded":"2024-09-05T12:25:34.264525+0000","last_fullsized":"2024-09-05T12:25:34.264525+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263138+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:25:34.263138+0000","last_peered":"2024-09-05T12:25:34.263138+0000","last_clean":"2024-09-05T12:25:34.263138+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:25:34.263138+0000","last_undegraded":"2024-09-05T12:25:34.263138+0000","last_fullsized":"2024-09-05T12:25:34.263138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":14,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313403+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:25:37.313403+0000","last_peered":"2024-09-05T12:25:37.313403+0000","last_clean":"2024-09-05T12:25:37.313403+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:25:37.313403+0000","last_undegraded":"2024-09-05T12:25:37.313403+0000","last_fullsized":"2024-09-05T12:25:37.313403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271242+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:25:34.271242+0000","last_peered":"2024-09-05T12:25:34.271242+0000","last_clean":"2024-09-05T12:25:34.271242+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:25:34.271242+0000","last_undegraded":"2024-09-05T12:25:34.271242+0000","last_fullsized":"2024-09-05T12:25:34.271242+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265378+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:25:34.265378+0000","last_peered":"2024-09-05T12:25:34.265378+0000","last_clean":"2024-09-05T12:25:34.265378+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:25:34.265378+0000","last_undegraded":"2024-09-05T12:25:34.265378+0000","last_fullsized":"2024-09-05T12:25:34.265378+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323200+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:25:38.323200+0000","last_peered":"2024-09-05T12:25:38.323200+0000","last_clean":"2024-09-05T12:25:38.323200+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:25:38.323200+0000","last_undegraded":"2024-09-05T12:25:38.323200+0000","last_fullsized":"2024-09-05T12:25:38.323200+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330571+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:25:34.330571+0000","last_peered":"2024-09-05T12:25:34.330571+0000","last_clean":"2024-09-05T12:25:34.330571+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:25:34.330571+0000","last_undegraded":"2024-09-05T12:25:34.330571+0000","last_fullsized":"2024-09-05T12:25:34.330571+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265675+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:25:34.265675+0000","last_peered":"2024-09-05T12:25:34.265675+0000","last_clean":"2024-09-05T12:25:34.265675+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:25:34.265675+0000","last_undegraded":"2024-09-05T12:25:34.265675+0000","last_fullsized":"2024-09-05T12:25:34.265675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:35.293537+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:25:35.293537+0000","last_peered":"2024-09-05T12:25:35.293537+0000","last_clean":"2024-09-05T12:25:35.293537+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:25:35.293537+0000","last_undegraded":"2024-09-05T12:25:35.293537+0000","last_fullsized":"2024-09-05T12:25:35.293537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264519+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:25:34.264519+0000","last_peered":"2024-09-05T12:25:34.264519+0000","last_clean":"2024-09-05T12:25:34.264519+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:25:34.264519+0000","last_undegraded":"2024-09-05T12:25:34.264519+0000","last_fullsized":"2024-09-05T12:25:34.264519+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268273+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:25:34.268273+0000","last_peered":"2024-09-05T12:25:34.268273+0000","last_clean":"2024-09-05T12:25:34.268273+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:25:34.268273+0000","last_undegraded":"2024-09-05T12:25:34.268273+0000","last_fullsized":"2024-09-05T12:25:34.268273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323273+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:25:38.323273+0000","last_peered":"2024-09-05T12:25:38.323273+0000","last_clean":"2024-09-05T12:25:38.323273+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:25:38.323273+0000","last_undegraded":"2024-09-05T12:25:38.323273+0000","last_fullsized":"2024-09-05T12:25:38.323273+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263934+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:25:34.263934+0000","last_peered":"2024-09-05T12:25:34.263934+0000","last_clean":"2024-09-05T12:25:34.263934+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:25:34.263934+0000","last_undegraded":"2024-09-05T12:25:34.263934+0000","last_fullsized":"2024-09-05T12:25:34.263934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263632+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:25:34.263632+0000","last_peered":"2024-09-05T12:25:34.263632+0000","last_clean":"2024-09-05T12:25:34.263632+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:25:34.263632+0000","last_undegraded":"2024-09-05T12:25:34.263632+0000","last_fullsized":"2024-09-05T12:25:34.263632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'14","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:54.522676+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:25:54.522676+0000","last_peered":"2024-09-05T12:25:54.522676+0000","last_clean":"2024-09-05T12:25:54.522676+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:25:54.522676+0000","last_undegraded":"2024-09-05T12:25:54.522676+0000","last_fullsized":"2024-09-05T12:25:54.522676+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.262906+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:25:34.262906+0000","last_peered":"2024-09-05T12:25:34.262906+0000","last_clean":"2024-09-05T12:25:34.262906+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:25:34.262906+0000","last_undegraded":"2024-09-05T12:25:34.262906+0000","last_fullsized":"2024-09-05T12:25:34.262906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267595+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:25:34.267595+0000","last_peered":"2024-09-05T12:25:34.267595+0000","last_clean":"2024-09-05T12:25:34.267595+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:25:34.267595+0000","last_undegraded":"2024-09-05T12:25:34.267595+0000","last_fullsized":"2024-09-05T12:25:34.267595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323069+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:25:38.323069+0000","last_peered":"2024-09-05T12:25:38.323069+0000","last_clean":"2024-09-05T12:25:38.323069+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:25:38.323069+0000","last_undegraded":"2024-09-05T12:25:38.323069+0000","last_fullsized":"2024-09-05T12:25:38.323069+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265409+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:25:34.265409+0000","last_peered":"2024-09-05T12:25:34.265409+0000","last_clean":"2024-09-05T12:25:34.265409+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:25:34.265409+0000","last_undegraded":"2024-09-05T12:25:34.265409+0000","last_fullsized":"2024-09-05T12:25:34.265409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267791+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:25:34.267791+0000","last_peered":"2024-09-05T12:25:34.267791+0000","last_clean":"2024-09-05T12:25:34.267791+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:25:34.267791+0000","last_undegraded":"2024-09-05T12:25:34.267791+0000","last_fullsized":"2024-09-05T12:25:34.267791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330392+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:25:34.330392+0000","last_peered":"2024-09-05T12:25:34.330392+0000","last_clean":"2024-09-05T12:25:34.330392+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:25:34.330392+0000","last_undegraded":"2024-09-05T12:25:34.330392+0000","last_fullsized":"2024-09-05T12:25:34.330392+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266261+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:25:34.266261+0000","last_peered":"2024-09-05T12:25:34.266261+0000","last_clean":"2024-09-05T12:25:34.266261+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:25:34.266261+0000","last_undegraded":"2024-09-05T12:25:34.266261+0000","last_fullsized":"2024-09-05T12:25:34.266261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267674+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:25:34.267674+0000","last_peered":"2024-09-05T12:25:34.267674+0000","last_clean":"2024-09-05T12:25:34.267674+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:25:34.267674+0000","last_undegraded":"2024-09-05T12:25:34.267674+0000","last_fullsized":"2024-09-05T12:25:34.267674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'5","reported_seq":108,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:42.428537+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:26:42.428537+0000","last_peered":"2024-09-05T12:26:42.428537+0000","last_clean":"2024-09-05T12:26:42.428537+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:26:42.428537+0000","last_undegraded":"2024-09-05T12:26:42.428537+0000","last_fullsized":"2024-09-05T12:26:42.428537+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267415+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:25:34.267415+0000","last_peered":"2024-09-05T12:25:34.267415+0000","last_clean":"2024-09-05T12:25:34.267415+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:25:34.267415+0000","last_undegraded":"2024-09-05T12:25:34.267415+0000","last_fullsized":"2024-09-05T12:25:34.267415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269430+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:25:34.269430+0000","last_peered":"2024-09-05T12:25:34.269430+0000","last_clean":"2024-09-05T12:25:34.269430+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:25:34.269430+0000","last_undegraded":"2024-09-05T12:25:34.269430+0000","last_fullsized":"2024-09-05T12:25:34.269430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:42.418898+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:26:42.418898+0000","last_peered":"2024-09-05T12:26:42.418898+0000","last_clean":"2024-09-05T12:26:42.418898+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:26:42.418898+0000","last_undegraded":"2024-09-05T12:26:42.418898+0000","last_fullsized":"2024-09-05T12:26:42.418898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271660+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:25:34.271660+0000","last_peered":"2024-09-05T12:25:34.271660+0000","last_clean":"2024-09-05T12:25:34.271660+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:25:34.271660+0000","last_undegraded":"2024-09-05T12:25:34.271660+0000","last_fullsized":"2024-09-05T12:25:34.271660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266417+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:25:34.266417+0000","last_peered":"2024-09-05T12:25:34.266417+0000","last_clean":"2024-09-05T12:25:34.266417+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:25:34.266417+0000","last_undegraded":"2024-09-05T12:25:34.266417+0000","last_fullsized":"2024-09-05T12:25:34.266417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.299617+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:25:36.299617+0000","last_peered":"2024-09-05T12:25:36.299617+0000","last_clean":"2024-09-05T12:25:36.299617+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:25:36.299617+0000","last_undegraded":"2024-09-05T12:25:36.299617+0000","last_fullsized":"2024-09-05T12:25:36.299617+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272425+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:25:34.272425+0000","last_peered":"2024-09-05T12:25:34.272425+0000","last_clean":"2024-09-05T12:25:34.272425+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:25:34.272425+0000","last_undegraded":"2024-09-05T12:25:34.272425+0000","last_fullsized":"2024-09-05T12:25:34.272425+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263271+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:25:34.263271+0000","last_peered":"2024-09-05T12:25:34.263271+0000","last_clean":"2024-09-05T12:25:34.263271+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:25:34.263271+0000","last_undegraded":"2024-09-05T12:25:34.263271+0000","last_fullsized":"2024-09-05T12:25:34.263271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.325619+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:25:39.325619+0000","last_peered":"2024-09-05T12:25:39.325619+0000","last_clean":"2024-09-05T12:25:39.325619+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:25:39.325619+0000","last_undegraded":"2024-09-05T12:25:39.325619+0000","last_fullsized":"2024-09-05T12:25:39.325619+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":362,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269785+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:25:34.269785+0000","last_peered":"2024-09-05T12:25:34.269785+0000","last_clean":"2024-09-05T12:25:34.269785+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:25:34.269785+0000","last_undegraded":"2024-09-05T12:25:34.269785+0000","last_fullsized":"2024-09-05T12:25:34.269785+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268797+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:25:34.268797+0000","last_peered":"2024-09-05T12:25:34.268797+0000","last_clean":"2024-09-05T12:25:34.268797+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:25:34.268797+0000","last_undegraded":"2024-09-05T12:25:34.268797+0000","last_fullsized":"2024-09-05T12:25:34.268797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269019+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:25:34.269019+0000","last_peered":"2024-09-05T12:25:34.269019+0000","last_clean":"2024-09-05T12:25:34.269019+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:25:34.269019+0000","last_undegraded":"2024-09-05T12:25:34.269019+0000","last_fullsized":"2024-09-05T12:25:34.269019+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'6","reported_seq":110,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:41.416545+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:26:41.416545+0000","last_peered":"2024-09-05T12:26:41.416545+0000","last_clean":"2024-09-05T12:26:41.416545+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:26:41.416545+0000","last_undegraded":"2024-09-05T12:26:41.416545+0000","last_fullsized":"2024-09-05T12:26:41.416545+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404323+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:25:34.404323+0000","last_peered":"2024-09-05T12:25:34.404323+0000","last_clean":"2024-09-05T12:25:34.404323+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:25:34.404323+0000","last_undegraded":"2024-09-05T12:25:34.404323+0000","last_fullsized":"2024-09-05T12:25:34.404323+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269554+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:25:34.269554+0000","last_peered":"2024-09-05T12:25:34.269554+0000","last_clean":"2024-09-05T12:25:34.269554+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:25:34.269554+0000","last_undegraded":"2024-09-05T12:25:34.269554+0000","last_fullsized":"2024-09-05T12:25:34.269554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321412+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:25:38.321412+0000","last_peered":"2024-09-05T12:25:38.321412+0000","last_clean":"2024-09-05T12:25:38.321412+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:25:38.321412+0000","last_undegraded":"2024-09-05T12:25:38.321412+0000","last_fullsized":"2024-09-05T12:25:38.321412+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267091+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:25:34.267091+0000","last_peered":"2024-09-05T12:25:34.267091+0000","last_clean":"2024-09-05T12:25:34.267091+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:25:34.267091+0000","last_undegraded":"2024-09-05T12:25:34.267091+0000","last_fullsized":"2024-09-05T12:25:34.267091+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268658+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:25:34.268658+0000","last_peered":"2024-09-05T12:25:34.268658+0000","last_clean":"2024-09-05T12:25:34.268658+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:25:34.268658+0000","last_undegraded":"2024-09-05T12:25:34.268658+0000","last_fullsized":"2024-09-05T12:25:34.268658+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.323388+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:25:38.323388+0000","last_peered":"2024-09-05T12:25:38.323388+0000","last_clean":"2024-09-05T12:25:38.323388+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:25:38.323388+0000","last_undegraded":"2024-09-05T12:25:38.323388+0000","last_fullsized":"2024-09-05T12:25:38.323388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269057+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:25:34.269057+0000","last_peered":"2024-09-05T12:25:34.269057+0000","last_clean":"2024-09-05T12:25:34.269057+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:25:34.269057+0000","last_undegraded":"2024-09-05T12:25:34.269057+0000","last_fullsized":"2024-09-05T12:25:34.269057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266821+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:25:34.266821+0000","last_peered":"2024-09-05T12:25:34.266821+0000","last_clean":"2024-09-05T12:25:34.266821+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:25:34.266821+0000","last_undegraded":"2024-09-05T12:25:34.266821+0000","last_fullsized":"2024-09-05T12:25:34.266821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.296703+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:25:38.296703+0000","last_peered":"2024-09-05T12:25:38.296703+0000","last_clean":"2024-09-05T12:25:38.296703+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:25:38.296703+0000","last_undegraded":"2024-09-05T12:25:38.296703+0000","last_fullsized":"2024-09-05T12:25:38.296703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270206+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:25:34.270206+0000","last_peered":"2024-09-05T12:25:34.270206+0000","last_clean":"2024-09-05T12:25:34.270206+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:25:34.270206+0000","last_undegraded":"2024-09-05T12:25:34.270206+0000","last_fullsized":"2024-09-05T12:25:34.270206+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270383+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:25:34.270383+0000","last_peered":"2024-09-05T12:25:34.270383+0000","last_clean":"2024-09-05T12:25:34.270383+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:25:34.270383+0000","last_undegraded":"2024-09-05T12:25:34.270383+0000","last_fullsized":"2024-09-05T12:25:34.270383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":56,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321005+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:25:38.321005+0000","last_peered":"2024-09-05T12:25:38.321005+0000","last_clean":"2024-09-05T12:25:38.321005+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:25:38.321005+0000","last_undegraded":"2024-09-05T12:25:38.321005+0000","last_fullsized":"2024-09-05T12:25:38.321005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267124+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:25:34.267124+0000","last_peered":"2024-09-05T12:25:34.267124+0000","last_clean":"2024-09-05T12:25:34.267124+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:25:34.267124+0000","last_undegraded":"2024-09-05T12:25:34.267124+0000","last_fullsized":"2024-09-05T12:25:34.267124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331448+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:25:34.331448+0000","last_peered":"2024-09-05T12:25:34.331448+0000","last_clean":"2024-09-05T12:25:34.331448+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:25:34.331448+0000","last_undegraded":"2024-09-05T12:25:34.331448+0000","last_fullsized":"2024-09-05T12:25:34.331448+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:42.419388+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:26:42.419388+0000","last_peered":"2024-09-05T12:26:42.419388+0000","last_clean":"2024-09-05T12:26:42.419388+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:26:42.419388+0000","last_undegraded":"2024-09-05T12:26:42.419388+0000","last_fullsized":"2024-09-05T12:26:42.419388+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330666+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:25:34.330666+0000","last_peered":"2024-09-05T12:25:34.330666+0000","last_clean":"2024-09-05T12:25:34.330666+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:25:34.330666+0000","last_undegraded":"2024-09-05T12:25:34.330666+0000","last_fullsized":"2024-09-05T12:25:34.330666+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268645+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:25:34.268645+0000","last_peered":"2024-09-05T12:25:34.268645+0000","last_clean":"2024-09-05T12:25:34.268645+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:25:34.268645+0000","last_undegraded":"2024-09-05T12:25:34.268645+0000","last_fullsized":"2024-09-05T12:25:34.268645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322440+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:25:38.322440+0000","last_peered":"2024-09-05T12:25:38.322440+0000","last_clean":"2024-09-05T12:25:38.322440+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:25:38.322440+0000","last_undegraded":"2024-09-05T12:25:38.322440+0000","last_fullsized":"2024-09-05T12:25:38.322440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.271674+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:25:34.271674+0000","last_peered":"2024-09-05T12:25:34.271674+0000","last_clean":"2024-09-05T12:25:34.271674+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:25:34.271674+0000","last_undegraded":"2024-09-05T12:25:34.271674+0000","last_fullsized":"2024-09-05T12:25:34.271674+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266874+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:25:34.266874+0000","last_peered":"2024-09-05T12:25:34.266874+0000","last_clean":"2024-09-05T12:25:34.266874+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:25:34.266874+0000","last_undegraded":"2024-09-05T12:25:34.266874+0000","last_fullsized":"2024-09-05T12:25:34.266874+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:39.326756+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:25:39.326756+0000","last_peered":"2024-09-05T12:25:39.326756+0000","last_clean":"2024-09-05T12:25:39.326756+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:25:39.326756+0000","last_undegraded":"2024-09-05T12:25:39.326756+0000","last_fullsized":"2024-09-05T12:25:39.326756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269780+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:25:34.269780+0000","last_peered":"2024-09-05T12:25:34.269780+0000","last_clean":"2024-09-05T12:25:34.269780+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:25:34.269780+0000","last_undegraded":"2024-09-05T12:25:34.269780+0000","last_fullsized":"2024-09-05T12:25:34.269780+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266924+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:25:34.266924+0000","last_peered":"2024-09-05T12:25:34.266924+0000","last_clean":"2024-09-05T12:25:34.266924+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:25:34.266924+0000","last_undegraded":"2024-09-05T12:25:34.266924+0000","last_fullsized":"2024-09-05T12:25:34.266924+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":211,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:43.418923+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:26:43.418923+0000","last_peered":"2024-09-05T12:26:43.418923+0000","last_clean":"2024-09-05T12:26:43.418923+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:26:43.418923+0000","last_undegraded":"2024-09-05T12:26:43.418923+0000","last_fullsized":"2024-09-05T12:26:43.418923+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62931,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":139,"num_read_kb":131,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330067+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:25:34.330067+0000","last_peered":"2024-09-05T12:25:34.330067+0000","last_clean":"2024-09-05T12:25:34.330067+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:25:34.330067+0000","last_undegraded":"2024-09-05T12:25:34.330067+0000","last_fullsized":"2024-09-05T12:25:34.330067+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404529+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:25:34.404529+0000","last_peered":"2024-09-05T12:25:34.404529+0000","last_clean":"2024-09-05T12:25:34.404529+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:25:34.404529+0000","last_undegraded":"2024-09-05T12:25:34.404529+0000","last_fullsized":"2024-09-05T12:25:34.404529+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313398+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:25:37.313398+0000","last_peered":"2024-09-05T12:25:37.313398+0000","last_clean":"2024-09-05T12:25:37.313398+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:25:37.313398+0000","last_undegraded":"2024-09-05T12:25:37.313398+0000","last_fullsized":"2024-09-05T12:25:37.313398+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264502+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:25:34.264502+0000","last_peered":"2024-09-05T12:25:34.264502+0000","last_clean":"2024-09-05T12:25:34.264502+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:25:34.264502+0000","last_undegraded":"2024-09-05T12:25:34.264502+0000","last_fullsized":"2024-09-05T12:25:34.264502+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268248+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:25:34.268248+0000","last_peered":"2024-09-05T12:25:34.268248+0000","last_clean":"2024-09-05T12:25:34.268248+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:25:34.268248+0000","last_undegraded":"2024-09-05T12:25:34.268248+0000","last_fullsized":"2024-09-05T12:25:34.268248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.321510+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:25:38.321510+0000","last_peered":"2024-09-05T12:25:38.321510+0000","last_clean":"2024-09-05T12:25:38.321510+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:25:38.321510+0000","last_undegraded":"2024-09-05T12:25:38.321510+0000","last_fullsized":"2024-09-05T12:25:38.321510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.272597+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:25:34.272597+0000","last_peered":"2024-09-05T12:25:34.272597+0000","last_clean":"2024-09-05T12:25:34.272597+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:25:34.272597+0000","last_undegraded":"2024-09-05T12:25:34.272597+0000","last_fullsized":"2024-09-05T12:25:34.272597+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404720+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:25:34.404720+0000","last_peered":"2024-09-05T12:25:34.404720+0000","last_clean":"2024-09-05T12:25:34.404720+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:25:34.404720+0000","last_undegraded":"2024-09-05T12:25:34.404720+0000","last_fullsized":"2024-09-05T12:25:34.404720+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":171,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:43.418937+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:26:43.418937+0000","last_peered":"2024-09-05T12:26:43.418937+0000","last_clean":"2024-09-05T12:26:43.418937+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:26:43.418937+0000","last_undegraded":"2024-09-05T12:26:43.418937+0000","last_fullsized":"2024-09-05T12:26:43.418937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263017+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:25:34.263017+0000","last_peered":"2024-09-05T12:25:34.263017+0000","last_clean":"2024-09-05T12:25:34.263017+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:25:34.263017+0000","last_undegraded":"2024-09-05T12:25:34.263017+0000","last_fullsized":"2024-09-05T12:25:34.263017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268073+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:25:34.268073+0000","last_peered":"2024-09-05T12:25:34.268073+0000","last_clean":"2024-09-05T12:25:34.268073+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:25:34.268073+0000","last_undegraded":"2024-09-05T12:25:34.268073+0000","last_fullsized":"2024-09-05T12:25:34.268073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.329887+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:25:34.329887+0000","last_peered":"2024-09-05T12:25:34.329887+0000","last_clean":"2024-09-05T12:25:34.329887+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:25:34.329887+0000","last_undegraded":"2024-09-05T12:25:34.329887+0000","last_fullsized":"2024-09-05T12:25:34.329887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:38.322579+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:25:38.322579+0000","last_peered":"2024-09-05T12:25:38.322579+0000","last_clean":"2024-09-05T12:25:38.322579+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:25:38.322579+0000","last_undegraded":"2024-09-05T12:25:38.322579+0000","last_fullsized":"2024-09-05T12:25:38.322579+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267082+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:25:34.267082+0000","last_peered":"2024-09-05T12:25:34.267082+0000","last_clean":"2024-09-05T12:25:34.267082+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:25:34.267082+0000","last_undegraded":"2024-09-05T12:25:34.267082+0000","last_fullsized":"2024-09-05T12:25:34.267082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263148+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:25:34.263148+0000","last_peered":"2024-09-05T12:25:34.263148+0000","last_clean":"2024-09-05T12:25:34.263148+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:25:34.263148+0000","last_undegraded":"2024-09-05T12:25:34.263148+0000","last_fullsized":"2024-09-05T12:25:34.263148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:36.302852+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:25:36.302852+0000","last_peered":"2024-09-05T12:25:36.302852+0000","last_clean":"2024-09-05T12:25:36.302852+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:25:36.302852+0000","last_undegraded":"2024-09-05T12:25:36.302852+0000","last_fullsized":"2024-09-05T12:25:36.302852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:37.313373+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:25:37.313373+0000","last_peered":"2024-09-05T12:25:37.313373+0000","last_clean":"2024-09-05T12:25:37.313373+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:25:37.313373+0000","last_undegraded":"2024-09-05T12:25:37.313373+0000","last_fullsized":"2024-09-05T12:25:37.313373+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269899+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:25:34.269899+0000","last_peered":"2024-09-05T12:25:34.269899+0000","last_clean":"2024-09-05T12:25:34.269899+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:25:34.269899+0000","last_undegraded":"2024-09-05T12:25:34.269899+0000","last_fullsized":"2024-09-05T12:25:34.269899+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264384+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:25:34.264384+0000","last_peered":"2024-09-05T12:25:34.264384+0000","last_clean":"2024-09-05T12:25:34.264384+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:25:34.264384+0000","last_undegraded":"2024-09-05T12:25:34.264384+0000","last_fullsized":"2024-09-05T12:25:34.264384+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'5","reported_seq":108,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:26:40.414900+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:26:40.414900+0000","last_peered":"2024-09-05T12:26:40.414900+0000","last_clean":"2024-09-05T12:26:40.414900+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:26:40.414900+0000","last_undegraded":"2024-09-05T12:26:40.414900+0000","last_fullsized":"2024-09-05T12:26:40.414900+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":65,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266460+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:25:34.266460+0000","last_peered":"2024-09-05T12:25:34.266460+0000","last_clean":"2024-09-05T12:25:34.266460+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:25:34.266460+0000","last_undegraded":"2024-09-05T12:25:34.266460+0000","last_fullsized":"2024-09-05T12:25:34.266460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264349+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:25:34.264349+0000","last_peered":"2024-09-05T12:25:34.264349+0000","last_clean":"2024-09-05T12:25:34.264349+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:25:34.264349+0000","last_undegraded":"2024-09-05T12:25:34.264349+0000","last_fullsized":"2024-09-05T12:25:34.264349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265242+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:25:34.265242+0000","last_peered":"2024-09-05T12:25:34.265242+0000","last_clean":"2024-09-05T12:25:34.265242+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:25:34.265242+0000","last_undegraded":"2024-09-05T12:25:34.265242+0000","last_fullsized":"2024-09-05T12:25:34.265242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264119+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:25:34.264119+0000","last_peered":"2024-09-05T12:25:34.264119+0000","last_clean":"2024-09-05T12:25:34.264119+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:25:34.264119+0000","last_undegraded":"2024-09-05T12:25:34.264119+0000","last_fullsized":"2024-09-05T12:25:34.264119+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263751+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:25:34.263751+0000","last_peered":"2024-09-05T12:25:34.263751+0000","last_clean":"2024-09-05T12:25:34.263751+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:25:34.263751+0000","last_undegraded":"2024-09-05T12:25:34.263751+0000","last_fullsized":"2024-09-05T12:25:34.263751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.270400+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:25:34.270400+0000","last_peered":"2024-09-05T12:25:34.270400+0000","last_clean":"2024-09-05T12:25:34.270400+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:25:34.270400+0000","last_undegraded":"2024-09-05T12:25:34.270400+0000","last_fullsized":"2024-09-05T12:25:34.270400+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331799+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:25:34.331799+0000","last_peered":"2024-09-05T12:25:34.331799+0000","last_clean":"2024-09-05T12:25:34.331799+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:25:34.331799+0000","last_undegraded":"2024-09-05T12:25:34.331799+0000","last_fullsized":"2024-09-05T12:25:34.331799+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266950+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:25:34.266950+0000","last_peered":"2024-09-05T12:25:34.266950+0000","last_clean":"2024-09-05T12:25:34.266950+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:25:34.266950+0000","last_undegraded":"2024-09-05T12:25:34.266950+0000","last_fullsized":"2024-09-05T12:25:34.266950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264875+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:25:34.264875+0000","last_peered":"2024-09-05T12:25:34.264875+0000","last_clean":"2024-09-05T12:25:34.264875+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:25:34.264875+0000","last_undegraded":"2024-09-05T12:25:34.264875+0000","last_fullsized":"2024-09-05T12:25:34.264875+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263394+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:25:34.263394+0000","last_peered":"2024-09-05T12:25:34.263394+0000","last_clean":"2024-09-05T12:25:34.263394+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:25:34.263394+0000","last_undegraded":"2024-09-05T12:25:34.263394+0000","last_fullsized":"2024-09-05T12:25:34.263394+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.269818+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:25:34.269818+0000","last_peered":"2024-09-05T12:25:34.269818+0000","last_clean":"2024-09-05T12:25:34.269818+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:25:34.269818+0000","last_undegraded":"2024-09-05T12:25:34.269818+0000","last_fullsized":"2024-09-05T12:25:34.269818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263113+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:25:34.263113+0000","last_peered":"2024-09-05T12:25:34.263113+0000","last_clean":"2024-09-05T12:25:34.263113+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:25:34.263113+0000","last_undegraded":"2024-09-05T12:25:34.263113+0000","last_fullsized":"2024-09-05T12:25:34.263113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404441+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:25:34.404441+0000","last_peered":"2024-09-05T12:25:34.404441+0000","last_clean":"2024-09-05T12:25:34.404441+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:25:34.404441+0000","last_undegraded":"2024-09-05T12:25:34.404441+0000","last_fullsized":"2024-09-05T12:25:34.404441+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267531+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:25:34.267531+0000","last_peered":"2024-09-05T12:25:34.267531+0000","last_clean":"2024-09-05T12:25:34.267531+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:25:34.267531+0000","last_undegraded":"2024-09-05T12:25:34.267531+0000","last_fullsized":"2024-09-05T12:25:34.267531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266222+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:25:34.266222+0000","last_peered":"2024-09-05T12:25:34.266222+0000","last_clean":"2024-09-05T12:25:34.266222+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:25:34.266222+0000","last_undegraded":"2024-09-05T12:25:34.266222+0000","last_fullsized":"2024-09-05T12:25:34.266222+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.264463+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:25:34.264463+0000","last_peered":"2024-09-05T12:25:34.264463+0000","last_clean":"2024-09-05T12:25:34.264463+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:25:34.264463+0000","last_undegraded":"2024-09-05T12:25:34.264463+0000","last_fullsized":"2024-09-05T12:25:34.264463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.330539+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:25:34.330539+0000","last_peered":"2024-09-05T12:25:34.330539+0000","last_clean":"2024-09-05T12:25:34.330539+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:25:34.330539+0000","last_undegraded":"2024-09-05T12:25:34.330539+0000","last_fullsized":"2024-09-05T12:25:34.330539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267312+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:25:34.267312+0000","last_peered":"2024-09-05T12:25:34.267312+0000","last_clean":"2024-09-05T12:25:34.267312+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:25:34.267312+0000","last_undegraded":"2024-09-05T12:25:34.267312+0000","last_fullsized":"2024-09-05T12:25:34.267312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404202+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:25:34.404202+0000","last_peered":"2024-09-05T12:25:34.404202+0000","last_clean":"2024-09-05T12:25:34.404202+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:25:34.404202+0000","last_undegraded":"2024-09-05T12:25:34.404202+0000","last_fullsized":"2024-09-05T12:25:34.404202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266403+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:25:34.266403+0000","last_peered":"2024-09-05T12:25:34.266403+0000","last_clean":"2024-09-05T12:25:34.266403+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:25:34.266403+0000","last_undegraded":"2024-09-05T12:25:34.266403+0000","last_fullsized":"2024-09-05T12:25:34.266403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.266842+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:25:34.266842+0000","last_peered":"2024-09-05T12:25:34.266842+0000","last_clean":"2024-09-05T12:25:34.266842+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:25:34.266842+0000","last_undegraded":"2024-09-05T12:25:34.266842+0000","last_fullsized":"2024-09-05T12:25:34.266842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267435+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:25:34.267435+0000","last_peered":"2024-09-05T12:25:34.267435+0000","last_clean":"2024-09-05T12:25:34.267435+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:25:34.267435+0000","last_undegraded":"2024-09-05T12:25:34.267435+0000","last_fullsized":"2024-09-05T12:25:34.267435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265181+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:25:34.265181+0000","last_peered":"2024-09-05T12:25:34.265181+0000","last_clean":"2024-09-05T12:25:34.265181+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:25:34.265181+0000","last_undegraded":"2024-09-05T12:25:34.265181+0000","last_fullsized":"2024-09-05T12:25:34.265181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268910+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:25:34.268910+0000","last_peered":"2024-09-05T12:25:34.268910+0000","last_clean":"2024-09-05T12:25:34.268910+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:25:34.268910+0000","last_undegraded":"2024-09-05T12:25:34.268910+0000","last_fullsized":"2024-09-05T12:25:34.268910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.331174+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:25:34.331174+0000","last_peered":"2024-09-05T12:25:34.331174+0000","last_clean":"2024-09-05T12:25:34.331174+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:25:34.331174+0000","last_undegraded":"2024-09-05T12:25:34.331174+0000","last_fullsized":"2024-09-05T12:25:34.331174+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.404780+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:25:34.404780+0000","last_peered":"2024-09-05T12:25:34.404780+0000","last_clean":"2024-09-05T12:25:34.404780+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:25:34.404780+0000","last_undegraded":"2024-09-05T12:25:34.404780+0000","last_fullsized":"2024-09-05T12:25:34.404780+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.263044+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:25:34.263044+0000","last_peered":"2024-09-05T12:25:34.263044+0000","last_clean":"2024-09-05T12:25:34.263044+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:25:34.263044+0000","last_undegraded":"2024-09-05T12:25:34.263044+0000","last_fullsized":"2024-09-05T12:25:34.263044+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268375+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:25:34.268375+0000","last_peered":"2024-09-05T12:25:34.268375+0000","last_clean":"2024-09-05T12:25:34.268375+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:25:34.268375+0000","last_undegraded":"2024-09-05T12:25:34.268375+0000","last_fullsized":"2024-09-05T12:25:34.268375+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267499+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:25:34.267499+0000","last_peered":"2024-09-05T12:25:34.267499+0000","last_clean":"2024-09-05T12:25:34.267499+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:25:34.267499+0000","last_undegraded":"2024-09-05T12:25:34.267499+0000","last_fullsized":"2024-09-05T12:25:34.267499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265767+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:25:34.265767+0000","last_peered":"2024-09-05T12:25:34.265767+0000","last_clean":"2024-09-05T12:25:34.265767+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:25:34.265767+0000","last_undegraded":"2024-09-05T12:25:34.265767+0000","last_fullsized":"2024-09-05T12:25:34.265767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.265061+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:25:34.265061+0000","last_peered":"2024-09-05T12:25:34.265061+0000","last_clean":"2024-09-05T12:25:34.265061+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:25:34.265061+0000","last_undegraded":"2024-09-05T12:25:34.265061+0000","last_fullsized":"2024-09-05T12:25:34.265061+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267876+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:25:34.267876+0000","last_peered":"2024-09-05T12:25:34.267876+0000","last_clean":"2024-09-05T12:25:34.267876+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:25:34.267876+0000","last_undegraded":"2024-09-05T12:25:34.267876+0000","last_fullsized":"2024-09-05T12:25:34.267876+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.268171+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:25:34.268171+0000","last_peered":"2024-09-05T12:25:34.268171+0000","last_clean":"2024-09-05T12:25:34.268171+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:25:34.268171+0000","last_undegraded":"2024-09-05T12:25:34.268171+0000","last_fullsized":"2024-09-05T12:25:34.268171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-05T12:25:34.267268+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:25:34.267268+0000","last_peered":"2024-09-05T12:25:34.267268+0000","last_clean":"2024-09-05T12:25:34.267268+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:25:34.267268+0000","last_undegraded":"2024-09-05T12:25:34.267268+0000","last_fullsized":"2024-09-05T12:25:34.267268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69746,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":687,"num_read_kb":422,"num_write":247,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":209238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":212,"ondisk_log_size":212,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449126,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":533938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678058,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1781760,"data_stored":1187366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841583,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":533453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103221,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1097728,"data_stored":533809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364858,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1699840,"data_stored":1129937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":528957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28692,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716460,"statfs":{"total":95995035648,"available":95965655040,"internally_reserved":0,"allocated":1708032,"data_stored":1125519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1089536,"data_stored":533031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1093632,"data_stored":528665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1171456,"data_stored":593531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1150976,"data_stored":592918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1105920,"data_stored":530806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:26:46.825 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T12:26:46.825 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-05T12:26:46.825 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-05T12:26:46.825 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph health --format=json 2024-09-05T12:26:47.191 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:47.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:47 smithi071 ceph-mon[29845]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:47.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:47 smithi123 ceph-mon[34321]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:47.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:47 smithi174 ceph-mon[34313]: from='client.24923 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 -- 172.21.15.71:0/1431479064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 msgr2=0x7fa490114c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1431479064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 0x7fa490114c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa488009ca0 tx=0x7fa48802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 -- 172.21.15.71:0/1431479064 shutdown_connections 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1431479064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 0x7fa490114c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1431479064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa490077040 0x7fa490112350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1431479064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa490076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 -- 172.21.15.71:0/1431479064 >> 172.21.15.71:0/1431479064 conn(0x7fa4900ff8b0 msgr2=0x7fa490101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 -- 172.21.15.71:0/1431479064 shutdown_connections 2024-09-05T12:26:47.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.806+0000 7fa48fe00640 1 -- 172.21.15.71:0/1431479064 wait complete. 2024-09-05T12:26:47.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.807+0000 7fa48fe00640 1 Processor -- start 2024-09-05T12:26:47.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.807+0000 7fa48fe00640 1 -- start start 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa4901a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa490077040 0x7fa4901a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 0x7fa49019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa490118540 con 0x7fa490076720 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901183c0 con 0x7fa490077040 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4901186c0 con 0x7fa490112890 2024-09-05T12:26:47.811 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa4901a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa4901a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50548/0 (socket says 172.21.15.71:50548) 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48ea00640 1 -- 172.21.15.71:0/1534389229 learned_addr learned my addr 172.21.15.71:0/1534389229 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48f400640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 0x7fa49019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.808+0000 7fa48e000640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa490077040 0x7fa4901a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48ea00640 1 -- 172.21.15.71:0/1534389229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 msgr2=0x7fa49019e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48ea00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 0x7fa49019e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48ea00640 1 -- 172.21.15.71:0/1534389229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa490077040 msgr2=0x7fa4901a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48ea00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa490077040 0x7fa4901a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48ea00640 1 -- 172.21.15.71:0/1534389229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49019ef30 con 0x7fa490076720 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48ea00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa4901a4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa48000d950 tx=0x7fa48000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa480017070 con 0x7fa490076720 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa480009c30 con 0x7fa490076720 2024-09-05T12:26:47.812 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49019f220 con 0x7fa490076720 2024-09-05T12:26:47.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48000f6b0 con 0x7fa490076720 2024-09-05T12:26:47.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.809+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4901ab470 con 0x7fa490076720 2024-09-05T12:26:47.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.810+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa490077e60 con 0x7fa490076720 2024-09-05T12:26:47.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.812+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa4800109d0 con 0x7fa490076720 2024-09-05T12:26:47.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.813+0000 7fa487400640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa4740a2c10 0x7fa4740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:47.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.813+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa4800cf480 con 0x7fa490076720 2024-09-05T12:26:47.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.814+0000 7fa48e000640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa4740a2c10 0x7fa4740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:47.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.814+0000 7fa48e000640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa4740a2c10 0x7fa4740a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0095f0 tx=0x7fa47c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:47.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:47.818+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa480009de0 con 0x7fa490076720 2024-09-05T12:26:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.026+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa490076b00 con 0x7fa490076720 2024-09-05T12:26:48.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.026+0000 7fa487400640 1 -- 172.21.15.71:0/1534389229 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa48009b870 con 0x7fa490076720 2024-09-05T12:26:48.029 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:48.029 INFO:teuthology.orchestra.run.smithi071.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-05T12:26:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa4740a2c10 msgr2=0x7fa4740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa4740a2c10 0x7fa4740a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0095f0 tx=0x7fa47c005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 msgr2=0x7fa4901a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:48.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa4901a4610 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa48000d950 tx=0x7fa48000de10 comp rx=0 tx=0).stop 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 shutdown_connections 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa4740a2c10 0x7fa4740a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa490112890 0x7fa49019e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa490077040 0x7fa4901a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 --2- 172.21.15.71:0/1534389229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa490076720 0x7fa4901a4610 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.029+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 >> 172.21.15.71:0/1534389229 conn(0x7fa4900ff8b0 msgr2=0x7fa490101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.030+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 shutdown_connections 2024-09-05T12:26:48.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:48.030+0000 7fa48fe00640 1 -- 172.21.15.71:0/1534389229 wait complete. 2024-09-05T12:26:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:48 smithi071 ceph-mon[29845]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:48 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1534389229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T12:26:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:48 smithi123 ceph-mon[34321]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:48 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1534389229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T12:26:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:48 smithi174 ceph-mon[34313]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:48 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1534389229' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-05T12:26:48.822 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-05T12:26:48.840 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:26:49.251 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:49.582 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:49 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:26:49.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:49 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:26:49.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:49 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:26:49.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 -- 172.21.15.71:0/3959153343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c103ff0 msgr2=0x7f617c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:49.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 --2- 172.21.15.71:0/3959153343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c103ff0 0x7f617c1043d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f616c009900 tx=0x7f616c02a410 comp rx=0 tx=0).stop 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 -- 172.21.15.71:0/3959153343 shutdown_connections 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 --2- 172.21.15.71:0/3959153343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f617c109120 0x7f617c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 --2- 172.21.15.71:0/3959153343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c1049a0 0x7f617c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 --2- 172.21.15.71:0/3959153343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c103ff0 0x7f617c1043d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 -- 172.21.15.71:0/3959153343 >> 172.21.15.71:0/3959153343 conn(0x7f617c0ffa90 msgr2=0x7f617c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 -- 172.21.15.71:0/3959153343 shutdown_connections 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.943+0000 7f6182a00640 1 -- 172.21.15.71:0/3959153343 wait complete. 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.944+0000 7f6182a00640 1 Processor -- start 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.944+0000 7f6182a00640 1 -- start start 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6182a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 0x7f617c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6182a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c1049a0 0x7f617c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:49.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6182a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f617c109120 0x7f617c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6182a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f617c114190 con 0x7f617c1049a0 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6182a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f617c114010 con 0x7f617c109120 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6182a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f617c114310 con 0x7f617c103ff0 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c1049a0 0x7f617c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 0x7f617c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 0x7f617c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46452/0 (socket says 172.21.15.71:46452) 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 -- 172.21.15.71:0/2817960951 learned_addr learned my addr 172.21.15.71:0/2817960951 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f6180a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f617c109120 0x7f617c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 -- 172.21.15.71:0/2817960951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f617c109120 msgr2=0x7f617c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f617c109120 0x7f617c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 -- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c1049a0 msgr2=0x7f617c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:49.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c1049a0 0x7f617c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.945+0000 7f617be00640 1 -- 172.21.15.71:0/2817960951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617c19f5c0 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.946+0000 7f617be00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 0x7f617c19a610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f616c004770 tx=0x7f616c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.946+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c03f070 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.946+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617c19f850 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.946+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f616c031ca0 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.946+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c035030 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.946+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617c1a7190 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.947+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f617c1054d0 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.949+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f616c042070 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.951+0000 7f6178c00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f61600a2c10 0x7f61600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.951+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f616c0efef0 con 0x7f617c103ff0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.951+0000 7f617b400640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f61600a2c10 0x7f61600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:49.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.952+0000 7f617b400640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f61600a2c10 0x7f61600a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f617c0fff20 tx=0x7f6168005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:49.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:49.952+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f616c0bc3a0 con 0x7f617c103ff0 2024-09-05T12:26:50.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.186+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f617c10c300 con 0x7f617c103ff0 2024-09-05T12:26:50.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.187+0000 7f6178c00640 1 -- 172.21.15.71:0/2817960951 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11968 (secure 0 0 0) 0x7f616c0c06a0 con 0x7f617c103ff0 2024-09-05T12:26:50.189 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:50.189 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"btime":"2024-09-05T12:25:40:321146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:40.307362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:26:50.190 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:26:50.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f61600a2c10 msgr2=0x7f61600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:50.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f61600a2c10 0x7f61600a50d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f617c0fff20 tx=0x7f6168005f90 comp rx=0 tx=0).stop 2024-09-05T12:26:50.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 msgr2=0x7f617c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:50.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 0x7f617c19a610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f616c004770 tx=0x7f616c003c10 comp rx=0 tx=0).stop 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 shutdown_connections 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f61600a2c10 0x7f61600a50d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f617c109120 0x7f617c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f617c1049a0 0x7f617c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 --2- 172.21.15.71:0/2817960951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f617c103ff0 0x7f617c19a610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.189+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 >> 172.21.15.71:0/2817960951 conn(0x7f617c0ffa90 msgr2=0x7f617c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.190+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 shutdown_connections 2024-09-05T12:26:50.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:50.190+0000 7f6182a00640 1 -- 172.21.15.71:0/2817960951 wait complete. 2024-09-05T12:26:50.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:50 smithi071 ceph-mon[29845]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:50.535 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:50 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2817960951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:50 smithi123 ceph-mon[34321]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:50 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2817960951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:50 smithi174 ceph-mon[34313]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:26:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:50 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2817960951' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:51.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.781 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:51 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.782 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:51 smithi071 ceph-mon[29845]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:51.820 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:26:51.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:51 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:51.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:51 smithi123 ceph-mon[34321]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:51 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:52.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:51 smithi174 ceph-mon[34313]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:26:52.235 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.911+0000 7efcaec00640 1 -- 172.21.15.71:0/458825626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 msgr2=0x7efca8112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.911+0000 7efcaec00640 1 --2- 172.21.15.71:0/458825626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 0x7efca8112510 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc94009900 tx=0x7efc9402a410 comp rx=0 tx=0).stop 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 -- 172.21.15.71:0/458825626 shutdown_connections 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 --2- 172.21.15.71:0/458825626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8112a50 0x7efca8114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 --2- 172.21.15.71:0/458825626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 0x7efca8112510 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 --2- 172.21.15.71:0/458825626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca8076720 0x7efca8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 -- 172.21.15.71:0/458825626 >> 172.21.15.71:0/458825626 conn(0x7efca80ffa50 msgr2=0x7efca8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:52.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 -- 172.21.15.71:0/458825626 shutdown_connections 2024-09-05T12:26:52.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 -- 172.21.15.71:0/458825626 wait complete. 2024-09-05T12:26:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.912+0000 7efcaec00640 1 Processor -- start 2024-09-05T12:26:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 -- start start 2024-09-05T12:26:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 0x7efca81a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:52.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 0x7efca81a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca8112a50 0x7efca819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efca81185d0 con 0x7efca8077040 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efca8118450 con 0x7efca8112a50 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcaec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efca8118750 con 0x7efca8076720 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efca7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 0x7efca81a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efca7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 0x7efca81a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efca7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 0x7efca81a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36988/0 (socket says 172.21.15.71:36988) 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efca7e00640 1 -- 172.21.15.71:0/926501510 learned_addr learned my addr 172.21.15.71:0/926501510 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.913+0000 7efcacc00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca8112a50 0x7efca819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca7e00640 1 -- 172.21.15.71:0/926501510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca8112a50 msgr2=0x7efca819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:52.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca7e00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca8112a50 0x7efca819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca7e00640 1 -- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 msgr2=0x7efca81a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca7e00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 0x7efca81a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca7e00640 1 -- 172.21.15.71:0/926501510 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca819f070 con 0x7efca8076720 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca7e00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 0x7efca81a4780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efc9800ea60 tx=0x7efc9800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9800cde0 con 0x7efca8076720 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca819f360 con 0x7efca8076720 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc980035f0 con 0x7efca8076720 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98010670 con 0x7efca8076720 2024-09-05T12:26:52.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.914+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca81ab650 con 0x7efca8076720 2024-09-05T12:26:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.915+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca8076b00 con 0x7efca8076720 2024-09-05T12:26:52.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.918+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efc980037a0 con 0x7efca8076720 2024-09-05T12:26:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.919+0000 7efca4c00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.919+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7efc980cf900 con 0x7efca8076720 2024-09-05T12:26:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.919+0000 7efca7400640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:52.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.920+0000 7efca7400640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc94009230 tx=0x7efc940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:52.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:52.921+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc9809c3d0 con 0x7efca8076720 2024-09-05T12:26:53.146 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.145+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efca819fe70 con 0x7efca8076720 2024-09-05T12:26:53.147 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.146+0000 7efca4c00640 1 -- 172.21.15.71:0/926501510 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11968 (secure 0 0 0) 0x7efc9809bcd0 con 0x7efca8076720 2024-09-05T12:26:53.148 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:53.149 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"btime":"2024-09-05T12:25:40:321146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:40.307362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:26:53.149 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:26:53.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.148+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7efc8c0a2c10 msgr2=0x7efc8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:53.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.148+0000 7efcaec00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7efc94009230 tx=0x7efc940058d0 comp rx=0 tx=0).stop 2024-09-05T12:26:53.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.148+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 msgr2=0x7efca81a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:53.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.148+0000 7efcaec00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 0x7efca81a4780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efc9800ea60 tx=0x7efc9800ef20 comp rx=0 tx=0).stop 2024-09-05T12:26:53.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 shutdown_connections 2024-09-05T12:26:53.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7efc8c0a2c10 0x7efc8c0a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:53.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca8112a50 0x7efca819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:53.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efca8077040 0x7efca81a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:53.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 --2- 172.21.15.71:0/926501510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efca8076720 0x7efca81a4780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:53.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 >> 172.21.15.71:0/926501510 conn(0x7efca80ffa50 msgr2=0x7efca8101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:53.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 shutdown_connections 2024-09-05T12:26:53.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:53.149+0000 7efcaec00640 1 -- 172.21.15.71:0/926501510 wait complete. 2024-09-05T12:26:54.016 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/926501510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:26:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:53 smithi174 ceph-mon[34313]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.385 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:54.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/926501510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:26:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:26:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:53 smithi071 ceph-mon[29845]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/926501510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:54.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:54.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:26:54.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:26:54.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:53 smithi123 ceph-mon[34321]: from='mgr.14150 172.21.15.71:0/3160655641' entity='mgr.x' 2024-09-05T12:26:55.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 -- 172.21.15.71:0/2852754725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978101de0 msgr2=0x7f89781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:55.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 --2- 172.21.15.71:0/2852754725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978101de0 0x7f89781021c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f896c009900 tx=0x7f896c02a410 comp rx=0 tx=0).stop 2024-09-05T12:26:55.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 -- 172.21.15.71:0/2852754725 shutdown_connections 2024-09-05T12:26:55.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 --2- 172.21.15.71:0/2852754725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978106f10 0x7f897810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 --2- 172.21.15.71:0/2852754725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8978102790 0x7f89781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 --2- 172.21.15.71:0/2852754725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978101de0 0x7f89781021c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.985+0000 7f8980400640 1 -- 172.21.15.71:0/2852754725 >> 172.21.15.71:0/2852754725 conn(0x7f89780fd8c0 msgr2=0x7f89780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.986+0000 7f8980400640 1 -- 172.21.15.71:0/2852754725 shutdown_connections 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.986+0000 7f8980400640 1 -- 172.21.15.71:0/2852754725 wait complete. 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 Processor -- start 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 -- start start 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 0x7f89780744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978102790 0x7f8978074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8978106f10 0x7f8978074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8978111fc0 con 0x7f8978102790 2024-09-05T12:26:55.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8978111e40 con 0x7f8978101de0 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f8980400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8978112140 con 0x7f8978106f10 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f897da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 0x7f89780744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f897da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 0x7f89780744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50568/0 (socket says 172.21.15.71:50568) 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f897da00640 1 -- 172.21.15.71:0/3325494188 learned_addr learned my addr 172.21.15.71:0/3325494188 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f897e400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8978106f10 0x7f8978074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.987+0000 7f897d000640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978102790 0x7f8978074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897da00640 1 -- 172.21.15.71:0/3325494188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8978106f10 msgr2=0x7f8978074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897da00640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8978106f10 0x7f8978074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897da00640 1 -- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978102790 msgr2=0x7f8978074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897da00640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978102790 0x7f8978074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897da00640 1 -- 172.21.15.71:0/3325494188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89781a68e0 con 0x7f8978101de0 2024-09-05T12:26:55.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897d000640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978102790 0x7f8978074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.988+0000 7f897da00640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 0x7f89780744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f896c004770 tx=0x7f896c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.989+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c03f070 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.989+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f896c031ca0 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.989+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f896c035030 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.989+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89781a6b70 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.989+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89781a7000 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.990+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89781021c0 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.992+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f896c042070 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.993+0000 7f8976000640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f89600a2c90 0x7f89600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.993+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f896c0efc40 con 0x7f8978101de0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.993+0000 7f897d000640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f89600a2c90 0x7f89600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:55.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.994+0000 7f897d000640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f89600a2c90 0x7f89600a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8978075f80 tx=0x7f896400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:55.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:54.995+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f896c0bc140 con 0x7f8978101de0 2024-09-05T12:26:55.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.175+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8978075a20 con 0x7f8978101de0 2024-09-05T12:26:55.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.177+0000 7f8976000640 1 -- 172.21.15.71:0/3325494188 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24741 (secure 0 0 0) 0x7f8978075a20 con 0x7f8978101de0 2024-09-05T12:26:55.179 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:55.179 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:34.259275+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:26:55.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f89600a2c90 msgr2=0x7f89600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:55.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f89600a2c90 0x7f89600a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8978075f80 tx=0x7f896400b410 comp rx=0 tx=0).stop 2024-09-05T12:26:55.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 msgr2=0x7f89780744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:55.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 0x7f89780744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f896c004770 tx=0x7f896c003c10 comp rx=0 tx=0).stop 2024-09-05T12:26:55.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 shutdown_connections 2024-09-05T12:26:55.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f89600a2c90 0x7f89600a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8978106f10 0x7f8978074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8978102790 0x7f8978074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 --2- 172.21.15.71:0/3325494188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8978101de0 0x7f89780744f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 >> 172.21.15.71:0/3325494188 conn(0x7f89780fd8c0 msgr2=0x7f89780fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 shutdown_connections 2024-09-05T12:26:55.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:55.179+0000 7f8980400640 1 -- 172.21.15.71:0/3325494188 wait complete. 2024-09-05T12:26:55.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:55 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3325494188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:55.568 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:55 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3325494188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:55 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3325494188' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:26:55.977 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:26:56.347 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:56.485 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:56 smithi071 ceph-mon[29845]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:26:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:56 smithi174 ceph-mon[34313]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:26:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:56 smithi123 ceph-mon[34321]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 -- 172.21.15.71:0/1584638761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 msgr2=0x7fb5b41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/1584638761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 0x7fb5b41043f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009900 tx=0x7fb5a402a410 comp rx=0 tx=0).stop 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 -- 172.21.15.71:0/1584638761 shutdown_connections 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/1584638761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5b4109140 0x7fb5b4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/1584638761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/1584638761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 0x7fb5b41043f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 -- 172.21.15.71:0/1584638761 >> 172.21.15.71:0/1584638761 conn(0x7fb5b40ffa50 msgr2=0x7fb5b4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 -- 172.21.15.71:0/1584638761 shutdown_connections 2024-09-05T12:26:56.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.982+0000 7fb5b9e00640 1 -- 172.21.15.71:0/1584638761 wait complete. 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.983+0000 7fb5b9e00640 1 Processor -- start 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 -- start start 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 0x7fb5b419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5b4109140 0x7fb5b419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4114160 con 0x7fb5b4104010 2024-09-05T12:26:56.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b4113fe0 con 0x7fb5b4109140 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b9e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b41142e0 con 0x7fb5b41049c0 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 0x7fb5b419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37030/0 (socket says 172.21.15.71:37030) 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b2000640 1 -- 172.21.15.71:0/4272781091 learned_addr learned my addr 172.21.15.71:0/4272781091 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.984+0000 7fb5b3400640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5b4109140 0x7fb5b419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b2000640 1 -- 172.21.15.71:0/4272781091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5b4109140 msgr2=0x7fb5b419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b2000640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5b4109140 0x7fb5b419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b2000640 1 -- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 msgr2=0x7fb5b419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b2000640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 0x7fb5b419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b2000640 1 -- 172.21.15.71:0/4272781091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b419f670 con 0x7fb5b41049c0 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b2000640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b419ac00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e9f0 tx=0x7fb5a000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:56.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000cd80 con 0x7fb5b41049c0 2024-09-05T12:26:56.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5b419f960 con 0x7fb5b41049c0 2024-09-05T12:26:56.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a0005500 con 0x7fb5b41049c0 2024-09-05T12:26:56.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0010690 con 0x7fb5b41049c0 2024-09-05T12:26:56.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.985+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5b41a7240 con 0x7fb5b41049c0 2024-09-05T12:26:56.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.986+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5b41043f0 con 0x7fb5b41049c0 2024-09-05T12:26:56.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.989+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5a00046c0 con 0x7fb5b41049c0 2024-09-05T12:26:56.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.990+0000 7fb5b8a00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb5980a2c10 0x7fb5980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:56.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.990+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fb5a00cfba0 con 0x7fb5b41049c0 2024-09-05T12:26:56.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.990+0000 7fb5b2a00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb5980a2c10 0x7fb5980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:56.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.991+0000 7fb5b2a00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb5980a2c10 0x7fb5980a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009230 tx=0x7fb5a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:56.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:56.992+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5a00541b0 con 0x7fb5b41049c0 2024-09-05T12:26:57.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.203+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5b419ba20 con 0x7fb5b41049c0 2024-09-05T12:26:57.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.204+0000 7fb5b8a00640 1 -- 172.21.15.71:0/4272781091 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11968 (secure 0 0 0) 0x7fb5a009c5e0 con 0x7fb5b41049c0 2024-09-05T12:26:57.206 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:26:57.206 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"btime":"2024-09-05T12:25:40:321146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:40.307362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:26:57.206 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb5980a2c10 msgr2=0x7fb5980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb5980a2c10 0x7fb5980a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4009230 tx=0x7fb5a40058d0 comp rx=0 tx=0).stop 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 msgr2=0x7fb5b419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b419ac00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000e9f0 tx=0x7fb5a000edf0 comp rx=0 tx=0).stop 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 shutdown_connections 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fb5980a2c10 0x7fb5980a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:57.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5b4109140 0x7fb5b419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:57.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5b41049c0 0x7fb5b419ac00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:57.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 --2- 172.21.15.71:0/4272781091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb5b4104010 0x7fb5b419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:57.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 >> 172.21.15.71:0/4272781091 conn(0x7fb5b40ffa50 msgr2=0x7fb5b4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:57.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 shutdown_connections 2024-09-05T12:26:57.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:57.206+0000 7fb5b9e00640 1 -- 172.21.15.71:0/4272781091 wait complete. 2024-09-05T12:26:57.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:57 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4272781091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:57.595 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:57 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4272781091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:57.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:57 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4272781091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:26:58.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T12:24:49.631865+0000', 'modified': '2024-09-05T12:25:40.307362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24514, 'mds_2': 24502, 'mds_3': 24490, 'mds_4': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14469': {'gid': 14469, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6841/991266062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6840', 'nonce': 991266062}, {'type': 'v1', 'addr': '172.21.15.71:6841', 'nonce': 991266062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/679053656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 679053656}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 679053656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/1436525377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1436525377}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1436525377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/2774200097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2774200097}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2774200097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/626911382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 626911382}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 626911382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.123:6837/3417870839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 3417870839}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3417870839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.174:6837/870087433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 870087433}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 870087433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.174:6839/584492863', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 584492863}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 584492863}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.71:6839/2593934735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 2593934735}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 2593934735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/782979893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 782979893}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 782979893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24485, 24524, 24490, 24502, 24514]} 2024-09-05T12:26:58.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T12:26:58.024 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T12:26:58.393 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:26:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:26:58 smithi174 ceph-mon[34313]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:58.531 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:26:58 smithi071 ceph-mon[29845]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:26:58 smithi123 ceph-mon[34321]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 -- 172.21.15.71:0/1286039143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911010e3c0 msgr2=0x7f9110110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 --2- 172.21.15.71:0/1286039143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911010e3c0 0x7f9110110810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ca0 tx=0x7f910c02a940 comp rx=0 tx=0).stop 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 -- 172.21.15.71:0/1286039143 shutdown_connections 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 --2- 172.21.15.71:0/1286039143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911010e3c0 0x7f9110110810 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 --2- 172.21.15.71:0/1286039143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9110074f40 0x7f911010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 --2- 172.21.15.71:0/1286039143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 0x7f9110074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 -- 172.21.15.71:0/1286039143 >> 172.21.15.71:0/1286039143 conn(0x7f91100fd8c0 msgr2=0x7f91100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.992+0000 7f9118800640 1 -- 172.21.15.71:0/1286039143 shutdown_connections 2024-09-05T12:26:58.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.993+0000 7f9118800640 1 -- 172.21.15.71:0/1286039143 wait complete. 2024-09-05T12:26:58.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.994+0000 7f9118800640 1 Processor -- start 2024-09-05T12:26:58.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.994+0000 7f9118800640 1 -- start start 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.994+0000 7f9118800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 0x7f911019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.994+0000 7f9118800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 0x7f911019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9118800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911010e3c0 0x7f911019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9118800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9110114150 con 0x7f9110074f40 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9118800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9110113fd0 con 0x7f911010e3c0 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9118800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91101142d0 con 0x7f9110074620 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 0x7f911019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 0x7f911019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59194/0 (socket says 172.21.15.71:59194) 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115400640 1 -- 172.21.15.71:0/1434217536 learned_addr learned my addr 172.21.15.71:0/1434217536 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115e00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 0x7f911019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9116800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911010e3c0 0x7f911019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115e00640 1 -- 172.21.15.71:0/1434217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911010e3c0 msgr2=0x7f911019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:58.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115e00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911010e3c0 0x7f911019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115e00640 1 -- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 msgr2=0x7f911019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115e00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 0x7f911019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115e00640 1 -- 172.21.15.71:0/1434217536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911019f5c0 con 0x7f9110074620 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.995+0000 7f9115400640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 0x7f911019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.996+0000 7f9115e00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 0x7f911019a610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f910400e9f0 tx=0x7f910400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.996+0000 7f910aa00640 1 -- 172.21.15.71:0/1434217536 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910400ccc0 con 0x7f9110074620 2024-09-05T12:26:58.999 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.996+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911019f8b0 con 0x7f9110074620 2024-09-05T12:26:59.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.996+0000 7f910aa00640 1 -- 172.21.15.71:0/1434217536 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9104010910 con 0x7f9110074620 2024-09-05T12:26:59.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.996+0000 7f910aa00640 1 -- 172.21.15.71:0/1434217536 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104018960 con 0x7f9110074620 2024-09-05T12:26:59.000 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.996+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91101a7190 con 0x7f9110074620 2024-09-05T12:26:59.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.999+0000 7f911b291740 1 Processor -- start 2024-09-05T12:26:59.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:58.999+0000 7f911b291740 1 -- start start 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f910aa00640 1 -- 172.21.15.71:0/1434217536 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9104010ac0 con 0x7f9110074620 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f911b291740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfe8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f911b291740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f911b291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cff3510 0x56157cff5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f911b291740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56157cf4e0b0 con 0x56157cfe8c50 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f911b291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56157cef5ab0 con 0x56157cff3510 2024-09-05T12:26:59.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f911b291740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56157cf4a070 con 0x56157cfe8330 2024-09-05T12:26:59.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f9115e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f9116800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfe8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f9116800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfe8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37068/0 (socket says 172.21.15.71:37068) 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f9116800640 1 -- 172.21.15.71:0/2615952534 learned_addr learned my addr 172.21.15.71:0/2615952534 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.000+0000 7f9115400640 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cff3510 0x56157cff5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9116800640 1 -- 172.21.15.71:0/2615952534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cff3510 msgr2=0x56157cff5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9116800640 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cff3510 0x56157cff5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9116800640 1 -- 172.21.15.71:0/2615952534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 msgr2=0x56157cff2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9116800640 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9116800640 1 -- 172.21.15.71:0/2615952534 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157cf024a0 con 0x56157cfe8330 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9115e00640 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:59.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f9116800640 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfe8710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f910c005fa0 tx=0x7f910c0093f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656ce4400a486ac8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:59.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f90ff000640 1 -- 172.21.15.71:0/2615952534 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c034040 con 0x56157cfe8330 2024-09-05T12:26:59.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f90ff000640 1 -- 172.21.15.71:0/2615952534 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910c02f930 con 0x56157cfe8330 2024-09-05T12:26:59.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f90ff000640 1 -- 172.21.15.71:0/2615952534 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c0308a0 con 0x56157cfe8330 2024-09-05T12:26:59.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.001+0000 7f910aa00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f90f80a2c90 0x7f90f80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f910aa00640 1 -- 172.21.15.71:0/1434217536 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f91040cf880 con 0x7f9110074620 2024-09-05T12:26:59.005 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f9115e00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f90f80a2c90 0x7f90f80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 -- 172.21.15.71:0/2615952534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 msgr2=0x56157cfe8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfe8710 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f910c005fa0 tx=0x7f910c0093f0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 -- 172.21.15.71:0/2615952534 shutdown_connections 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cff3510 0x56157cff5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 --2- 172.21.15.71:0/2615952534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfe8710 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 -- 172.21.15.71:0/2615952534 >> 172.21.15.71:0/2615952534 conn(0x56157cfe48c0 msgr2=0x56157cfe4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:59.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f9115e00640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f90f80a2c90 0x7f90f80a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f910409b970 tx=0x7f91040d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.002+0000 7f911b291740 1 -- 172.21.15.71:0/2615952534 shutdown_connections 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.003+0000 7f911b291740 1 -- 172.21.15.71:0/2615952534 wait complete. 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.003+0000 7f911b291740 1 -- start start 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.003+0000 7f911b291740 1 Processor -- start 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cff06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8c50 0x56157cff0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157cfff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56157cff6060 con 0x56157cff3510 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56157ced6920 con 0x56157cfe8330 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56157cff61e0 con 0x56157cfe8c50 2024-09-05T12:26:59.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f911b291740 10 client.0 _ll_register_callbacks cb 0x56157cefa8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f9116800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157cfff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f9116800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157cfff3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59222/0 (socket says 172.21.15.71:59222) 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f9116800640 1 -- 172.21.15.71:0/1549429000 learned_addr learned my addr 172.21.15.71:0/1549429000 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f9115400640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8c50 0x56157cff0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f9116800640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cff06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.004+0000 7f9115400640 1 -- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 msgr2=0x56157cff06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f9115400640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cff06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f9115400640 1 -- 172.21.15.71:0/1549429000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 msgr2=0x56157cfff3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f9115400640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157cfff3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f9115400640 1 -- 172.21.15.71:0/1549429000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157ccbf2b0 con 0x56157cfe8c50 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f9115400640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8c50 0x56157cff0bf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f911010f750 tx=0x7f91000039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8481e447455ea4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:59.011 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f90f3400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f90f3400640 1 -- 172.21.15.71:0/1549429000 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100011710 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910c02f930 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f90f3400640 1 -- 172.21.15.71:0/1549429000 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f91000118b0 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f90f3400640 1 -- 172.21.15.71:0/1549429000 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9100011ab0 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f911b291740 10 client.24970 fetch_fsmap learned FSMap version 29 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.005+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56157d001f50 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f90f3400640 1 -- 172.21.15.71:0/1549429000 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910c02f930 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f90f3400640 1 -- 172.21.15.71:0/1549429000 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9100010440 con 0x56157cfe8c50 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f911b291740 10 client.24970 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f911b291740 10 client.24970 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f911b291740 20 client.24970 populate_metadata read hostname 'smithi071' 2024-09-05T12:26:59.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f911b291740 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d004500 0x56157d005970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f911b291740 4 client.24970 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:26:59.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.006+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 --> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56157cf5b620 con 0x56157d004500 2024-09-05T12:26:59.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.007+0000 7f9116800640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d004500 0x56157d005970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.007+0000 7f9116800640 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d004500 0x56157d005970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=106d4e70aebc9085 server_cookie=51c0c83d4832b4bc in_seq=0 out_seq=0 2024-09-05T12:26:59.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.007+0000 7f90f3400640 10 client.24970 ms_handle_connect on v2:172.21.15.123:6836/3417870839 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f90f3400640 1 -- 172.21.15.71:0/1549429000 <== mds.4 v2:172.21.15.123:6836/3417870839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56157cf5b620 con 0x56157d004500 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f90f3400640 10 client.24970 handle_command_reply: tid=0 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 client.24970 shutdown 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8c50 msgr2=0x56157cff0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8c50 0x56157cff0bf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f911010f750 tx=0x7f91000039c0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 shutdown_connections 2024-09-05T12:26:59.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d004500 0x56157d005970 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157cfff3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8c50 0x56157cff0bf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 --2- 172.21.15.71:0/1549429000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cff06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f90f3400640 0 client.24970 ms_handle_reset on v2:172.21.15.123:6836/3417870839 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.008+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 >> 172.21.15.71:0/1549429000 conn(0x56157cfe48c0 msgr2=0x56157cfea5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.009+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 shutdown_connections 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.009+0000 7f911b291740 1 -- 172.21.15.71:0/1549429000 wait complete. 2024-09-05T12:26:59.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.009+0000 7f911b291740 20 client.24970 trim_cache size 0 max 16384 2024-09-05T12:26:59.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 Processor -- start 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 -- start start 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cfeba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cff3510 0x56157cff0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56157d003cb0 con 0x56157cfe8c50 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56157d004380 con 0x56157cfe8330 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.027+0000 7f911b291740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56157d004a50 con 0x56157cff3510 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cfeba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9115400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cfeba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50628/0 (socket says 172.21.15.71:50628) 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9115400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59234/0 (socket says 172.21.15.71:59234) 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 -- 172.21.15.71:0/2688901849 learned_addr learned my addr 172.21.15.71:0/2688901849 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cff3510 0x56157cff0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 -- 172.21.15.71:0/2688901849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cff3510 msgr2=0x56157cff0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cff3510 0x56157cff0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 -- 172.21.15.71:0/2688901849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 msgr2=0x56157cff0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 -- 172.21.15.71:0/2688901849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157cfe8950 con 0x56157cfe8330 2024-09-05T12:26:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f9116800640 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cfeba70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f910c000f80 tx=0x7f910c003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20704ca357ae34b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.028+0000 7f90fe600640 1 -- 172.21.15.71:0/2688901849 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c058070 con 0x56157cfe8330 2024-09-05T12:26:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f90fe600640 1 -- 172.21.15.71:0/2688901849 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910c02f930 con 0x56157cfe8330 2024-09-05T12:26:59.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f90fe600640 1 -- 172.21.15.71:0/2688901849 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c030a80 con 0x56157cfe8330 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 -- 172.21.15.71:0/2688901849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 msgr2=0x56157cfeba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cfeba70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f910c000f80 tx=0x7f910c003c30 comp rx=0 tx=0).stop 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 -- 172.21.15.71:0/2688901849 shutdown_connections 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cff3510 0x56157cff0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cfe8c50 0x56157cff0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 --2- 172.21.15.71:0/2688901849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8330 0x56157cfeba70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 -- 172.21.15.71:0/2688901849 >> 172.21.15.71:0/2688901849 conn(0x56157cfe48c0 msgr2=0x56157cfeaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 -- 172.21.15.71:0/2688901849 shutdown_connections 2024-09-05T12:26:59.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.029+0000 7f911b291740 1 -- 172.21.15.71:0/2688901849 wait complete. 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.030+0000 7f911b291740 1 -- start start 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.030+0000 7f911b291740 1 Processor -- start 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.030+0000 7f911b291740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.030+0000 7f911b291740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 0x56157cfed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f911b291740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157d00cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f911b291740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56157d0057f0 con 0x56157cff3510 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f911b291740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56157d005120 con 0x56157cfe8c50 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f911b291740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56157d005ec0 con 0x56157cfe8330 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f911b291740 10 client.0 _ll_register_callbacks cb 0x7f91000040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 0x56157cfed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9116800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9116800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37080/0 (socket says 172.21.15.71:37080) 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 0x56157cfed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50630/0 (socket says 172.21.15.71:50630) 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9116800640 1 -- 172.21.15.71:0/3597525453 learned_addr learned my addr 172.21.15.71:0/3597525453 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9116800640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157d00cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 -- 172.21.15.71:0/3597525453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 msgr2=0x56157cfecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 -- 172.21.15.71:0/3597525453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 msgr2=0x56157d00cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157d00cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9115400640 1 -- 172.21.15.71:0/3597525453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157cfedb30 con 0x56157cfe8c50 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.031+0000 7f9116800640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157d00cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:26:59.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f9115400640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 0x56157cfed260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9100010a50 tx=0x7f910000b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7887572598b92537 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:26:59.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f90f3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f90f3400640 1 -- 172.21.15.71:0/3597525453 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100010650 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f90f3400640 1 -- 172.21.15.71:0/3597525453 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f9100010ad0 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910c02f930 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f90f3400640 1 -- 172.21.15.71:0/3597525453 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9100010cd0 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f911b291740 10 client.24971 fetch_fsmap learned FSMap version 29 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.032+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56157d00e260 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f90f3400640 1 -- 172.21.15.71:0/3597525453 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910c02f930 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f90f3400640 1 -- 172.21.15.71:0/3597525453 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f910002a410 con 0x56157cfe8c50 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f911b291740 10 client.24971 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f911b291740 10 client.24971 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T12:26:59.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f911b291740 20 client.24971 populate_metadata read hostname 'smithi071' 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f911b291740 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d072fe0 0x56157d071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f911b291740 4 client.24971 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.033+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 --> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] -- command(tid 0: {"prefix": "status"}) -- 0x56157cf55b30 con 0x56157d072fe0 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f9116800640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d072fe0 0x56157d071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f9116800640 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d072fe0 0x56157d071cf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=892b3f91c80a2ccd server_cookie=61ece0708556af4 in_seq=0 out_seq=0 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f90f3400640 10 client.24971 ms_handle_connect on v2:172.21.15.123:6836/3417870839 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f90f3400640 1 -- 172.21.15.71:0/3597525453 <== mds.4 v2:172.21.15.123:6836/3417870839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56157cf55b30 con 0x56157d072fe0 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f90f3400640 10 client.24971 handle_command_reply: tid=0 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f911b291740 1 client.24971 shutdown 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 msgr2=0x56157cfed260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.034+0000 7f911b291740 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 0x56157cfed260 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9100010a50 tx=0x7f910000b650 comp rx=0 tx=0).stop 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 shutdown_connections 2024-09-05T12:26:59.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x56157d072fe0 0x56157d071cf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56157cff3510 0x56157d00cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56157cfe8c50 0x56157cfed260 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 --2- 172.21.15.71:0/3597525453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56157cfe8330 0x56157cfecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f90f3400640 0 client.24971 ms_handle_reset on v2:172.21.15.123:6836/3417870839 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 >> 172.21.15.71:0/3597525453 conn(0x56157cfe48c0 msgr2=0x56157cfeba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 shutdown_connections 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 1 -- 172.21.15.71:0/3597525453 wait complete. 2024-09-05T12:26:59.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.035+0000 7f911b291740 20 client.24971 trim_cache size 0 max 16384 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 4, 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24485, 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 80.722591875000006, 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:26:59.042 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:26:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:26:59.043 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 154.93407404300001 2024-09-05T12:26:59.043 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:26:59.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f90f80a2c90 msgr2=0x7f90f80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:59.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f90f80a2c90 0x7f90f80a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f910409b970 tx=0x7f91040d5530 comp rx=0 tx=0).stop 2024-09-05T12:26:59.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 msgr2=0x7f911019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 0x7f911019a610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f910400e9f0 tx=0x7f910400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 shutdown_connections 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f90f80a2c90 0x7f90f80a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911010e3c0 0x7f911019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9110074f40 0x7f911019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 --2- 172.21.15.71:0/1434217536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110074620 0x7f911019a610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 >> 172.21.15.71:0/1434217536 conn(0x7f91100fd8c0 msgr2=0x7f91100fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 shutdown_connections 2024-09-05T12:26:59.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:26:59.036+0000 7f9118800640 1 -- 172.21.15.71:0/1434217536 wait complete. 2024-09-05T12:26:59.859 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.c status 2024-09-05T12:27:00.224 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:00 smithi174 ceph-mon[34313]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:27:00.570 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:00 smithi071 ceph-mon[29845]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:27:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:00 smithi123 ceph-mon[34321]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:27:00.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 -- 172.21.15.71:0/565366848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 msgr2=0x7faa7c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 --2- 172.21.15.71:0/565366848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c10e890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7faa74009cd0 tx=0x7faa7402a8f0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 -- 172.21.15.71:0/565366848 shutdown_connections 2024-09-05T12:27:00.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 --2- 172.21.15.71:0/565366848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c10e890 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 --2- 172.21.15.71:0/565366848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c1019c0 0x7faa7c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 --2- 172.21.15.71:0/565366848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1079c0 0x7faa7c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 -- 172.21.15.71:0/565366848 >> 172.21.15.71:0/565366848 conn(0x7faa7c0fd740 msgr2=0x7faa7c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:00.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 -- 172.21.15.71:0/565366848 shutdown_connections 2024-09-05T12:27:00.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.820+0000 7faa81000640 1 -- 172.21.15.71:0/565366848 wait complete. 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.821+0000 7faa81000640 1 Processor -- start 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.821+0000 7faa81000640 1 -- start start 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa81000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1019c0 0x7faa7c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa81000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa81000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c1079c0 0x7faa7c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa81000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c111e10 con 0x7faa7c102360 2024-09-05T12:27:00.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa81000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c111c90 con 0x7faa7c1079c0 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa81000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c111f90 con 0x7faa7c1019c0 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1019c0 0x7faa7c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1019c0 0x7faa7c198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37088/0 (socket says 172.21.15.71:37088) 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59264/0 (socket says 172.21.15.71:59264) 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa72a00640 1 -- 172.21.15.71:0/3930366954 learned_addr learned my addr 172.21.15.71:0/3930366954 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:00.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.822+0000 7faa7be00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c1079c0 0x7faa7c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.823+0000 7faa72a00640 1 -- 172.21.15.71:0/3930366954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1019c0 msgr2=0x7faa7c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.823+0000 7faa72a00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1019c0 0x7faa7c198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.823+0000 7faa72a00640 1 -- 172.21.15.71:0/3930366954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c1079c0 msgr2=0x7faa7c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.823+0000 7faa72a00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c1079c0 0x7faa7c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.823+0000 7faa72a00640 1 -- 172.21.15.71:0/3930366954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c19d1c0 con 0x7faa7c102360 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.823+0000 7faa72a00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c198750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa6400e9f0 tx=0x7faa6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.824+0000 7faa78c00640 1 -- 172.21.15.71:0/3930366954 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6400cd80 con 0x7faa7c102360 2024-09-05T12:27:00.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.824+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c19d4b0 con 0x7faa7c102360 2024-09-05T12:27:00.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.824+0000 7faa78c00640 1 -- 172.21.15.71:0/3930366954 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa64010910 con 0x7faa7c102360 2024-09-05T12:27:00.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.824+0000 7faa78c00640 1 -- 172.21.15.71:0/3930366954 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640188f0 con 0x7faa7c102360 2024-09-05T12:27:00.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.824+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c1a4d90 con 0x7faa7c102360 2024-09-05T12:27:00.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.827+0000 7faa78c00640 1 -- 172.21.15.71:0/3930366954 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faa64010430 con 0x7faa7c102360 2024-09-05T12:27:00.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.827+0000 7faa8504c740 1 Processor -- start 2024-09-05T12:27:00.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.827+0000 7faa8504c740 1 -- start start 2024-09-05T12:27:00.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa8504c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9454e0 0x5653af9458c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa8504c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 0x5653af950180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa8504c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9506c0 0x5653af952ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa8504c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5653af8ab0b0 con 0x5653af9454e0 2024-09-05T12:27:00.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa8504c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653af852ab0 con 0x5653af9506c0 2024-09-05T12:27:00.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa8504c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653af8a7070 con 0x5653af945e00 2024-09-05T12:27:00.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 0x5653af950180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9454e0 0x5653af9458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 0x5653af950180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37094/0 (socket says 172.21.15.71:37094) 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa7b400640 1 -- 172.21.15.71:0/4136317393 learned_addr learned my addr 172.21.15.71:0/4136317393 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa72a00640 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9506c0 0x5653af952ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.828+0000 7faa78c00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7faa600a2c90 0x7faa600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7b400640 1 -- 172.21.15.71:0/4136317393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9506c0 msgr2=0x5653af952ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7b400640 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9506c0 0x5653af952ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7b400640 1 -- 172.21.15.71:0/4136317393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9454e0 msgr2=0x5653af9458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7b400640 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9454e0 0x5653af9458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7b400640 1 -- 172.21.15.71:0/4136317393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653af85f4a0 con 0x5653af945e00 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa78c00640 1 -- 172.21.15.71:0/3930366954 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7faa640cf770 con 0x7faa7c102360 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7b400640 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 0x5653af950180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00bc90 tx=0x7faa6c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a90ab2795690998 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:00.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa7be00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7faa600a2c90 0x7faa600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa70c00640 1 -- 172.21.15.71:0/4136317393 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c01c480 con 0x5653af945e00 2024-09-05T12:27:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa70c00640 1 -- 172.21.15.71:0/4136317393 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6c01cb00 con 0x5653af945e00 2024-09-05T12:27:00.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.829+0000 7faa70c00640 1 -- 172.21.15.71:0/4136317393 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011c60 con 0x5653af945e00 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 -- 172.21.15.71:0/4136317393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 msgr2=0x5653af950180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa7be00640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7faa600a2c90 0x7faa600a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa74004af0 tx=0x7faa740095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 0x5653af950180 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00bc90 tx=0x7faa6c0107a0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 -- 172.21.15.71:0/4136317393 shutdown_connections 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9506c0 0x5653af952ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af945e00 0x5653af950180 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 --2- 172.21.15.71:0/4136317393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9454e0 0x5653af9458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 -- 172.21.15.71:0/4136317393 >> 172.21.15.71:0/4136317393 conn(0x5653af941a70 msgr2=0x5653af941e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 -- 172.21.15.71:0/4136317393 shutdown_connections 2024-09-05T12:27:00.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.830+0000 7faa8504c740 1 -- 172.21.15.71:0/4136317393 wait complete. 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.831+0000 7faa8504c740 1 -- start start 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.831+0000 7faa8504c740 1 Processor -- start 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 0x5653af94d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af945e00 0x5653af94da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9506c0 0x5653af94dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5653af953330 con 0x5653af9506c0 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653af833920 con 0x5653af945e00 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653af9534b0 con 0x5653af9454e0 2024-09-05T12:27:00.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa8504c740 10 client.0 _ll_register_callbacks cb 0x5653af8578a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 0x5653af94d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 0x5653af94d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37102/0 (socket says 172.21.15.71:37102) 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 -- 172.21.15.71:0/4145103315 learned_addr learned my addr 172.21.15.71:0/4145103315 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7be00640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9506c0 0x5653af94dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa72a00640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af945e00 0x5653af94da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 -- 172.21.15.71:0/4145103315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af945e00 msgr2=0x5653af94da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af945e00 0x5653af94da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 -- 172.21.15.71:0/4145103315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9506c0 msgr2=0x5653af94dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9506c0 0x5653af94dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.832+0000 7faa7b400640 1 -- 172.21.15.71:0/4145103315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653af61c2b0 con 0x5653af9454e0 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa7b400640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 0x5653af94d520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa6c011840 tx=0x7faa6c012be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82168a7f236828e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:00.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa69600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:27:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa69600640 1 -- 172.21.15.71:0/4145103315 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c02c060 con 0x5653af9454e0 2024-09-05T12:27:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa69600640 1 -- 172.21.15.71:0/4145103315 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7faa6c012dd0 con 0x5653af9454e0 2024-09-05T12:27:00.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa6c01cb00 con 0x5653af9454e0 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa69600640 1 -- 172.21.15.71:0/4145103315 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa6c033050 con 0x5653af9454e0 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa8504c740 10 client.24988 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.833+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653af9633d0 con 0x5653af9454e0 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa69600640 1 -- 172.21.15.71:0/4145103315 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa6c01cb00 con 0x5653af9454e0 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa69600640 1 -- 172.21.15.71:0/4145103315 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faa6c0054c0 con 0x5653af9454e0 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa8504c740 10 client.24988 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:00.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa8504c740 10 client.24988 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa8504c740 20 client.24988 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa8504c740 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9657a0 0x5653af966c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa8504c740 4 client.24988 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.834+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 --> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653af8b8620 con 0x5653af9657a0 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.835+0000 7faa72a00640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9657a0 0x5653af966c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.835+0000 7faa72a00640 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9657a0 0x5653af966c10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=182d31d19d8ef582 server_cookie=38ed960808743401 in_seq=0 out_seq=0 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.835+0000 7faa69600640 10 client.24988 ms_handle_connect on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa69600640 1 -- 172.21.15.71:0/4145103315 <== mds.3 v2:172.21.15.174:6836/870087433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653af8b8620 con 0x5653af9657a0 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa69600640 10 client.24988 handle_command_reply: tid=0 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 client.24988 shutdown 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 msgr2=0x5653af94d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 0x5653af94d520 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa6c011840 tx=0x7faa6c012be0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 shutdown_connections 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9657a0 0x5653af966c10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af9506c0 0x5653af94dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af945e00 0x5653af94da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 --2- 172.21.15.71:0/4145103315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9454e0 0x5653af94d520 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa69600640 0 client.24988 ms_handle_reset on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.836+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 >> 172.21.15.71:0/4145103315 conn(0x5653af941a70 msgr2=0x5653af9477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.837+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 shutdown_connections 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.837+0000 7faa8504c740 1 -- 172.21.15.71:0/4145103315 wait complete. 2024-09-05T12:27:00.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.837+0000 7faa8504c740 20 client.24988 trim_cache size 0 max 16384 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 Processor -- start 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 -- start start 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af947f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af960300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af94da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5653af964f50 con 0x5653af945e00 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653af965620 con 0x5653af9454e0 2024-09-05T12:27:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa8504c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653af965cf0 con 0x5653af9506c0 2024-09-05T12:27:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.855+0000 7faa72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af960300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa7be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af94da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af947f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af960300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59302/0 (socket says 172.21.15.71:59302) 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 -- 172.21.15.71:0/3951056424 learned_addr learned my addr 172.21.15.71:0/3951056424 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 -- 172.21.15.71:0/3951056424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 msgr2=0x5653af94da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af94da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 -- 172.21.15.71:0/3951056424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 msgr2=0x5653af947f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af947f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 -- 172.21.15.71:0/3951056424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653af9418a0 con 0x5653af945e00 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa72a00640 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af960300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa6409baf0 tx=0x7faa64000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80db708f0a0a060e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa6be00640 1 -- 172.21.15.71:0/3951056424 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640a4720 con 0x5653af945e00 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa6be00640 1 -- 172.21.15.71:0/3951056424 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa64010720 con 0x5653af945e00 2024-09-05T12:27:00.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.856+0000 7faa6be00640 1 -- 172.21.15.71:0/3951056424 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640cfb60 con 0x5653af945e00 2024-09-05T12:27:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 -- 172.21.15.71:0/3951056424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 msgr2=0x5653af960300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af960300 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa6409baf0 tx=0x7faa64000bc0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 -- 172.21.15.71:0/3951056424 shutdown_connections 2024-09-05T12:27:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af94da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af960300 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 --2- 172.21.15.71:0/3951056424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af947f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 -- 172.21.15.71:0/3951056424 >> 172.21.15.71:0/3951056424 conn(0x5653af941a70 msgr2=0x5653af95fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:00.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 -- 172.21.15.71:0/3951056424 shutdown_connections 2024-09-05T12:27:00.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.857+0000 7faa8504c740 1 -- 172.21.15.71:0/3951056424 wait complete. 2024-09-05T12:27:00.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 -- start start 2024-09-05T12:27:00.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 Processor -- start 2024-09-05T12:27:00.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af969cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af96a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af96a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5653af966a90 con 0x5653af945e00 2024-09-05T12:27:00.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653af9663c0 con 0x5653af9454e0 2024-09-05T12:27:00.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653af967160 con 0x5653af9506c0 2024-09-05T12:27:00.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa8504c740 10 client.0 _ll_register_callbacks cb 0x7faa6c00bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:00.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.858+0000 7faa72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af96a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af969cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af96a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59312/0 (socket says 172.21.15.71:59312) 2024-09-05T12:27:00.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 -- 172.21.15.71:0/477671749 learned_addr learned my addr 172.21.15.71:0/477671749 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:00.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa7be00640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af96a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 -- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 msgr2=0x5653af96a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af96a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 -- 172.21.15.71:0/477671749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 msgr2=0x5653af969cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:00.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af969cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 -- 172.21.15.71:0/477671749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653af96ee50 con 0x5653af945e00 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa7be00640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af96a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa72a00640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af96a210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa6400e9c0 tx=0x7faa6409a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef29a9bf73ed614 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa69600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa69600640 1 -- 172.21.15.71:0/477671749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa640b3070 con 0x5653af945e00 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa69600640 1 -- 172.21.15.71:0/477671749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7faa640981d0 con 0x5653af945e00 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.859+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa64010720 con 0x5653af945e00 2024-09-05T12:27:00.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa69600640 1 -- 172.21.15.71:0/477671749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa640ccfb0 con 0x5653af945e00 2024-09-05T12:27:00.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 10 client.14940 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:00.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653af9d02a0 con 0x5653af945e00 2024-09-05T12:27:00.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa69600640 1 -- 172.21.15.71:0/477671749 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa64010720 con 0x5653af945e00 2024-09-05T12:27:00.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa69600640 1 -- 172.21.15.71:0/477671749 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7faa6409b020 con 0x5653af945e00 2024-09-05T12:27:00.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 10 client.14940 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:00.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 10 client.14940 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T12:27:00.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 20 client.14940 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:00.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9d49c0 0x5653af9d36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:00.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 4 client.14940 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:00.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.860+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 --> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] -- command(tid 0: {"prefix": "status"}) -- 0x5653af8b2b30 con 0x5653af9d49c0 2024-09-05T12:27:00.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.861+0000 7faa7b400640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9d49c0 0x5653af9d36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:00.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.861+0000 7faa7b400640 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9d49c0 0x5653af9d36d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ac136986ac9f1213 server_cookie=eda51aad2e6402e8 in_seq=0 out_seq=0 2024-09-05T12:27:00.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.861+0000 7faa69600640 10 client.14940 ms_handle_connect on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:00.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.861+0000 7faa69600640 1 -- 172.21.15.71:0/477671749 <== mds.3 v2:172.21.15.174:6836/870087433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5653af8b2b30 con 0x5653af9d49c0 2024-09-05T12:27:00.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.861+0000 7faa69600640 10 client.14940 handle_command_reply: tid=0 2024-09-05T12:27:00.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.861+0000 7faa8504c740 1 client.14940 shutdown 2024-09-05T12:27:00.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 msgr2=0x5653af96a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af96a210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa6400e9c0 tx=0x7faa6409a070 comp rx=0 tx=0).stop 2024-09-05T12:27:00.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 shutdown_connections 2024-09-05T12:27:00.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x5653af9d49c0 0x5653af9d36d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653af9506c0 0x5653af96a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5653af945e00 0x5653af96a210 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa69600640 0 client.14940 ms_handle_reset on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:00.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 --2- 172.21.15.71:0/477671749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653af9454e0 0x5653af969cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 >> 172.21.15.71:0/477671749 conn(0x5653af941a70 msgr2=0x5653af960410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:00.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 shutdown_connections 2024-09-05T12:27:00.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 1 -- 172.21.15.71:0/477671749 wait complete. 2024-09-05T12:27:00.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.862+0000 7faa8504c740 20 client.14940 trim_cache size 0 max 16384 2024-09-05T12:27:00.876 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 3, 2024-09-05T12:27:00.876 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24490, 2024-09-05T12:27:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 83.558575485000006, 2024-09-05T12:27:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:00.877 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:00.878 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 159.50787450999999 2024-09-05T12:27:00.878 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7faa600a2c90 msgr2=0x7faa600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7faa600a2c90 0x7faa600a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7faa74004af0 tx=0x7faa740095c0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 msgr2=0x7faa7c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:00.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c198750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa6400e9f0 tx=0x7faa6400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 shutdown_connections 2024-09-05T12:27:00.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7faa600a2c90 0x7faa600a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa7c1079c0 0x7faa7c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa7c102360 0x7faa7c198750 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 --2- 172.21.15.71:0/3930366954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa7c1019c0 0x7faa7c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:00.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 >> 172.21.15.71:0/3930366954 conn(0x7faa7c0fd740 msgr2=0x7faa7c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:00.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 shutdown_connections 2024-09-05T12:27:00.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:00.863+0000 7faa81000640 1 -- 172.21.15.71:0/3930366954 wait complete. 2024-09-05T12:27:01.603 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T12:27:01.969 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:02.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:02 smithi071 ceph-mon[29845]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:02 smithi174 ceph-mon[34313]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:02.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.556+0000 7f44c8a00640 1 -- 172.21.15.71:0/3566254544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4103e30 msgr2=0x7f44c4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.556+0000 7f44c8a00640 1 --2- 172.21.15.71:0/3566254544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4103e30 0x7f44c4104210 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b402a410 comp rx=0 tx=0).stop 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 -- 172.21.15.71:0/3566254544 shutdown_connections 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 --2- 172.21.15.71:0/3566254544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4108f60 0x7f44c4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 --2- 172.21.15.71:0/3566254544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 0x7f44c4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 --2- 172.21.15.71:0/3566254544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4103e30 0x7f44c4104210 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 -- 172.21.15.71:0/3566254544 >> 172.21.15.71:0/3566254544 conn(0x7f44c40ff8d0 msgr2=0x7f44c4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 -- 172.21.15.71:0/3566254544 shutdown_connections 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.557+0000 7f44c8a00640 1 -- 172.21.15.71:0/3566254544 wait complete. 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 Processor -- start 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 -- start start 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4103e30 0x7f44c419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 0x7f44c419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 0x7f44c419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4114010 con 0x7f44c4108f60 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4113e90 con 0x7f44c4103e30 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c8a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4114190 con 0x7f44c41047e0 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 0x7f44c419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4103e30 0x7f44c419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 0x7f44c419ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59326/0 (socket says 172.21.15.71:59326) 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44c3e00640 1 -- 172.21.15.71:0/4268774246 learned_addr learned my addr 172.21.15.71:0/4268774246 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.558+0000 7f44bbe00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 0x7f44c419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3e00640 1 -- 172.21.15.71:0/4268774246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 msgr2=0x7f44c419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3e00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 0x7f44c419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3e00640 1 -- 172.21.15.71:0/4268774246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4103e30 msgr2=0x7f44c419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3e00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4103e30 0x7f44c419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3e00640 1 -- 172.21.15.71:0/4268774246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c419f440 con 0x7f44c4108f60 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3400640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4103e30 0x7f44c419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44bbe00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 0x7f44c419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c3e00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 0x7f44c419ed60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009fd0 tx=0x7f44bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c0c00640 1 -- 172.21.15.71:0/4268774246 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc017070 con 0x7f44c4108f60 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c0c00640 1 -- 172.21.15.71:0/4268774246 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44bc003e50 con 0x7f44c4108f60 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c419f730 con 0x7f44c4108f60 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c0c00640 1 -- 172.21.15.71:0/4268774246 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0106c0 con 0x7f44c4108f60 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.559+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41a7010 con 0x7f44c4108f60 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 Processor -- start 2024-09-05T12:27:02.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44c0c00640 1 -- 172.21.15.71:0/4268774246 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f44bc003970 con 0x7f44c4108f60 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 -- start start 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e124e0 0x55c2d8e128c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8d780b0 con 0x55c2d8e12e00 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8d1fab0 con 0x55c2d8e124e0 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.564+0000 7f44cca4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8d74070 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e124e0 0x55c2d8e128c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59334/0 (socket says 172.21.15.71:59334) 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 -- 172.21.15.71:0/2370145800 learned_addr learned my addr 172.21.15.71:0/2370145800 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44bbe00640 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 -- 172.21.15.71:0/2370145800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e1d6c0 msgr2=0x55c2d8e1fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 -- 172.21.15.71:0/2370145800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e124e0 msgr2=0x55c2d8e128c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e124e0 0x55c2d8e128c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.565+0000 7f44c3400640 1 -- 172.21.15.71:0/2370145800 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d8d2c4a0 con 0x55c2d8e12e00 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44c0c00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f44a80a2c10 0x7f44a80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44c3400640 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1d180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b40043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e8a7f6cd376fbe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44c0c00640 1 -- 172.21.15.71:0/4268774246 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f44bc0cf1f0 con 0x7f44c4108f60 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44bbe00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f44a80a2c10 0x7f44a80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44b3e00640 1 -- 172.21.15.71:0/2370145800 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4035040 con 0x55c2d8e12e00 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44b3e00640 1 -- 172.21.15.71:0/2370145800 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4004760 con 0x55c2d8e12e00 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.566+0000 7f44b3e00640 1 -- 172.21.15.71:0/2370145800 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40039b0 con 0x55c2d8e12e00 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 -- 172.21.15.71:0/2370145800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 msgr2=0x55c2d8e1d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1d180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009900 tx=0x7f44b40043f0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44bbe00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f44a80a2c10 0x7f44a80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55c2d8e0fdf0 tx=0x7f44ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 -- 172.21.15.71:0/2370145800 shutdown_connections 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1d180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 --2- 172.21.15.71:0/2370145800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e124e0 0x55c2d8e128c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 -- 172.21.15.71:0/2370145800 >> 172.21.15.71:0/2370145800 conn(0x55c2d8e0ea70 msgr2=0x55c2d8e0ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 -- 172.21.15.71:0/2370145800 shutdown_connections 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 -- 172.21.15.71:0/2370145800 wait complete. 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.567+0000 7f44cca4c740 1 -- start start 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 Processor -- start 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e1a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e202d0 con 0x55c2d8e124e0 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8d00920 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e20450 con 0x55c2d8e12e00 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44cca4c740 10 client.0 _ll_register_callbacks cb 0x55c2d8d248a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e1a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44bbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e1a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59344/0 (socket says 172.21.15.71:59344) 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.568+0000 7f44c3400640 1 -- 172.21.15.71:0/1221848466 learned_addr learned my addr 172.21.15.71:0/1221848466 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3e00640 1 -- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 msgr2=0x55c2d8e1aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3e00640 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3e00640 1 -- 172.21.15.71:0/1221848466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 msgr2=0x55c2d8e1a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3e00640 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e1a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3e00640 1 -- 172.21.15.71:0/1221848466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d8ae92b0 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3400640 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e1a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44c3e00640 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1afe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0cc840 tx=0x7f44bc09b200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68e9a3466217f9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44b0c00640 1 -- 172.21.15.71:0/1221848466 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc020040 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b4004760 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.569+0000 7f44b0c00640 1 -- 172.21.15.71:0/1221848466 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f44bc0040b0 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44b0c00640 1 -- 172.21.15.71:0/1221848466 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44bc09b640 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 10 client.25001 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2d8e2bf80 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44b0c00640 1 -- 172.21.15.71:0/1221848466 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4004760 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44b0c00640 1 -- 172.21.15.71:0/1221848466 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f44bc097cb0 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 10 client.25001 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:02.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 10 client.25001 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 20 client.25001 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e2e610 0x55c2d8e2fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 4 client.25001 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.570+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 --> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2d8d85620 con 0x55c2d8e2e610 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.571+0000 7f44c3400640 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e2e610 0x55c2d8e2fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.571+0000 7f44c3400640 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e2e610 0x55c2d8e2fa80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b747172edb6c89c6 server_cookie=f1a8318febfecda4 in_seq=0 out_seq=0 2024-09-05T12:27:02.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.571+0000 7f44b0c00640 10 client.25001 ms_handle_connect on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.572+0000 7f44b0c00640 1 -- 172.21.15.71:0/1221848466 <== mds.2 v2:172.21.15.174:6838/584492863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2d8d85620 con 0x55c2d8e2e610 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.572+0000 7f44b0c00640 10 client.25001 handle_command_reply: tid=0 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.572+0000 7f44cca4c740 1 client.25001 shutdown 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.572+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 msgr2=0x55c2d8e1afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.572+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1afe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0cc840 tx=0x7f44bc09b200 comp rx=0 tx=0).stop 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 shutdown_connections 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e2e610 0x55c2d8e2fa80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e1afe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e1aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1221848466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e1a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 >> 172.21.15.71:0/1221848466 conn(0x55c2d8e0ea70 msgr2=0x55c2d8e147a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:02.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44b0c00640 0 client.25001 ms_handle_reset on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:02.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 shutdown_connections 2024-09-05T12:27:02.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 1 -- 172.21.15.71:0/1221848466 wait complete. 2024-09-05T12:27:02.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.573+0000 7f44cca4c740 20 client.25001 trim_cache size 0 max 16384 2024-09-05T12:27:02.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 Processor -- start 2024-09-05T12:27:02.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 -- start start 2024-09-05T12:27:02.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 0x55c2d8e14f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e12e00 0x55c2d8e09840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e18e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e2ddc0 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e2e490 con 0x55c2d8e12e00 2024-09-05T12:27:02.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44cca4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e2eb60 con 0x55c2d8e124e0 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44c3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 0x55c2d8e14f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44c3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 0x55c2d8e14f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37170/0 (socket says 172.21.15.71:37170) 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44c3400640 1 -- 172.21.15.71:0/3654120160 learned_addr learned my addr 172.21.15.71:0/3654120160 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44c3e00640 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e18e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.592+0000 7f44bbe00640 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e12e00 0x55c2d8e09840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44c3400640 1 -- 172.21.15.71:0/3654120160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e12e00 msgr2=0x55c2d8e09840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44c3400640 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e12e00 0x55c2d8e09840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44c3400640 1 -- 172.21.15.71:0/3654120160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e1d6c0 msgr2=0x55c2d8e18e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44c3400640 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e18e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44c3400640 1 -- 172.21.15.71:0/3654120160 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d8e0fc00 con 0x55c2d8e124e0 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44c3400640 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 0x55c2d8e14f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005ab0 tx=0x7f44b4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eeb838074a0cd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44b3400640 1 -- 172.21.15.71:0/3654120160 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4035070 con 0x55c2d8e124e0 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44b3400640 1 -- 172.21.15.71:0/3654120160 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4003e00 con 0x55c2d8e124e0 2024-09-05T12:27:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.593+0000 7f44b3400640 1 -- 172.21.15.71:0/3654120160 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4031e20 con 0x55c2d8e124e0 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 -- 172.21.15.71:0/3654120160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 msgr2=0x55c2d8e14f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 0x55c2d8e14f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005ab0 tx=0x7f44b4000bc0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 -- 172.21.15.71:0/3654120160 shutdown_connections 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e18e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e12e00 0x55c2d8e09840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 --2- 172.21.15.71:0/3654120160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e124e0 0x55c2d8e14f90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 -- 172.21.15.71:0/3654120160 >> 172.21.15.71:0/3654120160 conn(0x55c2d8e0ea70 msgr2=0x55c2d8e0ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 -- 172.21.15.71:0/3654120160 shutdown_connections 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 -- 172.21.15.71:0/3654120160 wait complete. 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.594+0000 7f44cca4c740 1 -- start start 2024-09-05T12:27:02.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 Processor -- start 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e30cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e31230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e31770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e2f900 con 0x55c2d8e124e0 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e2f230 con 0x55c2d8e1d6c0 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d8e2ffd0 con 0x55c2d8e12e00 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44cca4c740 10 client.0 _ll_register_callbacks cb 0x7f44bc09b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e30cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e30cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59356/0 (socket says 172.21.15.71:59356) 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44c3400640 1 -- 172.21.15.71:0/1222670684 learned_addr learned my addr 172.21.15.71:0/1222670684 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.595+0000 7f44bbe00640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e31230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3e00640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e31770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.598 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3400640 1 -- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 msgr2=0x55c2d8e31230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3400640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e31230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3400640 1 -- 172.21.15.71:0/1222670684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 msgr2=0x55c2d8e31770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3400640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e31770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3400640 1 -- 172.21.15.71:0/1222670684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d8e96dc0 con 0x55c2d8e124e0 2024-09-05T12:27:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44c3400640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e30cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44b4031550 tx=0x7f44b4005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b4dba592b0fc82 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:02.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.596+0000 7f44b0c00640 1 -- 172.21.15.71:0/1222670684 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4063060 con 0x55c2d8e124e0 2024-09-05T12:27:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44b0c00640 1 -- 172.21.15.71:0/1222670684 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f44b4031ba0 con 0x55c2d8e124e0 2024-09-05T12:27:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b4003e00 con 0x55c2d8e124e0 2024-09-05T12:27:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44b0c00640 1 -- 172.21.15.71:0/1222670684 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44b406a050 con 0x55c2d8e124e0 2024-09-05T12:27:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44cca4c740 10 client.14958 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:02.601 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2d8e980a0 con 0x55c2d8e124e0 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44b0c00640 1 -- 172.21.15.71:0/1222670684 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b4003e00 con 0x55c2d8e124e0 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44b0c00640 1 -- 172.21.15.71:0/1222670684 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f44b405a580 con 0x55c2d8e124e0 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44cca4c740 10 client.14958 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44cca4c740 10 client.14958 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.597+0000 7f44cca4c740 20 client.14958 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.598+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e9c590 0x55c2d8e9b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.598+0000 7f44cca4c740 4 client.14958 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.598+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 --> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] -- command(tid 0: {"prefix": "status"}) -- 0x55c2d8d7fb30 con 0x55c2d8e9c590 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.598+0000 7f44bbe00640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e9c590 0x55c2d8e9b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44b0c00640 10 client.14958 ms_handle_connect on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44bbe00640 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e9c590 0x55c2d8e9b2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a57c1ea2d990b968 server_cookie=1159e3b50381c4c5 in_seq=0 out_seq=0 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44b0c00640 1 -- 172.21.15.71:0/1222670684 <== mds.2 v2:172.21.15.174:6838/584492863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c2d8d7fb30 con 0x55c2d8e9c590 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44b0c00640 10 client.14958 handle_command_reply: tid=0 2024-09-05T12:27:02.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 client.14958 shutdown 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 msgr2=0x55c2d8e30cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e30cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f44b4031550 tx=0x7f44b4005a40 comp rx=0 tx=0).stop 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 shutdown_connections 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55c2d8e9c590 0x55c2d8e9b2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d8e1d6c0 0x55c2d8e31770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44b0c00640 0 client.14958 ms_handle_reset on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d8e12e00 0x55c2d8e31230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 --2- 172.21.15.71:0/1222670684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2d8e124e0 0x55c2d8e30cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 >> 172.21.15.71:0/1222670684 conn(0x55c2d8e0ea70 msgr2=0x55c2d8e0ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 shutdown_connections 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.599+0000 7f44cca4c740 1 -- 172.21.15.71:0/1222670684 wait complete. 2024-09-05T12:27:02.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.600+0000 7f44cca4c740 20 client.14958 trim_cache size 0 max 16384 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 2, 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24502, 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 86.308237720999998, 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 155.78897834899999 2024-09-05T12:27:02.604 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.600+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f44a80a2c10 msgr2=0x7f44a80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.600+0000 7f44c8a00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f44a80a2c10 0x7f44a80a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x55c2d8e0fdf0 tx=0x7f44ac005850 comp rx=0 tx=0).stop 2024-09-05T12:27:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.600+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 msgr2=0x7f44c419ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.600+0000 7f44c8a00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 0x7f44c419ed60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f44bc009fd0 tx=0x7f44bc00b260 comp rx=0 tx=0).stop 2024-09-05T12:27:02.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 shutdown_connections 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f44a80a2c10 0x7f44a80a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44c4108f60 0x7f44c419ed60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41047e0 0x7f44c419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 --2- 172.21.15.71:0/4268774246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44c4103e30 0x7f44c419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 >> 172.21.15.71:0/4268774246 conn(0x7f44c40ff8d0 msgr2=0x7f44c4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 shutdown_connections 2024-09-05T12:27:02.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:02.601+0000 7f44c8a00640 1 -- 172.21.15.71:0/4268774246 wait complete. 2024-09-05T12:27:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:02 smithi123 ceph-mon[34321]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:03.440 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.g status 2024-09-05T12:27:03.804 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:04.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 -- 172.21.15.71:0/671235339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8102790 msgr2=0x7ff4f81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 --2- 172.21.15.71:0/671235339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8102790 0x7ff4f81067e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4009900 tx=0x7ff4e402a410 comp rx=0 tx=0).stop 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 -- 172.21.15.71:0/671235339 shutdown_connections 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 --2- 172.21.15.71:0/671235339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4f8106f10 0x7ff4f810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 --2- 172.21.15.71:0/671235339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8102790 0x7ff4f81067e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 --2- 172.21.15.71:0/671235339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f8101de0 0x7ff4f81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 -- 172.21.15.71:0/671235339 >> 172.21.15.71:0/671235339 conn(0x7ff4f80fd8c0 msgr2=0x7ff4f80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 -- 172.21.15.71:0/671235339 shutdown_connections 2024-09-05T12:27:04.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.383+0000 7ff500600640 1 -- 172.21.15.71:0/671235339 wait complete. 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.384+0000 7ff500600640 1 Processor -- start 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.384+0000 7ff500600640 1 -- start start 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.384+0000 7ff500600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 0x7ff4f819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.384+0000 7ff500600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f8102790 0x7ff4f819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff500600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4f8106f10 0x7ff4f819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff500600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8111f70 con 0x7ff4f8101de0 2024-09-05T12:27:04.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff500600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f8111df0 con 0x7ff4f8102790 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff500600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f81120f0 con 0x7ff4f8106f10 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 0x7ff4f819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 0x7ff4f819a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60500/0 (socket says 172.21.15.71:60500) 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 -- 172.21.15.71:0/3184890552 learned_addr learned my addr 172.21.15.71:0/3184890552 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fd200640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f8102790 0x7ff4f819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fe600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4f8106f10 0x7ff4f819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 -- 172.21.15.71:0/3184890552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4f8106f10 msgr2=0x7ff4f819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4f8106f10 0x7ff4f819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 -- 172.21.15.71:0/3184890552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f8102790 msgr2=0x7ff4f819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f8102790 0x7ff4f819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.385+0000 7ff4fdc00640 1 -- 172.21.15.71:0/3184890552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f819f660 con 0x7ff4f8101de0 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.386+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 0x7ff4f819a6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e9f0 tx=0x7ff4ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.386+0000 7ff4f6000640 1 -- 172.21.15.71:0/3184890552 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec00cd80 con 0x7ff4f8101de0 2024-09-05T12:27:04.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.386+0000 7ff4f6000640 1 -- 172.21.15.71:0/3184890552 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4ec010910 con 0x7ff4f8101de0 2024-09-05T12:27:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.386+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f819f950 con 0x7ff4f8101de0 2024-09-05T12:27:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.386+0000 7ff4f6000640 1 -- 172.21.15.71:0/3184890552 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec018990 con 0x7ff4f8101de0 2024-09-05T12:27:04.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.386+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f81a7230 con 0x7ff4f8101de0 2024-09-05T12:27:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 Processor -- start 2024-09-05T12:27:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 -- start start 2024-09-05T12:27:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff4f6000640 1 -- 172.21.15.71:0/3184890552 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4ec010ac0 con 0x7ff4f8101de0 2024-09-05T12:27:04.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7c6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6c50 0x55f62f7d0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7d3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f62f72c0b0 con 0x55f62f7c6330 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f62f6d3ab0 con 0x55f62f7d1510 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.389+0000 7ff503254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f62f728070 con 0x55f62f7c6c50 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fe600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fe600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60504/0 (socket says 172.21.15.71:60504) 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fdc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6c50 0x55f62f7d0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fe600640 1 -- 172.21.15.71:0/2396431365 learned_addr learned my addr 172.21.15.71:0/2396431365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:04.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fd200640 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7d3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fd200640 1 -- 172.21.15.71:0/2396431365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6c50 msgr2=0x55f62f7d0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fd200640 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6c50 0x55f62f7d0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fd200640 1 -- 172.21.15.71:0/2396431365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 msgr2=0x55f62f7c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fd200640 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.390+0000 7ff4fd200640 1 -- 172.21.15.71:0/2396431365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62f6e04a0 con 0x55f62f7d1510 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4fd200640 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7d3900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4005940 tx=0x7ff4e4005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce490a7181787d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4eaa00640 1 -- 172.21.15.71:0/2396431365 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4040070 con 0x55f62f7d1510 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4eaa00640 1 -- 172.21.15.71:0/2396431365 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4031870 con 0x55f62f7d1510 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4eaa00640 1 -- 172.21.15.71:0/2396431365 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4035040 con 0x55f62f7d1510 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4f6000640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4e00a2c00 0x7ff4e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4f6000640 1 -- 172.21.15.71:0/3184890552 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff4ec0cf670 con 0x7ff4f8101de0 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.391+0000 7ff4fe600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4e00a2c00 0x7ff4e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 -- 172.21.15.71:0/2396431365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 msgr2=0x55f62f7d3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7d3900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4005940 tx=0x7ff4e4005be0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 -- 172.21.15.71:0/2396431365 shutdown_connections 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7d3900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.394 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6c50 0x55f62f7d0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 --2- 172.21.15.71:0/2396431365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 -- 172.21.15.71:0/2396431365 >> 172.21.15.71:0/2396431365 conn(0x55f62f7c28c0 msgr2=0x55f62f7c2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 -- 172.21.15.71:0/2396431365 shutdown_connections 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff4fe600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4e00a2c00 0x7ff4e00a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0011630 tx=0x7ff4f0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 -- 172.21.15.71:0/2396431365 wait complete. 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.392+0000 7ff503254740 1 -- start start 2024-09-05T12:27:04.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 Processor -- start 2024-09-05T12:27:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6c50 0x55f62f7cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7d4060 con 0x55f62f7c6330 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f62f6b4920 con 0x55f62f7c6c50 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7d41e0 con 0x55f62f7d1510 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff503254740 10 client.0 _ll_register_callbacks cb 0x55f62f6d88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff4fdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff4fdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60516/0 (socket says 172.21.15.71:60516) 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.393+0000 7ff4fdc00640 1 -- 172.21.15.71:0/339082588 learned_addr learned my addr 172.21.15.71:0/339082588 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fd200640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6c50 0x55f62f7cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fe600640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fdc00640 1 -- 172.21.15.71:0/339082588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 msgr2=0x55f62f7cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fdc00640 1 -- 172.21.15.71:0/339082588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6c50 msgr2=0x55f62f7cead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6c50 0x55f62f7cead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fdc00640 1 -- 172.21.15.71:0/339082588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62f49d2b0 con 0x55f62f7c6330 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fe600640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7ce590 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec098020 tx=0x7ff4ec097f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becdc21471cd6855 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4df400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4df400640 1 -- 172.21.15.71:0/339082588 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec09bbc0 con 0x55f62f7c6330 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e4031870 con 0x55f62f7c6330 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.394+0000 7ff4df400640 1 -- 172.21.15.71:0/339082588 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff4ec018b30 con 0x55f62f7c6330 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff4df400640 1 -- 172.21.15.71:0/339082588 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec09fec0 con 0x55f62f7c6330 2024-09-05T12:27:04.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 10 client.14970 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f62f7dff40 con 0x55f62f7c6330 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff4df400640 1 -- 172.21.15.71:0/339082588 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4031870 con 0x55f62f7c6330 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff4df400640 1 -- 172.21.15.71:0/339082588 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff4ec097910 con 0x55f62f7c6330 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 10 client.14970 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 10 client.14970 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 20 client.14970 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f7e2480 0x55f62f7e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 4 client.14970 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.395+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 --> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f62f739620 con 0x55f62f7e2480 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.396+0000 7ff4fd200640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f7e2480 0x55f62f7e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.396+0000 7ff4fd200640 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f7e2480 0x55f62f7e38f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdeb1bdbb53a7198 server_cookie=b3dd1e032448fb8d in_seq=0 out_seq=0 2024-09-05T12:27:04.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.396+0000 7ff4df400640 10 client.14970 ms_handle_connect on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.397+0000 7ff4df400640 1 -- 172.21.15.71:0/339082588 <== mds.1 v2:172.21.15.71:6838/2593934735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f62f739620 con 0x55f62f7e2480 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.397+0000 7ff4df400640 10 client.14970 handle_command_reply: tid=0 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.397+0000 7ff503254740 1 client.14970 shutdown 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.397+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 msgr2=0x55f62f7ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.397+0000 7ff503254740 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7ce590 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec098020 tx=0x7ff4ec097f50 comp rx=0 tx=0).stop 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 shutdown_connections 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f7e2480 0x55f62f7e38f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6c50 0x55f62f7cead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 --2- 172.21.15.71:0/339082588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6330 0x55f62f7ce590 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 >> 172.21.15.71:0/339082588 conn(0x55f62f7c28c0 msgr2=0x55f62f7c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff4df400640 0 client.14970 ms_handle_reset on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:04.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 shutdown_connections 2024-09-05T12:27:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 1 -- 172.21.15.71:0/339082588 wait complete. 2024-09-05T12:27:04.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.398+0000 7ff503254740 20 client.14970 trim_cache size 0 max 16384 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 Processor -- start 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 -- start start 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 0x55f62f7cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7e1c30 con 0x55f62f7c6c50 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7e2300 con 0x55f62f7c6330 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.416+0000 7ff503254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7e29d0 con 0x55f62f7d1510 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fe600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fdc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 0x55f62f7cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fe600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42044/0 (socket says 172.21.15.71:42044) 2024-09-05T12:27:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fe600640 1 -- 172.21.15.71:0/3730940770 learned_addr learned my addr 172.21.15.71:0/3730940770 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:04.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 0x55f62f7cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44900/0 (socket says 172.21.15.71:44900) 2024-09-05T12:27:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fd200640 1 -- 172.21.15.71:0/3730940770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 msgr2=0x55f62f7cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fd200640 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fd200640 1 -- 172.21.15.71:0/3730940770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 msgr2=0x55f62f7cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fd200640 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 0x55f62f7cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fd200640 1 -- 172.21.15.71:0/3730940770 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62f7c6a40 con 0x55f62f7c6c50 2024-09-05T12:27:04.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.417+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 0x55f62f7cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.418+0000 7ff4fd200640 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7cce60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4006010 tx=0x7ff4e40053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792ad1e56fcae15f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.418+0000 7ff4ea000640 1 -- 172.21.15.71:0/3730940770 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4053020 con 0x55f62f7c6c50 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.418+0000 7ff4ea000640 1 -- 172.21.15.71:0/3730940770 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4031870 con 0x55f62f7c6c50 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.418+0000 7ff4ea000640 1 -- 172.21.15.71:0/3730940770 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4040040 con 0x55f62f7c6c50 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 -- 172.21.15.71:0/3730940770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 msgr2=0x55f62f7cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7cce60 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4006010 tx=0x7ff4e40053f0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 -- 172.21.15.71:0/3730940770 shutdown_connections 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7d1510 0x55f62f7cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7cce60 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 --2- 172.21.15.71:0/3730940770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7c6330 0x55f62f7cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 -- 172.21.15.71:0/3730940770 >> 172.21.15.71:0/3730940770 conn(0x55f62f7c28c0 msgr2=0x55f62f7c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 -- 172.21.15.71:0/3730940770 shutdown_connections 2024-09-05T12:27:04.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 -- 172.21.15.71:0/3730940770 wait complete. 2024-09-05T12:27:04.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 -- start start 2024-09-05T12:27:04.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.419+0000 7ff503254740 1 Processor -- start 2024-09-05T12:27:04.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6330 0x55f62f7e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7eae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7e3770 con 0x55f62f7c6c50 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7e30a0 con 0x55f62f7d1510 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f62f7e3e40 con 0x55f62f7c6330 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff503254740 10 client.0 _ll_register_callbacks cb 0x7ff4ec009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff4fd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff4fd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7e6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60534/0 (socket says 172.21.15.71:60534) 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff4fd200640 1 -- 172.21.15.71:0/584879041 learned_addr learned my addr 172.21.15.71:0/584879041 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff4fe600640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.420+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6330 0x55f62f7e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4fd200640 1 -- 172.21.15.71:0/584879041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6330 msgr2=0x55f62f7e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4fd200640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6330 0x55f62f7e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4fd200640 1 -- 172.21.15.71:0/584879041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 msgr2=0x55f62f7eae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4fd200640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7eae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4fd200640 1 -- 172.21.15.71:0/584879041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f62f7eb520 con 0x55f62f7c6c50 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4fd200640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7e6520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4000c00 tx=0x7ff4e402ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c62b68407ee87fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4df400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4df400640 1 -- 172.21.15.71:0/584879041 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4040030 con 0x55f62f7c6c50 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4df400640 1 -- 172.21.15.71:0/584879041 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff4e4004230 con 0x55f62f7c6c50 2024-09-05T12:27:04.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e4031870 con 0x55f62f7c6c50 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.421+0000 7ff4df400640 1 -- 172.21.15.71:0/584879041 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff4e406c050 con 0x55f62f7c6c50 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 10 client.14976 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f62f7ec820 con 0x55f62f7c6c50 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff4df400640 1 -- 172.21.15.71:0/584879041 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4e4031870 con 0x55f62f7c6c50 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff4df400640 1 -- 172.21.15.71:0/584879041 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff4e403c440 con 0x55f62f7c6c50 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 10 client.14976 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 10 client.14976 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 20 client.14976 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f850e80 0x55f62f84fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 4 client.14976 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:04.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 --> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] -- command(tid 0: {"prefix": "status"}) -- 0x55f62f733b30 con 0x55f62f850e80 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.422+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f850e80 0x55f62f84fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff4fdc00640 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f850e80 0x55f62f84fb90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f05a8b5d4f80ea35 server_cookie=65841774f0315732 in_seq=0 out_seq=0 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff4df400640 10 client.14976 ms_handle_connect on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff4df400640 1 -- 172.21.15.71:0/584879041 <== mds.1 v2:172.21.15.71:6838/2593934735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f62f733b30 con 0x55f62f850e80 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff4df400640 10 client.14976 handle_command_reply: tid=0 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 client.14976 shutdown 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 msgr2=0x55f62f7e6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7e6520 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff4e4000c00 tx=0x7ff4e402ab10 comp rx=0 tx=0).stop 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 shutdown_connections 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f62f850e80 0x55f62f84fb90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62f7d1510 0x55f62f7eae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62f7c6c50 0x55f62f7e6520 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 --2- 172.21.15.71:0/584879041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62f7c6330 0x55f62f7e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 >> 172.21.15.71:0/584879041 conn(0x55f62f7c28c0 msgr2=0x55f62f7cdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:04.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.423+0000 7ff4df400640 0 client.14976 ms_handle_reset on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:04.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 shutdown_connections 2024-09-05T12:27:04.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff503254740 1 -- 172.21.15.71:0/584879041 wait complete. 2024-09-05T12:27:04.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff503254740 20 client.14976 trim_cache size 0 max 16384 2024-09-05T12:27:04.429 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 1, 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24514, 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 89.140319070000004, 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 153.67106424100001 2024-09-05T12:27:04.430 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4e00a2c00 msgr2=0x7ff4e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4e00a2c00 0x7ff4e00a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0011630 tx=0x7ff4f0009500 comp rx=0 tx=0).stop 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 msgr2=0x7ff4f819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 0x7ff4f819a6b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec00e9f0 tx=0x7ff4ec00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 shutdown_connections 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff4e00a2c00 0x7ff4e00a50c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4f8106f10 0x7ff4f819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4f8102790 0x7ff4f819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.424+0000 7ff500600640 1 --2- 172.21.15.71:0/3184890552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4f8101de0 0x7ff4f819a6b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.425+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 >> 172.21.15.71:0/3184890552 conn(0x7ff4f80fd8c0 msgr2=0x7ff4f80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.425+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 shutdown_connections 2024-09-05T12:27:04.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:04.425+0000 7ff500600640 1 -- 172.21.15.71:0/3184890552 wait complete. 2024-09-05T12:27:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:04 smithi174 ceph-mon[34313]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:04 smithi071 ceph-mon[29845]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:04 smithi123 ceph-mon[34321]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:05.323 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T12:27:05.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:05 smithi071 ceph-mon[29845]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T12:27:05.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:05 smithi123 ceph-mon[34321]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T12:27:05.690 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:05.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:05 smithi174 ceph-mon[34313]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-05T12:27:06.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 -- 172.21.15.71:0/1610091146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac074eb0 msgr2=0x7feeac112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1610091146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac074eb0 0x7feeac112650 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee9402a480 comp rx=0 tx=0).stop 2024-09-05T12:27:06.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 -- 172.21.15.71:0/1610091146 shutdown_connections 2024-09-05T12:27:06.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1610091146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac112b90 0x7feeac114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1610091146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac074eb0 0x7feeac112650 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1610091146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 -- 172.21.15.71:0/1610091146 >> 172.21.15.71:0/1610091146 conn(0x7feeac0fd8c0 msgr2=0x7feeac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:06.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 -- 172.21.15.71:0/1610091146 shutdown_connections 2024-09-05T12:27:06.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.261+0000 7feeb1e00640 1 -- 172.21.15.71:0/1610091146 wait complete. 2024-09-05T12:27:06.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.262+0000 7feeb1e00640 1 Processor -- start 2024-09-05T12:27:06.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.262+0000 7feeb1e00640 1 -- start start 2024-09-05T12:27:06.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeb1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac1a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeb1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 0x7feeac1a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeb1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac112b90 0x7feeac19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeb1e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feeac118590 con 0x7feeac112b90 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeb1e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feeac118410 con 0x7feeac074eb0 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeb1e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feeac118710 con 0x7feeac074590 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac1a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feea2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 0x7feeac1a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac1a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42068/0 (socket says 172.21.15.71:42068) 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 -- 172.21.15.71:0/1343228776 learned_addr learned my addr 172.21.15.71:0/1343228776 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feea2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 0x7feeac1a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44936/0 (socket says 172.21.15.71:44936) 2024-09-05T12:27:06.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeab400640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac112b90 0x7feeac19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 -- 172.21.15.71:0/1343228776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 msgr2=0x7feeac1a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 0x7feeac1a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 -- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac112b90 msgr2=0x7feeac19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac112b90 0x7feeac19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feeaaa00640 1 -- 172.21.15.71:0/1343228776 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeac19f050 con 0x7feeac074590 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.263+0000 7feea2000640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 0x7feeac1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:06.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.264+0000 7feeaaa00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac1a4760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e9f0 tx=0x7fee9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:06.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.266+0000 7feeb0a00640 1 -- 172.21.15.71:0/1343228776 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00cd80 con 0x7feeac074590 2024-09-05T12:27:06.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.266+0000 7feeb0a00640 1 -- 172.21.15.71:0/1343228776 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c010910 con 0x7feeac074590 2024-09-05T12:27:06.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.266+0000 7feeb0a00640 1 -- 172.21.15.71:0/1343228776 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018990 con 0x7feeac074590 2024-09-05T12:27:06.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.266+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeac19f340 con 0x7feeac074590 2024-09-05T12:27:06.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.266+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeac1ab630 con 0x7feeac074590 2024-09-05T12:27:06.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.269+0000 7feeb0a00640 1 -- 172.21.15.71:0/1343228776 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee9c010ac0 con 0x7feeac074590 2024-09-05T12:27:06.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.269+0000 7feeb49ec740 1 Processor -- start 2024-09-05T12:27:06.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.269+0000 7feeb49ec740 1 -- start start 2024-09-05T12:27:06.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc3245710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc3252900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb49ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556bc31ab0b0 con 0x556bc3245330 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb49ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556bc3152ab0 con 0x556bc3245c50 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb49ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556bc31a7070 con 0x556bc3250510 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc3252900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc3252900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42076/0 (socket says 172.21.15.71:42076) 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 -- 172.21.15.71:0/651660376 learned_addr learned my addr 172.21.15.71:0/651660376 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeab400640 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc3245710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeaaa00640 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 -- 172.21.15.71:0/651660376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 msgr2=0x556bc324ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 -- 172.21.15.71:0/651660376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 msgr2=0x556bc3245710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc3245710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feeb0a00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fee900a2c00 0x7fee900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.270+0000 7feea2000640 1 -- 172.21.15.71:0/651660376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc315f4a0 con 0x556bc3250510 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7feeb0a00640 1 -- 172.21.15.71:0/1343228776 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fee9c0cf660 con 0x7feeac074590 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7feeaaa00640 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7feeab400640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fee900a2c00 0x7fee900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7feea2000640 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc3252900 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee94005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6abdcb76f9fc6b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7fee9be00640 1 -- 172.21.15.71:0/651660376 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94035040 con 0x556bc3250510 2024-09-05T12:27:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7fee9be00640 1 -- 172.21.15.71:0/651660376 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee94004760 con 0x556bc3250510 2024-09-05T12:27:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.271+0000 7fee9be00640 1 -- 172.21.15.71:0/651660376 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee94003970 con 0x556bc3250510 2024-09-05T12:27:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 -- 172.21.15.71:0/651660376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 msgr2=0x556bc3252900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc3252900 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee94009900 tx=0x7fee94005de0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeab400640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fee900a2c00 0x7fee900a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feea4005c40 tx=0x7feea4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 -- 172.21.15.71:0/651660376 shutdown_connections 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc3252900 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 --2- 172.21.15.71:0/651660376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc3245710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 -- 172.21.15.71:0/651660376 >> 172.21.15.71:0/651660376 conn(0x556bc32418c0 msgr2=0x556bc3241ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 -- 172.21.15.71:0/651660376 shutdown_connections 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.272+0000 7feeb49ec740 1 -- 172.21.15.71:0/651660376 wait complete. 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 -- start start 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 Processor -- start 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 0x556bc324d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3250510 0x556bc324e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556bc3253060 con 0x556bc3250510 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556bc3133920 con 0x556bc3245c50 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556bc32531e0 con 0x556bc3245330 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.273+0000 7feeb49ec740 10 client.0 _ll_register_callbacks cb 0x556bc31578a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 0x556bc324d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 0x556bc324d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42092/0 (socket says 172.21.15.71:42092) 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feea2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 -- 172.21.15.71:0/3320755562 learned_addr learned my addr 172.21.15.71:0/3320755562 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeab400640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3250510 0x556bc324e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 -- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 msgr2=0x556bc324dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 -- 172.21.15.71:0/3320755562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3250510 msgr2=0x556bc324e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3250510 0x556bc324e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 -- 172.21.15.71:0/3320755562 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc2f1c2b0 con 0x556bc3245330 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feea2000640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7feeaaa00640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 0x556bc324d590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ccda0 tx=0x7fee9c09b6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc3c81355f9e883d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.274+0000 7fee98c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7fee98c00640 1 -- 172.21.15.71:0/3320755562 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c09feb0 con 0x556bc3245330 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee94004760 con 0x556bc3245330 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7fee98c00640 1 -- 172.21.15.71:0/3320755562 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fee9c0cceb0 con 0x556bc3245330 2024-09-05T12:27:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7fee98c00640 1 -- 172.21.15.71:0/3320755562 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee9c0a4770 con 0x556bc3245330 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7feeb49ec740 10 client.25039 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bc325eee0 con 0x556bc3245330 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7fee98c00640 1 -- 172.21.15.71:0/3320755562 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee94004760 con 0x556bc3245330 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7fee98c00640 1 -- 172.21.15.71:0/3320755562 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fee9c010430 con 0x556bc3245330 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7feeb49ec740 10 client.25039 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.275+0000 7feeb49ec740 10 client.25039 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.276+0000 7feeb49ec740 20 client.25039 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.276+0000 7feeb49ec740 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc3261430 0x556bc32628a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.276+0000 7feeb49ec740 4 client.25039 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.276+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bc31b8620 con 0x556bc3261430 2024-09-05T12:27:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.276+0000 7feea2000640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc3261430 0x556bc32628a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.277+0000 7fee98c00640 10 client.25039 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:06.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.276+0000 7feea2000640 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc3261430 0x556bc32628a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f5379c74599e9 server_cookie=f0bf239ee67e720b in_seq=0 out_seq=0 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.277+0000 7fee98c00640 1 -- 172.21.15.71:0/3320755562 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556bc31b8620 con 0x556bc3261430 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.277+0000 7fee98c00640 10 client.25039 handle_command_reply: tid=0 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 client.25039 shutdown 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 msgr2=0x556bc324d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 0x556bc324d590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fee9c0ccda0 tx=0x7fee9c09b6f0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 shutdown_connections 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc3261430 0x556bc32628a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3250510 0x556bc324e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245c50 0x556bc324dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 --2- 172.21.15.71:0/3320755562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245330 0x556bc324d590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7fee98c00640 0 client.25039 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 >> 172.21.15.71:0/3320755562 conn(0x556bc32418c0 msgr2=0x556bc32475f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 shutdown_connections 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 1 -- 172.21.15.71:0/3320755562 wait complete. 2024-09-05T12:27:06.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.278+0000 7feeb49ec740 20 client.25039 trim_cache size 0 max 16384 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.296+0000 7feeb49ec740 1 Processor -- start 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 -- start start 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 0x556bc324de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245c50 0x556bc324a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc324ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556bc3260be0 con 0x556bc3245c50 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556bc32612b0 con 0x556bc3245330 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeb49ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556bc3261980 con 0x556bc3250510 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feea2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245c50 0x556bc324a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 0x556bc324de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 0x556bc324de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44954/0 (socket says 172.21.15.71:44954) 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeaaa00640 1 -- 172.21.15.71:0/1814082512 learned_addr learned my addr 172.21.15.71:0/1814082512 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.297+0000 7feeab400640 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc324ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feeaaa00640 1 -- 172.21.15.71:0/1814082512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 msgr2=0x556bc324ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feeaaa00640 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc324ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feeaaa00640 1 -- 172.21.15.71:0/1814082512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245c50 msgr2=0x556bc324a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feeaaa00640 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245c50 0x556bc324a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feeaaa00640 1 -- 172.21.15.71:0/1814082512 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bc3242a50 con 0x556bc3245330 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feea2000640 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245c50 0x556bc324a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7feeaaa00640 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 0x556bc324de60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee9c098010 tx=0x7fee9c09bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc3650fcd37638 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7fee9b400640 1 -- 172.21.15.71:0/1814082512 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c09feb0 con 0x556bc3245330 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7fee9b400640 1 -- 172.21.15.71:0/1814082512 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c099eb0 con 0x556bc3245330 2024-09-05T12:27:06.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.298+0000 7fee9b400640 1 -- 172.21.15.71:0/1814082512 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c018b30 con 0x556bc3245330 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 -- 172.21.15.71:0/1814082512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 msgr2=0x556bc324de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 0x556bc324de60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee9c098010 tx=0x7fee9c09bd30 comp rx=0 tx=0).stop 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 -- 172.21.15.71:0/1814082512 shutdown_connections 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3250510 0x556bc324ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245c50 0x556bc324a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 --2- 172.21.15.71:0/1814082512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3245330 0x556bc324de60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 -- 172.21.15.71:0/1814082512 >> 172.21.15.71:0/1814082512 conn(0x556bc32418c0 msgr2=0x556bc324be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 -- 172.21.15.71:0/1814082512 shutdown_connections 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.299+0000 7feeb49ec740 1 -- 172.21.15.71:0/1814082512 wait complete. 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 -- start start 2024-09-05T12:27:06.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 Processor -- start 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc32649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245c50 0x556bc3264f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3250510 0x556bc3265450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556bc3262720 con 0x556bc3245330 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556bc3262050 con 0x556bc3250510 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556bc3262df0 con 0x556bc3245c50 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeb49ec740 10 client.0 _ll_register_callbacks cb 0x7fee9c097ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.300+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc32649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc32649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60588/0 (socket says 172.21.15.71:60588) 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 -- 172.21.15.71:0/884939381 learned_addr learned my addr 172.21.15.71:0/884939381 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feea2000640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245c50 0x556bc3264f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeab400640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3250510 0x556bc3265450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 -- 172.21.15.71:0/884939381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245c50 msgr2=0x556bc3264f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245c50 0x556bc3264f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 -- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3250510 msgr2=0x556bc3265450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3250510 0x556bc3265450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 -- 172.21.15.71:0/884939381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bc3269220 con 0x556bc3245330 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7feeaaa00640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc32649d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fee9c098530 tx=0x7fee9c09b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ba3fb2c4706c8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7fee98c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.301+0000 7fee98c00640 1 -- 172.21.15.71:0/884939381 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0ba2b0 con 0x556bc3245330 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7fee98c00640 1 -- 172.21.15.71:0/884939381 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fee9c0ba450 con 0x556bc3245330 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c099eb0 con 0x556bc3245330 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7fee98c00640 1 -- 172.21.15.71:0/884939381 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee9c018b30 con 0x556bc3245330 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 10 client.14994 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556bc3269220 con 0x556bc3245330 2024-09-05T12:27:06.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7fee98c00640 1 -- 172.21.15.71:0/884939381 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee9c099eb0 con 0x556bc3245330 2024-09-05T12:27:06.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7fee98c00640 1 -- 172.21.15.71:0/884939381 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fee9c0861b0 con 0x556bc3245330 2024-09-05T12:27:06.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 10 client.14994 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 10 client.14994 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 20 client.14994 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc32d0440 0x556bc32cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 4 client.14994 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.302+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "status"}) -- 0x556bc3260a30 con 0x556bc32d0440 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.303+0000 7feea2000640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc32d0440 0x556bc32cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.303+0000 7feea2000640 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc32d0440 0x556bc32cf150 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=771fffcd4f27dc57 server_cookie=9e780d863da22a79 in_seq=0 out_seq=0 2024-09-05T12:27:06.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.303+0000 7fee98c00640 10 client.14994 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.303+0000 7fee98c00640 1 -- 172.21.15.71:0/884939381 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556bc3260a30 con 0x556bc32d0440 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.303+0000 7fee98c00640 10 client.14994 handle_command_reply: tid=0 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.303+0000 7feeb49ec740 1 client.14994 shutdown 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 msgr2=0x556bc32649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc32649d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fee9c098530 tx=0x7fee9c09b6f0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 shutdown_connections 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556bc32d0440 0x556bc32cf150 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556bc3250510 0x556bc3265450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556bc3245c50 0x556bc3264f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 --2- 172.21.15.71:0/884939381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556bc3245330 0x556bc32649d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 >> 172.21.15.71:0/884939381 conn(0x556bc32418c0 msgr2=0x556bc325b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7fee98c00640 0 client.14994 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:06.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 shutdown_connections 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 1 -- 172.21.15.71:0/884939381 wait complete. 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.304+0000 7feeb49ec740 20 client.14994 trim_cache size 0 max 16384 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24524, 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:06.309 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 135.79591132199999, 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 149.372129427 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fee900a2c00 msgr2=0x7fee900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fee900a2c00 0x7fee900a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feea4005c40 tx=0x7feea4009500 comp rx=0 tx=0).stop 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 msgr2=0x7feeac1a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac1a4760 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00e9f0 tx=0x7fee9c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 shutdown_connections 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fee900a2c00 0x7fee900a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feeac112b90 0x7feeac19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feeac074eb0 0x7feeac1a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 --2- 172.21.15.71:0/1343228776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeac074590 0x7feeac1a4760 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 >> 172.21.15.71:0/1343228776 conn(0x7feeac0fd8c0 msgr2=0x7feeac0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 shutdown_connections 2024-09-05T12:27:06.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:06.305+0000 7feeb1e00640 1 -- 172.21.15.71:0/1343228776 wait complete. 2024-09-05T12:27:06.311 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:06 smithi071 ceph-mon[29845]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T12:27:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:06 smithi123 ceph-mon[34321]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T12:27:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:06 smithi174 ceph-mon[34313]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T12:27:07.266 INFO:tasks.cephadm:Setup complete, yielding 2024-09-05T12:27:07.267 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T12:27:07.277 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:27:07.278 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch status' 2024-09-05T12:27:07.657 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:08.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 -- 172.21.15.71:0/3641770098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 msgr2=0x7fd1a810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 --2- 172.21.15.71:0/3641770098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a810a050 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009ca0 tx=0x7fd1a002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 -- 172.21.15.71:0/3641770098 shutdown_connections 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 --2- 172.21.15.71:0/3641770098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a810a050 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 --2- 172.21.15.71:0/3641770098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 --2- 172.21.15.71:0/3641770098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 0x7fd1a8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 -- 172.21.15.71:0/3641770098 >> 172.21.15.71:0/3641770098 conn(0x7fd1a80fd390 msgr2=0x7fd1a80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 -- 172.21.15.71:0/3641770098 shutdown_connections 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.239+0000 7fd1afe00640 1 -- 172.21.15.71:0/3641770098 wait complete. 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.240+0000 7fd1afe00640 1 Processor -- start 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.240+0000 7fd1afe00640 1 -- start start 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 0x7fd1a8197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:08.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a81983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1afe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a819f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1afe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a810d860 con 0x7fd1a81037c0 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1afe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a810d6e0 con 0x7fd1a8101c00 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1afe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a810d9e0 con 0x7fd1a8102e00 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1ade00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a819f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1aca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a81983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1aca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a81983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42146/0 (socket says 172.21.15.71:42146) 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1aca00640 1 -- 172.21.15.71:0/581185998 learned_addr learned my addr 172.21.15.71:0/581185998 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.241+0000 7fd1ad400640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 0x7fd1a8197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ad400640 1 -- 172.21.15.71:0/581185998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 msgr2=0x7fd1a81983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ad400640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a81983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ad400640 1 -- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 msgr2=0x7fd1a819f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:08.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ad400640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a819f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ad400640 1 -- 172.21.15.71:0/581185998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a819fa90 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ade00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1aca00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1ad400640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 0x7fd1a8197e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8102c60 tx=0x7fd19c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c017070 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a819fd80 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd19c009c30 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00f620 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.242+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a01e0 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.243+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8108fb0 con 0x7fd1a8101c00 2024-09-05T12:27:08.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.245+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd19c01a070 con 0x7fd1a8101c00 2024-09-05T12:27:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.247+0000 7fd1a6000640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd1900a2bb0 0x7fd1900a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.247+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fd19c0cf120 con 0x7fd1a8101c00 2024-09-05T12:27:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.247+0000 7fd1aca00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd1900a2bb0 0x7fd1900a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.247+0000 7fd1aca00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd1900a2bb0 0x7fd1900a5070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8199380 tx=0x7fd194005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:08.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.249+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd19c09b670 con 0x7fd1a8101c00 2024-09-05T12:27:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:08 smithi071 ceph-mon[29845]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T12:27:08.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.446+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd1a8062630 con 0x7fd1900a2bb0 2024-09-05T12:27:08.448 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.447+0000 7fd1a6000640 1 -- 172.21.15.71:0/581185998 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd1a8062630 con 0x7fd1900a2bb0 2024-09-05T12:27:08.448 INFO:teuthology.orchestra.run.smithi071.stdout:Backend: cephadm 2024-09-05T12:27:08.448 INFO:teuthology.orchestra.run.smithi071.stdout:Available: Yes 2024-09-05T12:27:08.448 INFO:teuthology.orchestra.run.smithi071.stdout:Paused: No 2024-09-05T12:27:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd1900a2bb0 msgr2=0x7fd1900a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd1900a2bb0 0x7fd1900a5070 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8199380 tx=0x7fd194005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:08.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 msgr2=0x7fd1a8197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 0x7fd1a8197e60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8102c60 tx=0x7fd19c00dde0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 shutdown_connections 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fd1900a2bb0 0x7fd1900a5070 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1a81037c0 0x7fd1a819f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a8102e00 0x7fd1a81983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 --2- 172.21.15.71:0/581185998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a8101c00 0x7fd1a8197e60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.449+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 >> 172.21.15.71:0/581185998 conn(0x7fd1a80fd390 msgr2=0x7fd1a8106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.450+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 shutdown_connections 2024-09-05T12:27:08.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:08.450+0000 7fd1afe00640 1 -- 172.21.15.71:0/581185998 wait complete. 2024-09-05T12:27:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:08 smithi123 ceph-mon[34321]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T12:27:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:08 smithi174 ceph-mon[34313]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T12:27:09.264 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-05T12:27:09.695 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.266+0000 7fa214800640 1 -- 172.21.15.71:0/3428808982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c10e1e0 msgr2=0x7fa20c1105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:10.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 --2- 172.21.15.71:0/3428808982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c10e1e0 0x7fa20c1105d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa208009ca0 tx=0x7fa20802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 -- 172.21.15.71:0/3428808982 shutdown_connections 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 --2- 172.21.15.71:0/3428808982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c10e1e0 0x7fa20c1105d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 --2- 172.21.15.71:0/3428808982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20c074c50 0x7fa20c0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 --2- 172.21.15.71:0/3428808982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c076220 0x7fa20c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 -- 172.21.15.71:0/3428808982 >> 172.21.15.71:0/3428808982 conn(0x7fa20c0fd390 msgr2=0x7fa20c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 -- 172.21.15.71:0/3428808982 shutdown_connections 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.267+0000 7fa214800640 1 -- 172.21.15.71:0/3428808982 wait complete. 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 Processor -- start 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 -- start start 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c074c50 0x7fa20c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c076220 0x7fa20c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20c10e1e0 0x7fa20c1a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:10.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c113ea0 con 0x7fa20c076220 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c113d20 con 0x7fa20c074c50 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa214800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c114020 con 0x7fa20c10e1e0 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa211400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c076220 0x7fa20c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa211e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c074c50 0x7fa20c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.268+0000 7fa211e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c074c50 0x7fa20c19e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44990/0 (socket says 172.21.15.71:44990) 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211e00640 1 -- 172.21.15.71:0/596303265 learned_addr learned my addr 172.21.15.71:0/596303265 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa212800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20c10e1e0 0x7fa20c1a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211400640 1 -- 172.21.15.71:0/596303265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20c10e1e0 msgr2=0x7fa20c1a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211400640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20c10e1e0 0x7fa20c1a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211400640 1 -- 172.21.15.71:0/596303265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c074c50 msgr2=0x7fa20c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211400640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c074c50 0x7fa20c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211400640 1 -- 172.21.15.71:0/596303265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c1a60a0 con 0x7fa20c076220 2024-09-05T12:27:10.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.269+0000 7fa211400640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c076220 0x7fa20c19e9b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00e9f0 tx=0x7fa1fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.272+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00ccc0 con 0x7fa20c076220 2024-09-05T12:27:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.272+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc005020 con 0x7fa20c076220 2024-09-05T12:27:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.272+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0049e0 con 0x7fa20c076220 2024-09-05T12:27:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.272+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c1a6390 con 0x7fa20c076220 2024-09-05T12:27:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.273+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1a6880 con 0x7fa20c076220 2024-09-05T12:27:10.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.274+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa20c1043b0 con 0x7fa20c076220 2024-09-05T12:27:10.276 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.275+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa1fc005560 con 0x7fa20c076220 2024-09-05T12:27:10.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.277+0000 7fa206a00640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa1f40a2c30 0x7fa1f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:10.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.277+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fa1fc0cf2d0 con 0x7fa20c076220 2024-09-05T12:27:10.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.277+0000 7fa211e00640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa1f40a2c30 0x7fa1f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:10.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.278+0000 7fa211e00640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa1f40a2c30 0x7fa1f40a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa2000058c0 tx=0x7fa200005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:10.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.281+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1fc0051d0 con 0x7fa20c076220 2024-09-05T12:27:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:10 smithi071 ceph-mon[29845]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:10 smithi071 ceph-mon[29845]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T12:27:10.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.467+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa20c062630 con 0x7fa1f40a2c30 2024-09-05T12:27:10.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.479+0000 7fa206a00640 1 -- 172.21.15.71:0/596303265 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7fa20c062630 con 0x7fa1f40a2c30 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.a smithi071 running (2m) 17s ago 2m 16.7M - 19.1.1-261-ge856d6b2 1d96423d1332 351b4bf41e1a 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.b smithi123 running (2m) 19s ago 2m 26.5M - 19.1.1-261-ge856d6b2 1d96423d1332 50cf64e4c8ba 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.c smithi174 running (2m) 19s ago 2m 18.9M - 19.1.1-261-ge856d6b2 1d96423d1332 0482b6e873df 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.d smithi071 running (2m) 17s ago 2m 13.1M - 19.1.1-261-ge856d6b2 1d96423d1332 1b24c4f72981 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.e smithi123 running (2m) 19s ago 2m 18.2M - 19.1.1-261-ge856d6b2 1d96423d1332 d44c02bcb32f 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.f smithi174 running (2m) 19s ago 2m 21.7M - 19.1.1-261-ge856d6b2 1d96423d1332 7d7208420c38 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.g smithi071 running (2m) 17s ago 2m 17.3M - 19.1.1-261-ge856d6b2 1d96423d1332 f59cfe6fb680 2024-09-05T12:27:10.480 INFO:teuthology.orchestra.run.smithi071.stdout:mds.h smithi123 running (2m) 19s ago 2m 18.4M - 19.1.1-261-ge856d6b2 1d96423d1332 0e4f9bd171dd 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mds.i smithi174 running (2m) 19s ago 2m 20.3M - 19.1.1-261-ge856d6b2 1d96423d1332 6f51e998a793 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mds.j smithi071 running (3m) 17s ago 3m 13.1M - 19.1.1-261-ge856d6b2 1d96423d1332 ff9bc1b3844b 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mds.k smithi123 running (3m) 19s ago 3m 19.9M - 19.1.1-261-ge856d6b2 1d96423d1332 3da75bb26ba5 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mds.l smithi174 running (3m) 19s ago 3m 22.6M - 19.1.1-261-ge856d6b2 1d96423d1332 4708f31af45f 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.x smithi071 *:9283,8765 running (10m) 17s ago 10m 510M - 19.1.1-261-ge856d6b2 1d96423d1332 549122bed565 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.y smithi123 *:8443,8765 running (8m) 19s ago 8m 441M - 19.1.1-261-ge856d6b2 1d96423d1332 c9167a856f55 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mgr.z smithi174 *:8443,8765 running (8m) 19s ago 8m 443M - 19.1.1-261-ge856d6b2 1d96423d1332 1841ff4db577 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mon.a smithi071 running (10m) 17s ago 10m 49.4M 2048M 19.1.1-261-ge856d6b2 1d96423d1332 e2795b500320 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mon.b smithi123 running (8m) 19s ago 8m 44.1M 2048M 19.1.1-261-ge856d6b2 1d96423d1332 1dcd4e6c34fc 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:mon.c smithi174 running (9m) 19s ago 8m 41.2M 2048M 19.1.1-261-ge856d6b2 1d96423d1332 1b2c606f8355 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 smithi071 running (8m) 17s ago 8m 46.8M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 b22ff260ac7b 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 smithi123 running (7m) 19s ago 7m 55.5M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 0e1ba30d5f2f 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 smithi174 running (7m) 19s ago 7m 56.4M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 7908211a270c 2024-09-05T12:27:10.481 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 smithi071 running (6m) 17s ago 6m 45.7M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 d33033dee444 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 smithi123 running (6m) 19s ago 6m 43.6M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 b15007424ca1 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 smithi174 running (6m) 19s ago 6m 48.0M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 6316a7839e5a 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 smithi071 running (5m) 17s ago 5m 41.9M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 b1d9750bb3ed 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 smithi123 running (5m) 19s ago 5m 52.8M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 14e217548d5c 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.8 smithi174 running (4m) 19s ago 4m 53.7M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 9ba614411c8d 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.9 smithi071 running (4m) 17s ago 4m 51.0M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 3bbc3561934a 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.10 smithi123 running (3m) 19s ago 3m 48.7M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 b4324beaa073 2024-09-05T12:27:10.482 INFO:teuthology.orchestra.run.smithi071.stdout:osd.11 smithi174 running (3m) 19s ago 3m 41.4M 4272M 19.1.1-261-ge856d6b2 1d96423d1332 ff837d0ac31a 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa1f40a2c30 msgr2=0x7fa1f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa1f40a2c30 0x7fa1f40a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa2000058c0 tx=0x7fa200005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c076220 msgr2=0x7fa20c19e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c076220 0x7fa20c19e9b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00e9f0 tx=0x7fa1fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 shutdown_connections 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fa1f40a2c30 0x7fa1f40a50f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa20c10e1e0 0x7fa20c1a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa20c076220 0x7fa20c19e9b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 --2- 172.21.15.71:0/596303265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa20c074c50 0x7fa20c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 >> 172.21.15.71:0/596303265 conn(0x7fa20c0fd390 msgr2=0x7fa20c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 shutdown_connections 2024-09-05T12:27:10.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:10.481+0000 7fa214800640 1 -- 172.21.15.71:0/596303265 wait complete. 2024-09-05T12:27:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:10 smithi123 ceph-mon[34321]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:10 smithi123 ceph-mon[34321]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T12:27:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:10 smithi174 ceph-mon[34313]: from='client.25049 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:10 smithi174 ceph-mon[34313]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-05T12:27:11.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-05T12:27:11.671 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:12.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.260+0000 7f1b3fe00640 1 -- 172.21.15.71:0/725932176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 msgr2=0x7f1b3810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:12.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.260+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/725932176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 0x7f1b3810a050 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009ca0 tx=0x7f1b3002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 -- 172.21.15.71:0/725932176 shutdown_connections 2024-09-05T12:27:12.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/725932176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 0x7f1b3810a050 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/725932176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b38102e00 0x7f1b38103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/725932176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b38102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 -- 172.21.15.71:0/725932176 >> 172.21.15.71:0/725932176 conn(0x7f1b380fd390 msgr2=0x7f1b380ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 -- 172.21.15.71:0/725932176 shutdown_connections 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.261+0000 7f1b3fe00640 1 -- 172.21.15.71:0/725932176 wait complete. 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 Processor -- start 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 -- start start 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b3819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b38102e00 0x7f1b3819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 0x7f1b381a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3810d8b0 con 0x7f1b381037c0 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3810d730 con 0x7f1b38101c00 2024-09-05T12:27:12.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3810da30 con 0x7f1b38102e00 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b3819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b3819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45000/0 (socket says 172.21.15.71:45000) 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3d400640 1 -- 172.21.15.71:0/1965704266 learned_addr learned my addr 172.21.15.71:0/1965704266 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.262+0000 7f1b3de00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 0x7f1b381a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3ca00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b38102e00 0x7f1b3819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3d400640 1 -- 172.21.15.71:0/1965704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b38102e00 msgr2=0x7f1b3819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3d400640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b38102e00 0x7f1b3819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3d400640 1 -- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 msgr2=0x7f1b381a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3d400640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 0x7f1b381a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3d400640 1 -- 172.21.15.71:0/1965704266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b381a1ca0 con 0x7f1b38101c00 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3d400640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b3819a070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9c0 tx=0x7f1b2c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00cd80 con 0x7f1b38101c00 2024-09-05T12:27:12.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b2c0035f0 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b381a1f90 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.263+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c010690 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.264+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b381a23e0 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.264+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b381a26a0 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.267+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b2c0037a0 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.268+0000 7f1b36000640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b200a2bb0 0x7f1b200a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.268+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f1b2c0cf8d0 con 0x7f1b38101c00 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.268+0000 7f1b3ca00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b200a2bb0 0x7f1b200a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.269+0000 7f1b3ca00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b200a2bb0 0x7f1b200a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1b3819b590 tx=0x7f1b24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:12.295 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.270+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1b2c09c500 con 0x7f1b38101c00 2024-09-05T12:27:12.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:12 smithi071 ceph-mon[29845]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:12.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:12 smithi071 ceph-mon[29845]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-05T12:27:12.456 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.455+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1b381a2870 con 0x7f1b200a2bb0 2024-09-05T12:27:12.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.461+0000 7f1b36000640 1 -- 172.21.15.71:0/1965704266 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1b381a2870 con 0x7f1b200a2bb0 2024-09-05T12:27:12.462 INFO:teuthology.orchestra.run.smithi071.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-05T12:27:12.462 INFO:teuthology.orchestra.run.smithi071.stdout:mds.all 12/12 21s ago 3m smithi071=a;smithi071=d;smithi071=g;smithi071=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-05T12:27:12.462 INFO:teuthology.orchestra.run.smithi071.stdout:mgr 3/3 21s ago 8m smithi071=x;smithi123=y;smithi174=z;count:3 2024-09-05T12:27:12.462 INFO:teuthology.orchestra.run.smithi071.stdout:mon 3/3 21s ago 9m smithi071:172.21.15.71=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c;count:3 2024-09-05T12:27:12.462 INFO:teuthology.orchestra.run.smithi071.stdout:osd 12 21s ago - 2024-09-05T12:27:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b200a2bb0 msgr2=0x7f1b200a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b200a2bb0 0x7f1b200a5070 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1b3819b590 tx=0x7f1b24005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:12.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 msgr2=0x7f1b3819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b3819a070 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9c0 tx=0x7f1b2c00ee80 comp rx=0 tx=0).stop 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 shutdown_connections 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f1b200a2bb0 0x7f1b200a5070 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1b381037c0 0x7f1b381a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b38102e00 0x7f1b3819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 --2- 172.21.15.71:0/1965704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b38101c00 0x7f1b3819a070 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 >> 172.21.15.71:0/1965704266 conn(0x7f1b380fd390 msgr2=0x7f1b38074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.463+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 shutdown_connections 2024-09-05T12:27:12.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:12.464+0000 7f1b3fe00640 1 -- 172.21.15.71:0/1965704266 wait complete. 2024-09-05T12:27:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:12 smithi123 ceph-mon[34321]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:12 smithi123 ceph-mon[34321]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-05T12:27:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:12 smithi174 ceph-mon[34313]: from='client.15006 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:12 smithi174 ceph-mon[34313]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-05T12:27:13.268 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-05T12:27:13.634 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 -- 172.21.15.71:0/905818090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154103c50 msgr2=0x7f2154104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 --2- 172.21.15.71:0/905818090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154103c50 0x7f2154104050 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2144009900 tx=0x7f214402a410 comp rx=0 tx=0).stop 2024-09-05T12:27:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 -- 172.21.15.71:0/905818090 shutdown_connections 2024-09-05T12:27:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 --2- 172.21.15.71:0/905818090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154105810 0x7f215410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 --2- 172.21.15.71:0/905818090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154104e50 0x7f21541052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 --2- 172.21.15.71:0/905818090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154103c50 0x7f2154104050 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 -- 172.21.15.71:0/905818090 >> 172.21.15.71:0/905818090 conn(0x7f21540ff3e0 msgr2=0x7f2154101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:14.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 -- 172.21.15.71:0/905818090 shutdown_connections 2024-09-05T12:27:14.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.238+0000 7f2158a00640 1 -- 172.21.15.71:0/905818090 wait complete. 2024-09-05T12:27:14.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 Processor -- start 2024-09-05T12:27:14.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 -- start start 2024-09-05T12:27:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154103c50 0x7f2154199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154104e50 0x7f215419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154105810 0x7f21541a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f215410f8d0 con 0x7f2154105810 2024-09-05T12:27:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f215410f750 con 0x7f2154104e50 2024-09-05T12:27:14.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2158a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f215410fa50 con 0x7f2154103c50 2024-09-05T12:27:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2153400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154103c50 0x7f2154199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f214be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154104e50 0x7f215419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2153e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154105810 0x7f21541a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2153400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154103c50 0x7f2154199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37868/0 (socket says 172.21.15.71:37868) 2024-09-05T12:27:14.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.240+0000 7f2153400640 1 -- 172.21.15.71:0/2075570865 learned_addr learned my addr 172.21.15.71:0/2075570865 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f214be00640 1 -- 172.21.15.71:0/2075570865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154103c50 msgr2=0x7f2154199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f214be00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154103c50 0x7f2154199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f214be00640 1 -- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154105810 msgr2=0x7f21541a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f214be00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154105810 0x7f21541a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f214be00640 1 -- 172.21.15.71:0/2075570865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21541a1b60 con 0x7f2154104e50 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f2153e00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154105810 0x7f21541a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:14.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.241+0000 7f214be00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154104e50 0x7f215419a470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e9f0 tx=0x7f213c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.242+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c00ccc0 con 0x7f2154104e50 2024-09-05T12:27:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.242+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f213c005020 con 0x7f2154104e50 2024-09-05T12:27:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.242+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21541a1e50 con 0x7f2154104e50 2024-09-05T12:27:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.242+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c0049e0 con 0x7f2154104e50 2024-09-05T12:27:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.242+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21541a22b0 con 0x7f2154104e50 2024-09-05T12:27:14.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.243+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f215410b0f0 con 0x7f2154104e50 2024-09-05T12:27:14.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.245+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f213c005560 con 0x7f2154104e50 2024-09-05T12:27:14.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.246+0000 7f2150c00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21380a2bb0 0x7f21380a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:14.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.246+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f213c0cf2a0 con 0x7f2154104e50 2024-09-05T12:27:14.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.246+0000 7f2153400640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21380a2bb0 0x7f21380a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:14.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.247+0000 7f2153400640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21380a2bb0 0x7f21380a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2144000c00 tx=0x7f21440046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:14.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.248+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f213c0051d0 con 0x7f2154104e50 2024-09-05T12:27:14.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:14 smithi071 ceph-mon[29845]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:14.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:14 smithi071 ceph-mon[29845]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T12:27:14.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.434+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f2154062630 con 0x7f21380a2bb0 2024-09-05T12:27:14.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.435+0000 7f2150c00640 1 -- 172.21.15.71:0/2075570865 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f2154062630 con 0x7f21380a2bb0 2024-09-05T12:27:14.436 INFO:teuthology.orchestra.run.smithi071.stdout:HOST ADDR LABELS STATUS 2024-09-05T12:27:14.436 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 172.21.15.71 2024-09-05T12:27:14.437 INFO:teuthology.orchestra.run.smithi071.stdout:smithi123 172.21.15.123 2024-09-05T12:27:14.437 INFO:teuthology.orchestra.run.smithi071.stdout:smithi174 172.21.15.174 2024-09-05T12:27:14.437 INFO:teuthology.orchestra.run.smithi071.stdout:3 hosts in cluster 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21380a2bb0 msgr2=0x7f21380a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21380a2bb0 0x7f21380a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2144000c00 tx=0x7f21440046c0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154104e50 msgr2=0x7f215419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154104e50 0x7f215419a470 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f213c00e9f0 tx=0x7f213c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 shutdown_connections 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f21380a2bb0 0x7f21380a5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2154105810 0x7f21541a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2154104e50 0x7f215419a470 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 --2- 172.21.15.71:0/2075570865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2154103c50 0x7f2154199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 >> 172.21.15.71:0/2075570865 conn(0x7f21540ff3e0 msgr2=0x7f21541082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 shutdown_connections 2024-09-05T12:27:14.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:14.438+0000 7f2158a00640 1 -- 172.21.15.71:0/2075570865 wait complete. 2024-09-05T12:27:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:14 smithi123 ceph-mon[34321]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:14.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:14 smithi123 ceph-mon[34321]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T12:27:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:14 smithi174 ceph-mon[34313]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:14 smithi174 ceph-mon[34313]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-09-05T12:27:15.229 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-05T12:27:15.593 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 -- 172.21.15.71:0/58153233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 msgr2=0x7f1758074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 --2- 172.21.15.71:0/58153233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 0x7f1758074600 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1748009900 tx=0x7f174802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 -- 172.21.15.71:0/58153233 shutdown_connections 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 --2- 172.21.15.71:0/58153233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17581098b0 0x7f175810bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 --2- 172.21.15.71:0/58153233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1758074b40 0x7f1758074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 --2- 172.21.15.71:0/58153233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 0x7f1758074600 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.167+0000 7f175d800640 1 -- 172.21.15.71:0/58153233 >> 172.21.15.71:0/58153233 conn(0x7f17580fd1f0 msgr2=0x7f17580ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.168+0000 7f175d800640 1 -- 172.21.15.71:0/58153233 shutdown_connections 2024-09-05T12:27:16.169 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.168+0000 7f175d800640 1 -- 172.21.15.71:0/58153233 wait complete. 2024-09-05T12:27:16.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 Processor -- start 2024-09-05T12:27:16.170 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 -- start start 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 0x7f1758199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 0x7f175819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17581098b0 0x7f17581a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f175810f8b0 con 0x7f17580761a0 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f175810f730 con 0x7f17581098b0 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f175d800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f175810fa30 con 0x7f1758074b40 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f1757e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 0x7f1758199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f1757e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 0x7f1758199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37876/0 (socket says 172.21.15.71:37876) 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f1757e00640 1 -- 172.21.15.71:0/1244727119 learned_addr learned my addr 172.21.15.71:0/1244727119 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.169+0000 7f1757400640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 0x7f175819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f175ce00640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17581098b0 0x7f17581a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f1757e00640 1 -- 172.21.15.71:0/1244727119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17581098b0 msgr2=0x7f17581a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:16.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f1757e00640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17581098b0 0x7f17581a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f1757e00640 1 -- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 msgr2=0x7f175819a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:16.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f1757e00640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 0x7f175819a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f1757e00640 1 -- 172.21.15.71:0/1244727119 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17581a1ad0 con 0x7f1758074b40 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.170+0000 7f1757e00640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 0x7f1758199ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1748009230 tx=0x7f17480042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.171+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174803f030 con 0x7f1758074b40 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.171+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17581a1d60 con 0x7f1758074b40 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.171+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1748003ce0 con 0x7f1758074b40 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.171+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748035070 con 0x7f1758074b40 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.171+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581a2180 con 0x7f1758074b40 2024-09-05T12:27:16.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.172+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17581086c0 con 0x7f1758074b40 2024-09-05T12:27:16.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.174+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1748042030 con 0x7f1758074b40 2024-09-05T12:27:16.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.175+0000 7f1754c00640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f173c0a2bb0 0x7f173c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:16.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.175+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f17480efc40 con 0x7f1758074b40 2024-09-05T12:27:16.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.175+0000 7f1757400640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f173c0a2bb0 0x7f173c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:16.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.176+0000 7f1757400640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f173c0a2bb0 0x7f173c0a5070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f175819b3c0 tx=0x7f1744005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:16.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.178+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17480bc190 con 0x7f1758074b40 2024-09-05T12:27:16.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.368+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 --> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f1758062630 con 0x7f173c0a2bb0 2024-09-05T12:27:16.369 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:16 smithi071 ceph-mon[29845]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:16.369 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:16 smithi071 ceph-mon[29845]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T12:27:16.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.370+0000 7f1754c00640 1 -- 172.21.15.71:0/1244727119 <== mgr.14150 v2:172.21.15.71:6800/2050574497 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f1758062630 con 0x7f173c0a2bb0 2024-09-05T12:27:16.371 INFO:teuthology.orchestra.run.smithi071.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-05T12:27:16.371 INFO:teuthology.orchestra.run.smithi071.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-09-05T12:27:16.371 INFO:teuthology.orchestra.run.smithi071.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T12:27:16.371 INFO:teuthology.orchestra.run.smithi071.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f173c0a2bb0 msgr2=0x7f173c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f173c0a2bb0 0x7f173c0a5070 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f175819b3c0 tx=0x7f1744005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 msgr2=0x7f1758199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 0x7f1758199ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1748009230 tx=0x7f17480042f0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 shutdown_connections 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f173c0a2bb0 0x7f173c0a5070 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17581098b0 0x7f17581a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f17580761a0 0x7f175819a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 --2- 172.21.15.71:0/1244727119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1758074b40 0x7f1758199ea0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 >> 172.21.15.71:0/1244727119 conn(0x7f17580fd1f0 msgr2=0x7f1758105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:16.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 shutdown_connections 2024-09-05T12:27:16.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:16.374+0000 7f175d800640 1 -- 172.21.15.71:0/1244727119 wait complete. 2024-09-05T12:27:16.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:16 smithi174 ceph-mon[34313]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:16.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:16 smithi174 ceph-mon[34313]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T12:27:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:16 smithi123 ceph-mon[34321]: from='client.25067 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:16 smithi123 ceph-mon[34321]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 426 B/s wr, 12 op/s 2024-09-05T12:27:17.196 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-05T12:27:17.205 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:27:17.205 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-05T12:27:17.585 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:18.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 -- 172.21.15.71:0/4052405249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c108400 msgr2=0x7f075c108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:18.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 --2- 172.21.15.71:0/4052405249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c108400 0x7f075c108800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f074c009900 tx=0x7f074c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 -- 172.21.15.71:0/4052405249 shutdown_connections 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 --2- 172.21.15.71:0/4052405249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f075c109d80 0x7f075c1106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 --2- 172.21.15.71:0/4052405249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c1093c0 0x7f075c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 --2- 172.21.15.71:0/4052405249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c108400 0x7f075c108800 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 -- 172.21.15.71:0/4052405249 >> 172.21.15.71:0/4052405249 conn(0x7f075c0773d0 msgr2=0x7f075c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 -- 172.21.15.71:0/4052405249 shutdown_connections 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.262+0000 7f0761e00640 1 -- 172.21.15.71:0/4052405249 wait complete. 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.263+0000 7f0761e00640 1 Processor -- start 2024-09-05T12:27:18.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 -- start start 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c108400 0x7f075c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 0x7f075c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f075c109d80 0x7f075c1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f075c113ea0 con 0x7f075c108400 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f075c113d20 con 0x7f075c109d80 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f0761e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f075c114020 con 0x7f075c1093c0 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f075a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 0x7f075c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f075aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c108400 0x7f075c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:18.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f075a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 0x7f075c19ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37894/0 (socket says 172.21.15.71:37894) 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f075a000640 1 -- 172.21.15.71:0/1697899954 learned_addr learned my addr 172.21.15.71:0/1697899954 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.264+0000 7f075b400640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f075c109d80 0x7f075c1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f075a000640 1 -- 172.21.15.71:0/1697899954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f075c109d80 msgr2=0x7f075c1a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f075a000640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f075c109d80 0x7f075c1a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f075a000640 1 -- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c108400 msgr2=0x7f075c19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f075a000640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c108400 0x7f075c19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f075a000640 1 -- 172.21.15.71:0/1697899954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a6100 con 0x7f075c1093c0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f075a000640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 0x7f075c19ea10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f074800e9c0 tx=0x7f074800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074800cd80 con 0x7f075c1093c0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a63f0 con 0x7f075c1093c0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07480035f0 con 0x7f075c1093c0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748010690 con 0x7f075c1093c0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.265+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a6930 con 0x7f075c1093c0 2024-09-05T12:27:18.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.266+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c10f690 con 0x7f075c1093c0 2024-09-05T12:27:18.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.268+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f07480037a0 con 0x7f075c1093c0 2024-09-05T12:27:18.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.269+0000 7f0760a00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f07400a2bb0 0x7f07400a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:18.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.269+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f07480cf790 con 0x7f075c1093c0 2024-09-05T12:27:18.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.270+0000 7f075aa00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f07400a2bb0 0x7f07400a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:18.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.270+0000 7f075aa00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f07400a2bb0 0x7f07400a5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f074c0314e0 tx=0x7f074c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:18.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.273+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f074809c330 con 0x7f075c1093c0 2024-09-05T12:27:18.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.485+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f075c02d7c0 con 0x7f075c1093c0 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.486+0000 7f0760a00640 1 -- 172.21.15.71:0/1697899954 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3126 (secure 0 0 0) 0x7f074809bc30 con 0x7f075c1093c0 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:e29 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:btime 2024-09-05T12:25:40:321146+0000 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:legacy client fscid: 1 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:Filesystem 'cephfs' (1) 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:fs_name cephfs 2024-09-05T12:27:18.487 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 29 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-05T12:24:49.631865+0000 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-09-05T12:25:40.307362+0000 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:tableserver 0 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:root 0 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:session_timeout 300 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:session_autoclose 300 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:max_file_size 1099511627776 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:max_xattr_size 65536 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:required_client_features {} 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure 0 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:last_failure_osd_epoch 0 2024-09-05T12:27:18.488 INFO:teuthology.orchestra.run.smithi071.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:max_mds 5 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:in 0,1,2,3,4 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:up {0=24524,1=24514,2=24502,3=24490,4=24485} 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:failed 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:damaged 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:stopped 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:data_pools [3,4] 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:metadata_pool 2 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:inline_data disabled 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:balancer 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:bal_rank_mask -1 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:standby_count_wanted 0 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:qdb_cluster leader: 24524 members: 24485,24524,24490,24502,24514 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.b{0:24524} state up:active seq 5 addr [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.489 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.f{0:24478} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/626911382,v1:172.21.15.174:6835/626911382] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.g{1:24514} state up:active seq 18 addr [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.e{1:24464} state up:standby-replay seq 1 addr [v2:172.21.15.123:6834/2774200097,v1:172.21.15.123:6835/2774200097] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.i{2:24502} state up:active seq 19 addr [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.k{2:24434} state up:standby-replay seq 1 addr [v2:172.21.15.123:6832/1436525377,v1:172.21.15.123:6833/1436525377] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.c{3:24490} state up:active seq 20 addr [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.l{3:24433} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/679053656,v1:172.21.15.174:6833/679053656] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.h{4:24485} state up:active seq 20 addr [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.a{4:14469} state up:standby-replay seq 1 addr [v2:172.21.15.71:6840/991266062,v1:172.21.15.71:6841/991266062] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.490 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:18.491 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:18.491 INFO:teuthology.orchestra.run.smithi071.stdout:Standby daemons: 2024-09-05T12:27:18.491 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:18.491 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.j{-1:14418} state up:standby seq 1 addr [v2:172.21.15.71:6834/83855172,v1:172.21.15.71:6835/83855172] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.491 INFO:teuthology.orchestra.run.smithi071.stdout:[mds.d{-1:14430} state up:standby seq 1 addr [v2:172.21.15.71:6836/2997614259,v1:172.21.15.71:6837/2997614259] compat {c=[1],r=[1],i=[1fff]}] 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f07400a2bb0 msgr2=0x7f07400a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f07400a2bb0 0x7f07400a5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f074c0314e0 tx=0x7f074c000bc0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 msgr2=0x7f075c19ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 0x7f075c19ea10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f074800e9c0 tx=0x7f074800ee80 comp rx=0 tx=0).stop 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 shutdown_connections 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f07400a2bb0 0x7f07400a5070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f075c109d80 0x7f075c1a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f075c1093c0 0x7f075c19ea10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 --2- 172.21.15.71:0/1697899954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f075c108400 0x7f075c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 >> 172.21.15.71:0/1697899954 conn(0x7f075c0773d0 msgr2=0x7f075c10c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:18.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 shutdown_connections 2024-09-05T12:27:18.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:18.488+0000 7f0761e00640 1 -- 172.21.15.71:0/1697899954 wait complete. 2024-09-05T12:27:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:18 smithi071 ceph-mon[29845]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:18 smithi071 ceph-mon[29845]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:18 smithi123 ceph-mon[34321]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:18 smithi123 ceph-mon[34321]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:18 smithi174 ceph-mon[34313]: from='client.25075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:18 smithi174 ceph-mon[34313]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:19.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-05T12:27:19.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:19 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1697899954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T12:27:19.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1697899954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T12:27:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:19 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1697899954' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-05T12:27:19.845 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:20.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 -- 172.21.15.71:0/2390630038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0240102e00 msgr2=0x7f0240103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:20.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 --2- 172.21.15.71:0/2390630038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0240102e00 0x7f0240103280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0228009900 tx=0x7f022802a410 comp rx=0 tx=0).stop 2024-09-05T12:27:20.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 -- 172.21.15.71:0/2390630038 shutdown_connections 2024-09-05T12:27:20.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 --2- 172.21.15.71:0/2390630038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f02401037c0 0x7f024010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 --2- 172.21.15.71:0/2390630038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0240102e00 0x7f0240103280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 --2- 172.21.15.71:0/2390630038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240101c00 0x7f0240102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 -- 172.21.15.71:0/2390630038 >> 172.21.15.71:0/2390630038 conn(0x7f02400fd390 msgr2=0x7f02400ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:20.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 -- 172.21.15.71:0/2390630038 shutdown_connections 2024-09-05T12:27:20.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.481+0000 7f0246000640 1 -- 172.21.15.71:0/2390630038 wait complete. 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.482+0000 7f0246000640 1 Processor -- start 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.482+0000 7f0246000640 1 -- start start 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0246000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0240101c00 0x7f024019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0246000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 0x7f024019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0246000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02401037c0 0x7f02401a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0246000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f024010d8b0 con 0x7f0240101c00 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0246000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f024010d730 con 0x7f02401037c0 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0246000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f024010da30 con 0x7f0240102e00 2024-09-05T12:27:20.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f023f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0240101c00 0x7f024019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:20.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0237e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 0x7f024019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:20.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0237e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 0x7f024019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37912/0 (socket says 172.21.15.71:37912) 2024-09-05T12:27:20.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0237e00640 1 -- 172.21.15.71:0/3914599148 learned_addr learned my addr 172.21.15.71:0/3914599148 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:20.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f023fe00640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02401037c0 0x7f02401a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:20.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.483+0000 7f0237e00640 1 -- 172.21.15.71:0/3914599148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02401037c0 msgr2=0x7f02401a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0237e00640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02401037c0 0x7f02401a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0237e00640 1 -- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0240101c00 msgr2=0x7f024019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0237e00640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0240101c00 0x7f024019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0237e00640 1 -- 172.21.15.71:0/3914599148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02401a1d00 con 0x7f0240102e00 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0237e00640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 0x7f024019a610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0228031440 tx=0x7f0228032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022803f030 con 0x7f0240102e00 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02401a1f90 con 0x7f0240102e00 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.484+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02401a23f0 con 0x7f0240102e00 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.485+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0228003ce0 con 0x7f0240102e00 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.485+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0228049060 con 0x7f0240102e00 2024-09-05T12:27:20.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.485+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021c0052c0 con 0x7f0240102e00 2024-09-05T12:27:20.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.488+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0228042030 con 0x7f0240102e00 2024-09-05T12:27:20.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.489+0000 7f023cc00640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02240a2c30 0x7f02240a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:20.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.489+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f02280ef320 con 0x7f0240102e00 2024-09-05T12:27:20.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.489+0000 7f023f400640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02240a2c30 0x7f02240a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:20.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.490+0000 7f023f400640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02240a2c30 0x7f02240a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0240102c60 tx=0x7f0230005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:20.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.491+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02280bb870 con 0x7f0240102e00 2024-09-05T12:27:20.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:20 smithi071 ceph-mon[29845]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-05T12:27:20.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.673+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f021c005880 con 0x7f0240102e00 2024-09-05T12:27:20.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.675+0000 7f023cc00640 1 -- 172.21.15.71:0/3914599148 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5600 (secure 0 0 0) 0x7f02280bfb70 con 0x7f0240102e00 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:epoch 94 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:created 2024-09-05T12:16:33.931712+0000 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:modified 2024-09-05T12:25:34.259275+0000 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:crush_version 27 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:full_ratio 0.95 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:backfillfull_ratio 0.9 2024-09-05T12:27:20.676 INFO:teuthology.orchestra.run.smithi071.stdout:nearfull_ratio 0.85 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:require_min_compat_client luminous 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:min_compat_client luminous 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:require_osd_release squid 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:stretch_mode_enabled false 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:max_osd 12 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6802/245899056,v1:172.21.15.71:6803/245899056] [v2:172.21.15.71:6804/245899056,v1:172.21.15.71:6805/245899056] exists,up 43d56d11-d83e-4c6b-be5b-80824118d6a2 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/1018377338,v1:172.21.15.123:6801/1018377338] [v2:172.21.15.123:6802/1018377338,v1:172.21.15.123:6803/1018377338] exists,up 4234dbed-6c5c-4abf-892a-d13fc22b44b8 2024-09-05T12:27:20.677 INFO:teuthology.orchestra.run.smithi071.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/1307000891,v1:172.21.15.174:6801/1307000891] [v2:172.21.15.174:6802/1307000891,v1:172.21.15.174:6803/1307000891] exists,up c1b0236e-25dc-47d7-b952-edec01683c2e 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.3 up in weight 1 up_from 25 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6810/3336671100,v1:172.21.15.71:6811/3336671100] [v2:172.21.15.71:6812/3336671100,v1:172.21.15.71:6813/3336671100] exists,up 9b33ca22-8dc7-41ca-8cd6-44d700c3d3da 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.4 up in weight 1 up_from 32 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/1661996031,v1:172.21.15.123:6809/1661996031] [v2:172.21.15.123:6810/1661996031,v1:172.21.15.123:6811/1661996031] exists,up f983652a-0773-4d6d-9cac-5a0f4602a729 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.5 up in weight 1 up_from 37 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/2379389937,v1:172.21.15.174:6809/2379389937] [v2:172.21.15.174:6810/2379389937,v1:172.21.15.174:6811/2379389937] exists,up 379b7d31-7108-49f2-a223-93c1214c9460 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.6 up in weight 1 up_from 45 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6818/1490057676,v1:172.21.15.71:6819/1490057676] [v2:172.21.15.71:6820/1490057676,v1:172.21.15.71:6821/1490057676] exists,up 6ac35b8a-e077-44a6-91ea-1e5efd56d8b8 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.7 up in weight 1 up_from 50 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/83143340,v1:172.21.15.123:6817/83143340] [v2:172.21.15.123:6818/83143340,v1:172.21.15.123:6819/83143340] exists,up 43640d60-2b1a-44c8-8be5-e1540265c057 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.8 up in weight 1 up_from 58 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/2584468270,v1:172.21.15.174:6817/2584468270] [v2:172.21.15.174:6818/2584468270,v1:172.21.15.174:6819/2584468270] exists,up 8a331c8c-f658-4b3c-b1a8-25199643e7d1 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.9 up in weight 1 up_from 66 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6826/4063696611,v1:172.21.15.71:6827/4063696611] [v2:172.21.15.71:6828/4063696611,v1:172.21.15.71:6829/4063696611] exists,up 9fcc4a9e-318d-482f-babb-6ad8e5ff5790 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.10 up in weight 1 up_from 71 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/256837648,v1:172.21.15.123:6825/256837648] [v2:172.21.15.123:6826/256837648,v1:172.21.15.123:6827/256837648] exists,up cbae3521-9c6b-45e7-9912-2afebeecdf3b 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:osd.11 up in weight 1 up_from 78 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/1335368098,v1:172.21.15.174:6825/1335368098] [v2:172.21.15.174:6826/1335368098,v1:172.21.15.174:6827/1335368098] exists,up 3749498e-3656-4d1e-aa2c-40b565b46488 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 2.a [2,11] 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 2.f [8,11] 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.10 [5,9] 2024-09-05T12:27:20.678 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.23 [4,8] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.24 [4,9] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 3.36 [4,9] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.7 [5,8] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.c [9,10] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.10 [9,8] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:pg_upmap_items 4.14 [9,8] 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/1492294353 expires 2024-09-06T12:17:33.045867+0000 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6801/3648851306 expires 2024-09-06T12:17:33.045867+0000 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6800/3648851306 expires 2024-09-06T12:17:33.045867+0000 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6801/1559694750 expires 2024-09-06T12:17:04.816418+0000 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/2578458480 expires 2024-09-06T12:17:04.816418+0000 2024-09-05T12:27:20.679 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/3836892613 expires 2024-09-06T12:17:33.045867+0000 2024-09-05T12:27:20.680 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:6800/1559694750 expires 2024-09-06T12:17:04.816418+0000 2024-09-05T12:27:20.680 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/4092691810 expires 2024-09-06T12:17:04.816418+0000 2024-09-05T12:27:20.680 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/3671256491 expires 2024-09-06T12:17:33.045867+0000 2024-09-05T12:27:20.680 INFO:teuthology.orchestra.run.smithi071.stdout:blocklist 172.21.15.71:0/4234071631 expires 2024-09-06T12:17:04.816418+0000 2024-09-05T12:27:20.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:20 smithi123 ceph-mon[34321]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-05T12:27:20.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02240a2c30 msgr2=0x7f02240a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:20.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02240a2c30 0x7f02240a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0240102c60 tx=0x7f0230005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:20.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 msgr2=0x7f024019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:20.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 0x7f024019a610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0228031440 tx=0x7f0228032ce0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 shutdown_connections 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f02240a2c30 0x7f02240a50f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02401037c0 0x7f02401a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0240102e00 0x7f024019a610 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 --2- 172.21.15.71:0/3914599148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0240101c00 0x7f024019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.677+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 >> 172.21.15.71:0/3914599148 conn(0x7f02400fd390 msgr2=0x7f0240074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.678+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 shutdown_connections 2024-09-05T12:27:20.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:20.678+0000 7f0246000640 1 -- 172.21.15.71:0/3914599148 wait complete. 2024-09-05T12:27:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:20 smithi174 ceph-mon[34313]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-09-05T12:27:21.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:21 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3914599148' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T12:27:21.505 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-05T12:27:21.517 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:27:21.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:21 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3914599148' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T12:27:21.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:21 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3914599148' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-05T12:27:21.885 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:22.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 -- 172.21.15.71:0/4005321918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 msgr2=0x7f70e4112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:22.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 --2- 172.21.15.71:0/4005321918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e4112650 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f70cc009900 tx=0x7f70cc02a480 comp rx=0 tx=0).stop 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 -- 172.21.15.71:0/4005321918 shutdown_connections 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 --2- 172.21.15.71:0/4005321918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70e4112b90 0x7f70e4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 --2- 172.21.15.71:0/4005321918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e4112650 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 --2- 172.21.15.71:0/4005321918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e4074590 0x7f70e4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.506+0000 7f70eb800640 1 -- 172.21.15.71:0/4005321918 >> 172.21.15.71:0/4005321918 conn(0x7f70e40fd8c0 msgr2=0x7f70e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.507+0000 7f70eb800640 1 -- 172.21.15.71:0/4005321918 shutdown_connections 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.507+0000 7f70eb800640 1 -- 172.21.15.71:0/4005321918 wait complete. 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.507+0000 7f70eb800640 1 Processor -- start 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 -- start start 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70e4074590 0x7f70e41a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e41a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e4112b90 0x7f70e419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:22.565 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4118590 con 0x7f70e4074eb0 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4118410 con 0x7f70e4112b90 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70eb800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70e4118710 con 0x7f70e4074590 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e41a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70e8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70e4074590 0x7f70e41a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e41a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50404/0 (socket says 172.21.15.71:50404) 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70dbe00640 1 -- 172.21.15.71:0/596902944 learned_addr learned my addr 172.21.15.71:0/596902944 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.508+0000 7f70e9800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e4112b90 0x7f70e419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70dbe00640 1 -- 172.21.15.71:0/596902944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70e4074590 msgr2=0x7f70e41a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70dbe00640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70e4074590 0x7f70e41a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70dbe00640 1 -- 172.21.15.71:0/596902944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e4112b90 msgr2=0x7f70e419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70dbe00640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e4112b90 0x7f70e419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70dbe00640 1 -- 172.21.15.71:0/596902944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70e419f000 con 0x7f70e4074eb0 2024-09-05T12:27:22.566 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70dbe00640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e41a4c50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004770 tx=0x7f70cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc03f070 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70cc031ca0 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70e419f290 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc035030 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.509+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70e41ab5e0 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.510+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70e4074970 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.512+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70cc042070 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.513+0000 7f70e2000640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70c80a2b80 0x7f70c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.514+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f70cc0efaa0 con 0x7f70e4074eb0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.514+0000 7f70e8e00640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70c80a2b80 0x7f70c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.515+0000 7f70e8e00640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70c80a2b80 0x7f70c80a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e419fa10 tx=0x7f70d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:22.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.518+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70cc0bbff0 con 0x7f70e4074eb0 2024-09-05T12:27:22.568 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:22 smithi071 ceph-mon[29845]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:22 smithi123 ceph-mon[34321]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:22.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.734+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70e4105000 con 0x7f70e4074eb0 2024-09-05T12:27:22.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.735+0000 7f70e2000640 1 -- 172.21.15.71:0/596902944 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11968 (secure 0 0 0) 0x7f70cc0c02f0 con 0x7f70e4074eb0 2024-09-05T12:27:22.736 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:22.736 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"btime":"2024-09-05T12:25:40:321146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:40.307362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:27:22.737 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:27:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70c80a2b80 msgr2=0x7f70c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70c80a2b80 0x7f70c80a5040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f70e419fa10 tx=0x7f70d4005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 msgr2=0x7f70e41a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:22.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e41a4c50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004770 tx=0x7f70cc003c10 comp rx=0 tx=0).stop 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 shutdown_connections 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f70c80a2b80 0x7f70c80a5040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70e4112b90 0x7f70e419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70e4074eb0 0x7f70e41a4c50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 --2- 172.21.15.71:0/596902944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70e4074590 0x7f70e41a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 >> 172.21.15.71:0/596902944 conn(0x7f70e40fd8c0 msgr2=0x7f70e40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:22.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 shutdown_connections 2024-09-05T12:27:22.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:22.738+0000 7f70eb800640 1 -- 172.21.15.71:0/596902944 wait complete. 2024-09-05T12:27:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:22 smithi174 ceph-mon[34313]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:23.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:23 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/596902944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:23.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:23 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/596902944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:23.750 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:27:23.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:23 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/596902944' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:24.118 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:24.609 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:24 smithi071 ceph-mon[29845]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:24 smithi123 ceph-mon[34321]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:24.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 -- 172.21.15.71:0/2338295182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658108f60 msgr2=0x7fe658110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:24.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 --2- 172.21.15.71:0/2338295182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658108f60 0x7fe658110ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe650009ca0 tx=0x7fe65002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 -- 172.21.15.71:0/2338295182 shutdown_connections 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 --2- 172.21.15.71:0/2338295182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658108f60 0x7fe658110ae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 --2- 172.21.15.71:0/2338295182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6581047e0 0x7fe658108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 --2- 172.21.15.71:0/2338295182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe658103e30 0x7fe658104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 -- 172.21.15.71:0/2338295182 >> 172.21.15.71:0/2338295182 conn(0x7fe6580ff8d0 msgr2=0x7fe658101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 -- 172.21.15.71:0/2338295182 shutdown_connections 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.718+0000 7fe65d600640 1 -- 172.21.15.71:0/2338295182 wait complete. 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 Processor -- start 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 -- start start 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 0x7fe65819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6581047e0 0x7fe65819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe658108f60 0x7fe65819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:24.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe658114010 con 0x7fe658103e30 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe658113e90 con 0x7fe6581047e0 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.719+0000 7fe65d600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe658114190 con 0x7fe658108f60 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 0x7fe65819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 0x7fe65819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55572/0 (socket says 172.21.15.71:55572) 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 -- 172.21.15.71:0/368770214 learned_addr learned my addr 172.21.15.71:0/368770214 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe65cc00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe658108f60 0x7fe65819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe64fe00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6581047e0 0x7fe65819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 -- 172.21.15.71:0/368770214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe658108f60 msgr2=0x7fe65819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe658108f60 0x7fe65819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 -- 172.21.15.71:0/368770214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6581047e0 msgr2=0x7fe65819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:24.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6581047e0 0x7fe65819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 -- 172.21.15.71:0/368770214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65819f3a0 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.720+0000 7fe657e00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 0x7fe65819a3f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe64800e9f0 tx=0x7fe64800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.721+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64800cd80 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.721+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65819f690 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.721+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe648010910 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.721+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6480188f0 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.721+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6581a6f70 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.722+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe658104210 con 0x7fe658103e30 2024-09-05T12:27:24.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.724+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe648010ac0 con 0x7fe658103e30 2024-09-05T12:27:24.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.725+0000 7fe655600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe63c0a2b80 0x7fe63c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:24.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.725+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fe6480cf790 con 0x7fe658103e30 2024-09-05T12:27:24.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.725+0000 7fe64fe00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe63c0a2b80 0x7fe63c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:24.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.726+0000 7fe64fe00640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe63c0a2b80 0x7fe63c0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe658100180 tx=0x7fe640005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:24.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.729+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe64809bce0 con 0x7fe658103e30 2024-09-05T12:27:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:24 smithi174 ceph-mon[34313]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:24.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.938+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe65819b750 con 0x7fe658103e30 2024-09-05T12:27:24.939 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.939+0000 7fe655600640 1 -- 172.21.15.71:0/368770214 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11968 (secure 0 0 0) 0x7fe64809ffe0 con 0x7fe658103e30 2024-09-05T12:27:24.940 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:24.940 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":29,"btime":"2024-09-05T12:25:40:321146+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:25:40.307362+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:27:24.940 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 29 2024-09-05T12:27:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe63c0a2b80 msgr2=0x7fe63c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe63c0a2b80 0x7fe63c0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe658100180 tx=0x7fe640005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 msgr2=0x7fe65819a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 0x7fe65819a3f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe64800e9f0 tx=0x7fe64800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 shutdown_connections 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fe63c0a2b80 0x7fe63c0a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe658108f60 0x7fe65819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6581047e0 0x7fe65819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 --2- 172.21.15.71:0/368770214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe658103e30 0x7fe65819a3f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.941+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 >> 172.21.15.71:0/368770214 conn(0x7fe6580ff8d0 msgr2=0x7fe658101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.942+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 shutdown_connections 2024-09-05T12:27:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:24.942+0000 7fe65d600640 1 -- 172.21.15.71:0/368770214 wait complete. 2024-09-05T12:27:25.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:25 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/368770214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:25.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:25 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/368770214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:25.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:25 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/368770214' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:25.788 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:27:26.155 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:26.652 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:26 smithi071 ceph-mon[29845]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:27:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:26 smithi123 ceph-mon[34321]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:27:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:26 smithi174 ceph-mon[34313]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 -- 172.21.15.71:0/749783852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 msgr2=0x7fbc5010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 --2- 172.21.15.71:0/749783852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5010de80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009900 tx=0x7fbc3802a480 comp rx=0 tx=0).stop 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 -- 172.21.15.71:0/749783852 shutdown_connections 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 --2- 172.21.15.71:0/749783852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5010e3c0 0x7fbc50110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 --2- 172.21.15.71:0/749783852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5010de80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 --2- 172.21.15.71:0/749783852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc50074620 0x7fbc50074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 -- 172.21.15.71:0/749783852 >> 172.21.15.71:0/749783852 conn(0x7fbc500fd8c0 msgr2=0x7fbc500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 -- 172.21.15.71:0/749783852 shutdown_connections 2024-09-05T12:27:26.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.761+0000 7fbc56c00640 1 -- 172.21.15.71:0/749783852 wait complete. 2024-09-05T12:27:26.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.762+0000 7fbc56c00640 1 Processor -- start 2024-09-05T12:27:26.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.762+0000 7fbc56c00640 1 -- start start 2024-09-05T12:27:26.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc56c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc50074620 0x7fbc5019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:26.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc56c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:26.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc56c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc5010e3c0 0x7fbc5019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:26.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc56c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbc501141b0 con 0x7fbc50074f40 2024-09-05T12:27:26.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc56c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50114030 con 0x7fbc5010e3c0 2024-09-05T12:27:26.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc56c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc50114330 con 0x7fbc50074620 2024-09-05T12:27:26.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc47400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:26.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc47400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55582/0 (socket says 172.21.15.71:55582) 2024-09-05T12:27:26.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc4fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc50074620 0x7fbc5019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:26.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc47400640 1 -- 172.21.15.71:0/3293082811 learned_addr learned my addr 172.21.15.71:0/3293082811 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:26.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.763+0000 7fbc54c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc5010e3c0 0x7fbc5019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:26.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4fe00640 1 -- 172.21.15.71:0/3293082811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc5010e3c0 msgr2=0x7fbc5019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:26.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4fe00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc5010e3c0 0x7fbc5019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4fe00640 1 -- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 msgr2=0x7fbc5019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:26.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4fe00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4fe00640 1 -- 172.21.15.71:0/3293082811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5019f620 con 0x7fbc50074620 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc47400640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc54c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc5010e3c0 0x7fbc5019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4fe00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc50074620 0x7fbc5019a670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000e9f0 tx=0x7fbc4000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000ccc0 con 0x7fbc50074620 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc40010910 con 0x7fbc50074620 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5019f910 con 0x7fbc50074620 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40018960 con 0x7fbc50074620 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.764+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc501a71f0 con 0x7fbc50074620 2024-09-05T12:27:26.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.765+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc50074a00 con 0x7fbc50074620 2024-09-05T12:27:26.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.767+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc40010ac0 con 0x7fbc50074620 2024-09-05T12:27:26.769 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.768+0000 7fbc4d600640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbc340a2b80 0x7fbc340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:26.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.769+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fbc400cf790 con 0x7fbc50074620 2024-09-05T12:27:26.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.769+0000 7fbc47400640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbc340a2b80 0x7fbc340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:26.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.769+0000 7fbc47400640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbc340a2b80 0x7fbc340a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc380096c0 tx=0x7fbc380046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:26.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.773+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc4009bce0 con 0x7fbc50074620 2024-09-05T12:27:26.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.952+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc50108fb0 con 0x7fbc50074620 2024-09-05T12:27:26.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.954+0000 7fbc4d600640 1 -- 172.21.15.71:0/3293082811 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24741 (secure 0 0 0) 0x7fbc4009ffe0 con 0x7fbc50074620 2024-09-05T12:27:26.955 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:26.956 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":94,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:25:34.259275+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:27:26.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbc340a2b80 msgr2=0x7fbc340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:26.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbc340a2b80 0x7fbc340a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fbc380096c0 tx=0x7fbc380046c0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc50074620 msgr2=0x7fbc5019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc50074620 0x7fbc5019a670 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000e9f0 tx=0x7fbc4000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 shutdown_connections 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fbc340a2b80 0x7fbc340a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc5010e3c0 0x7fbc5019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50074f40 0x7fbc5019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 --2- 172.21.15.71:0/3293082811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc50074620 0x7fbc5019a670 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.956+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 >> 172.21.15.71:0/3293082811 conn(0x7fbc500fd8c0 msgr2=0x7fbc500fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.957+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 shutdown_connections 2024-09-05T12:27:26.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:26.957+0000 7fbc56c00640 1 -- 172.21.15.71:0/3293082811 wait complete. 2024-09-05T12:27:27.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:27 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3293082811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:27:27.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:27 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3293082811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:27:27.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:27 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3293082811' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:27:27.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T12:24:49.631865+0000', 'modified': '2024-09-05T12:25:40.307362+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24514, 'mds_2': 24502, 'mds_3': 24490, 'mds_4': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14469': {'gid': 14469, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6841/991266062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6840', 'nonce': 991266062}, {'type': 'v1', 'addr': '172.21.15.71:6841', 'nonce': 991266062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/679053656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 679053656}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 679053656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/1436525377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1436525377}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1436525377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/2774200097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2774200097}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2774200097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/626911382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 626911382}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 626911382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.123:6837/3417870839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 3417870839}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3417870839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.174:6837/870087433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 870087433}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 870087433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.174:6839/584492863', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 584492863}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 584492863}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.71:6839/2593934735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 2593934735}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 2593934735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/782979893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 782979893}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 782979893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24485, 24524, 24490, 24502, 24514]} 2024-09-05T12:27:27.779 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T12:27:27.779 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T12:27:28.146 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:28.646 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:28 smithi071 ceph-mon[29845]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:28 smithi123 ceph-mon[34321]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:28 smithi174 ceph-mon[34313]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:28.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 -- 172.21.15.71:0/2022926246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540109140 msgr2=0x7f8540110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 --2- 172.21.15.71:0/2022926246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540109140 0x7f8540110cc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8538009ca0 tx=0x7f853802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 -- 172.21.15.71:0/2022926246 shutdown_connections 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 --2- 172.21.15.71:0/2022926246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540109140 0x7f8540110cc0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 --2- 172.21.15.71:0/2022926246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85401049c0 0x7f8540108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 --2- 172.21.15.71:0/2022926246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8540104010 0x7f85401043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.756+0000 7f8546200640 1 -- 172.21.15.71:0/2022926246 >> 172.21.15.71:0/2022926246 conn(0x7f85400ffa50 msgr2=0x7f8540101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.757+0000 7f8546200640 1 -- 172.21.15.71:0/2022926246 shutdown_connections 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.757+0000 7f8546200640 1 -- 172.21.15.71:0/2022926246 wait complete. 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.757+0000 7f8546200640 1 Processor -- start 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 -- start start 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 0x7f854019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85401049c0 0x7f854019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8540109140 0x7f854019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f85401141c0 con 0x7f8540104010 2024-09-05T12:27:28.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8540114040 con 0x7f8540109140 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8546200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8540114340 con 0x7f85401049c0 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 0x7f854019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f8536a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85401049c0 0x7f854019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 0x7f854019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55604/0 (socket says 172.21.15.71:55604) 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f853f400640 1 -- 172.21.15.71:0/1184421452 learned_addr learned my addr 172.21.15.71:0/1184421452 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.758+0000 7f853fe00640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8540109140 0x7f854019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8536a00640 1 -- 172.21.15.71:0/1184421452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8540109140 msgr2=0x7f854019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8536a00640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8540109140 0x7f854019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8536a00640 1 -- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 msgr2=0x7f854019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8536a00640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 0x7f854019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8536a00640 1 -- 172.21.15.71:0/1184421452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854019f610 con 0x7f85401049c0 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f853f400640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 0x7f854019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:28.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8536a00640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85401049c0 0x7f854019aba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f852800e9f0 tx=0x7f852800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f853cc00640 1 -- 172.21.15.71:0/1184421452 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852800ccc0 con 0x7f85401049c0 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854019f900 con 0x7f85401049c0 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f853cc00640 1 -- 172.21.15.71:0/1184421452 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8528010910 con 0x7f85401049c0 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.759+0000 7f853cc00640 1 -- 172.21.15.71:0/1184421452 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8528018960 con 0x7f85401049c0 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.760+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a71e0 con 0x7f85401049c0 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.762+0000 7f8548e3b740 1 Processor -- start 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 -- start start 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196ee3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eedfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196ef0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560196e490b0 con 0x560196ee3330 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560196df0ab0 con 0x560196ee3c50 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8548e3b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560196e45070 con 0x560196eee510 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f853cc00640 1 -- 172.21.15.71:0/1184421452 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8528010ac0 con 0x7f85401049c0 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f853fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196ee3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f853fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196ee3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55618/0 (socket says 172.21.15.71:55618) 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f853fe00640 1 -- 172.21.15.71:0/2972002907 learned_addr learned my addr 172.21.15.71:0/2972002907 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f853f400640 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eedfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.763+0000 7f8536a00640 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196ef0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f853fe00640 1 -- 172.21.15.71:0/2972002907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 msgr2=0x560196ef0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f853fe00640 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196ef0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f853fe00640 1 -- 172.21.15.71:0/2972002907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 msgr2=0x560196eedfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f853fe00640 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eedfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f853fe00640 1 -- 172.21.15.71:0/2972002907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560196dfd4a0 con 0x560196ee3330 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f853fe00640 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196ee3710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8538009ca0 tx=0x7f85380093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3b6d26460bc57c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f852fe00640 1 -- 172.21.15.71:0/2972002907 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538031db0 con 0x560196ee3330 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f852fe00640 1 -- 172.21.15.71:0/2972002907 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538036030 con 0x560196ee3330 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.764+0000 7f852fe00640 1 -- 172.21.15.71:0/2972002907 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85380042d0 con 0x560196ee3330 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f853cc00640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f85240a2b80 0x7f85240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f853cc00640 1 -- 172.21.15.71:0/1184421452 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f85280cf720 con 0x7f85401049c0 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 -- 172.21.15.71:0/2972002907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 msgr2=0x560196ee3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196ee3710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8538009ca0 tx=0x7f85380093e0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f853f400640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f85240a2b80 0x7f85240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 -- 172.21.15.71:0/2972002907 shutdown_connections 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196ef0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eedfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 --2- 172.21.15.71:0/2972002907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196ee3710 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 -- 172.21.15.71:0/2972002907 >> 172.21.15.71:0/2972002907 conn(0x560196edf8c0 msgr2=0x560196edfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 -- 172.21.15.71:0/2972002907 shutdown_connections 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.765+0000 7f8548e3b740 1 -- 172.21.15.71:0/2972002907 wait complete. 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f853f400640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f85240a2b80 0x7f85240a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8530009fd0 tx=0x7f8530005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 -- start start 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 Processor -- start 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eeb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196eebed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560196ef0fa0 con 0x560196ee3330 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560196dd1920 con 0x560196ee3c50 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.766+0000 7f8548e3b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560196ef1120 con 0x560196eee510 2024-09-05T12:27:28.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f8548e3b740 10 client.0 _ll_register_callbacks cb 0x560196df58a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55628/0 (socket says 172.21.15.71:55628) 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 -- 172.21.15.71:0/3429745551 learned_addr learned my addr 172.21.15.71:0/3429745551 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f8536a00640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eeb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853fe00640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196eebed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 -- 172.21.15.71:0/3429745551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 msgr2=0x560196eebed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196eebed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 -- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 msgr2=0x560196eeb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eeb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f853f400640 1 -- 172.21.15.71:0/3429745551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560196bba2b0 con 0x560196ee3330 2024-09-05T12:27:28.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.767+0000 7f8536a00640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eeb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f853f400640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f853000f8c0 tx=0x7f8530004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f8ecd77d2a77bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f852cc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f852cc00640 1 -- 172.21.15.71:0/3429745551 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530011bd0 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f852cc00640 1 -- 172.21.15.71:0/3429745551 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f8530011d70 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8538036030 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f852cc00640 1 -- 172.21.15.71:0/3429745551 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8530031050 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f8548e3b740 10 client.15066 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560196efcbf0 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.768+0000 7f852cc00640 1 -- 172.21.15.71:0/3429745551 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8538036030 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f852cc00640 1 -- 172.21.15.71:0/3429745551 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8530027030 con 0x560196ee3330 2024-09-05T12:27:28.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8548e3b740 10 client.15066 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8548e3b740 10 client.15066 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8548e3b740 20 client.15066 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196eff230 0x560196f006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8548e3b740 4 client.15066 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 --> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560196e56620 con 0x560196eff230 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.769+0000 7f8536a00640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196eff230 0x560196f006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.770+0000 7f8536a00640 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196eff230 0x560196f006a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e3f32ee7d54fc15f server_cookie=fe861127ad8c44c7 in_seq=0 out_seq=0 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.770+0000 7f852cc00640 10 client.15066 ms_handle_connect on v2:172.21.15.123:6836/3417870839 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.770+0000 7f852cc00640 1 -- 172.21.15.71:0/3429745551 <== mds.4 v2:172.21.15.123:6836/3417870839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560196e56620 con 0x560196eff230 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.770+0000 7f852cc00640 10 client.15066 handle_command_reply: tid=0 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 client.15066 shutdown 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 msgr2=0x560196eeb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f853000f8c0 tx=0x7f8530004730 comp rx=0 tx=0).stop 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 shutdown_connections 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196eff230 0x560196f006a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196eebed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196eeb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3429745551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 >> 172.21.15.71:0/3429745551 conn(0x560196edf8c0 msgr2=0x560196ee55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f852cc00640 0 client.15066 ms_handle_reset on v2:172.21.15.123:6836/3417870839 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 shutdown_connections 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 1 -- 172.21.15.71:0/3429745551 wait complete. 2024-09-05T12:27:28.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.771+0000 7f8548e3b740 20 client.15066 trim_cache size 0 max 16384 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 Processor -- start 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 -- start start 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196ee3c50 0x560196eebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196eee510 0x560196ee86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560196efe9e0 con 0x560196ee3330 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560196eff0b0 con 0x560196eee510 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8548e3b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560196eff780 con 0x560196ee3c50 2024-09-05T12:27:28.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55642/0 (socket says 172.21.15.71:55642) 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f8536a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196ee3c50 0x560196eebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f853f400640 1 -- 172.21.15.71:0/3958930784 learned_addr learned my addr 172.21.15.71:0/3958930784 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.790+0000 7f853fe00640 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196eee510 0x560196ee86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f8536a00640 1 -- 172.21.15.71:0/3958930784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196eee510 msgr2=0x560196ee86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f8536a00640 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196eee510 0x560196ee86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f8536a00640 1 -- 172.21.15.71:0/3958930784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 msgr2=0x560196eeb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f8536a00640 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f8536a00640 1 -- 172.21.15.71:0/3958930784 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560196ee99a0 con 0x560196ee3c50 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f8536a00640 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196ee3c50 0x560196eebee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x560196ee0c40 tx=0x7f85280ccd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2369e26feb7b2a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f852f400640 1 -- 172.21.15.71:0/3958930784 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852809ff70 con 0x560196ee3c50 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f852f400640 1 -- 172.21.15.71:0/3958930784 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85280a4830 con 0x560196ee3c50 2024-09-05T12:27:28.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.791+0000 7f852f400640 1 -- 172.21.15.71:0/3958930784 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280cfb10 con 0x560196ee3c50 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 -- 172.21.15.71:0/3958930784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196ee3c50 msgr2=0x560196eebee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196ee3c50 0x560196eebee0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x560196ee0c40 tx=0x7f85280ccd30 comp rx=0 tx=0).stop 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 -- 172.21.15.71:0/3958930784 shutdown_connections 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196eee510 0x560196ee86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196ee3c50 0x560196eebee0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 --2- 172.21.15.71:0/3958930784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196eeb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 -- 172.21.15.71:0/3958930784 >> 172.21.15.71:0/3958930784 conn(0x560196edf8c0 msgr2=0x560196ee9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 -- 172.21.15.71:0/3958930784 shutdown_connections 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 -- 172.21.15.71:0/3958930784 wait complete. 2024-09-05T12:27:28.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.792+0000 7f8548e3b740 1 -- start start 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 Processor -- start 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196f08e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196f021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196f02700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560196f00520 con 0x560196ee3330 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560196effe50 con 0x560196ee3c50 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560196f00bf0 con 0x560196eee510 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8548e3b740 10 client.0 _ll_register_callbacks cb 0x7f853000f720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196f08e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f853f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196f08e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55644/0 (socket says 172.21.15.71:55644) 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f853f400640 1 -- 172.21.15.71:0/1079046843 learned_addr learned my addr 172.21.15.71:0/1079046843 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f853fe00640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196f02700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.793+0000 7f8536a00640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196f021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f853f400640 1 -- 172.21.15.71:0/1079046843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 msgr2=0x560196f02700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f853f400640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196f02700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f853f400640 1 -- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 msgr2=0x560196f021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f853f400640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196f021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f853f400640 1 -- 172.21.15.71:0/1079046843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560196f02cc0 con 0x560196ee3330 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f853f400640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196f08e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f853000f860 tx=0x7f853001d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abad40bec838efda server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f852cc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f852cc00640 1 -- 172.21.15.71:0/1079046843 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530011810 con 0x560196ee3330 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f852cc00640 1 -- 172.21.15.71:0/1079046843 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f8530011de0 con 0x560196ee3330 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.794+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85280a4830 con 0x560196ee3330 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f852cc00640 1 -- 172.21.15.71:0/1079046843 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8530045050 con 0x560196ee3330 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 10 client.15072 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560196f077a0 con 0x560196ee3330 2024-09-05T12:27:28.798 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f852cc00640 1 -- 172.21.15.71:0/1079046843 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85280a4830 con 0x560196ee3330 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f852cc00640 1 -- 172.21.15.71:0/1079046843 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8530027040 con 0x560196ee3330 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 10 client.15072 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 10 client.15072 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 20 client.15072 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196f6e3b0 0x560196f6d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 4 client.15072 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:28.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.795+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 --> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] -- command(tid 0: {"prefix": "status"}) -- 0x560196e50b30 con 0x560196f6e3b0 2024-09-05T12:27:28.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.796+0000 7f8536a00640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196f6e3b0 0x560196f6d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:28.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.796+0000 7f8536a00640 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196f6e3b0 0x560196f6d0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cdb1a26edb7b2491 server_cookie=6b0769d23b4cc2bb in_seq=0 out_seq=0 2024-09-05T12:27:28.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.796+0000 7f852cc00640 10 client.15072 ms_handle_connect on v2:172.21.15.123:6836/3417870839 2024-09-05T12:27:28.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.796+0000 7f852cc00640 1 -- 172.21.15.71:0/1079046843 <== mds.4 v2:172.21.15.123:6836/3417870839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560196e50b30 con 0x560196f6e3b0 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.796+0000 7f852cc00640 10 client.15072 handle_command_reply: tid=0 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.796+0000 7f8548e3b740 1 client.15072 shutdown 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 msgr2=0x560196f08e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196f08e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f853000f860 tx=0x7f853001d8e0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 shutdown_connections 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x560196f6e3b0 0x560196f6d0c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560196eee510 0x560196f02700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560196ee3c50 0x560196f021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 --2- 172.21.15.71:0/1079046843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560196ee3330 0x560196f08e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f852cc00640 0 client.15072 ms_handle_reset on v2:172.21.15.123:6836/3417870839 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 >> 172.21.15.71:0/1079046843 conn(0x560196edf8c0 msgr2=0x560196ee4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 shutdown_connections 2024-09-05T12:27:28.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 1 -- 172.21.15.71:0/1079046843 wait complete. 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.797+0000 7f8548e3b740 20 client.15072 trim_cache size 0 max 16384 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 4, 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24485, 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 110.48369438100001, 2024-09-05T12:27:28.802 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:28.803 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:28.803 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:28.803 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 184.69517582700001 2024-09-05T12:27:28.803 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:28.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f85240a2b80 msgr2=0x7f85240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f85240a2b80 0x7f85240a5040 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8530009fd0 tx=0x7f8530005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85401049c0 msgr2=0x7f854019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85401049c0 0x7f854019aba0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f852800e9f0 tx=0x7f852800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 shutdown_connections 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f85240a2b80 0x7f85240a5040 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8540109140 0x7f854019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85401049c0 0x7f854019aba0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 --2- 172.21.15.71:0/1184421452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8540104010 0x7f854019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 >> 172.21.15.71:0/1184421452 conn(0x7f85400ffa50 msgr2=0x7f8540100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 shutdown_connections 2024-09-05T12:27:28.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:28.798+0000 7f8546200640 1 -- 172.21.15.71:0/1184421452 wait complete. 2024-09-05T12:27:29.649 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.c status 2024-09-05T12:27:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:30.406 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:30 smithi071 ceph-mon[29845]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:27:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:30 smithi123 ceph-mon[34321]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 -- 172.21.15.71:0/4194603462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8106f20 msgr2=0x7ff1c810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 --2- 172.21.15.71:0/4194603462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8106f20 0x7ff1c810eaa0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009ca0 tx=0x7ff1c002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 -- 172.21.15.71:0/4194603462 shutdown_connections 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 --2- 172.21.15.71:0/4194603462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c8106f20 0x7ff1c810eaa0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 --2- 172.21.15.71:0/4194603462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c81027a0 0x7ff1c81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 --2- 172.21.15.71:0/4194603462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8101df0 0x7ff1c81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 -- 172.21.15.71:0/4194603462 >> 172.21.15.71:0/4194603462 conn(0x7ff1c80fd8b0 msgr2=0x7ff1c80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.663+0000 7ff1cde00640 1 -- 172.21.15.71:0/4194603462 shutdown_connections 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.664+0000 7ff1cde00640 1 -- 172.21.15.71:0/4194603462 wait complete. 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.664+0000 7ff1cde00640 1 Processor -- start 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 -- start start 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c8101df0 0x7ff1c819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c81027a0 0x7ff1c819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8106f20 0x7ff1c819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8111fd0 con 0x7ff1c81027a0 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8111e50 con 0x7ff1c8101df0 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1cde00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8112150 con 0x7ff1c8106f20 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c8101df0 0x7ff1c819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8106f20 0x7ff1c819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c8101df0 0x7ff1c819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:38232/0 (socket says 172.21.15.71:38232) 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2304793860 learned_addr learned my addr 172.21.15.71:0/2304793860 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.665+0000 7ff1c6000640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c81027a0 0x7ff1c819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1c7400640 1 -- 172.21.15.71:0/2304793860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c8101df0 msgr2=0x7ff1c819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1c7400640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c8101df0 0x7ff1c819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1c7400640 1 -- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c81027a0 msgr2=0x7ff1c819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1c7400640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c81027a0 0x7ff1c819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1c7400640 1 -- 172.21.15.71:0/2304793860 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c819f5a0 con 0x7ff1c8106f20 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1c7400640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8106f20 0x7ff1c819eec0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0005e30 tx=0x7ff1c0003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1cca00640 1 -- 172.21.15.71:0/2304793860 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0031650 con 0x7ff1c8106f20 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c819f830 con 0x7ff1c8106f20 2024-09-05T12:27:30.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1cca00640 1 -- 172.21.15.71:0/2304793860 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1c00317f0 con 0x7ff1c8106f20 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.666+0000 7ff1cca00640 1 -- 172.21.15.71:0/2304793860 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c002f990 con 0x7ff1c8106f20 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.667+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c81a7170 con 0x7ff1c8106f20 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 Processor -- start 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1cca00640 1 -- 172.21.15.71:0/2304793860 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1c002fb50 con 0x7ff1c8106f20 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 -- start start 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c0b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c15fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c18900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f254b710b0 con 0x55f254c0b330 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f254b18ab0 con 0x55f254c16510 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.670+0000 7ff1d0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f254b6d070 con 0x55f254c0bc50 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c0b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c15fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c0b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55666/0 (socket says 172.21.15.71:55666) 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c15fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45762/0 (socket says 172.21.15.71:45762) 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6000640 1 -- 172.21.15.71:0/2512968028 learned_addr learned my addr 172.21.15.71:0/2512968028 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6000640 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c18900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2512968028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 msgr2=0x55f254c18900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c18900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2512968028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 msgr2=0x55f254c0b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c0b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2512968028 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f254b254a0 con 0x55f254c0bc50 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6000640 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c0b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c15fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009cf0 tx=0x7ff1b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a80dfbd1bbe9b8a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.671+0000 7ff1cca00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff1ac0a2b80 0x7ff1ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1b3e00640 1 -- 172.21.15.71:0/2512968028 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8005220 con 0x55f254c0bc50 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1cca00640 1 -- 172.21.15.71:0/2304793860 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff1c00f05b0 con 0x7ff1c8106f20 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1b3e00640 1 -- 172.21.15.71:0/2512968028 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b8004c80 con 0x55f254c0bc50 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1b3e00640 1 -- 172.21.15.71:0/2512968028 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8019970 con 0x55f254c0bc50 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1c6000640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff1ac0a2b80 0x7ff1ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 -- 172.21.15.71:0/2512968028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 msgr2=0x55f254c15fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1c6000640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff1ac0a2b80 0x7ff1ac0a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009fd0 tx=0x7ff1b4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c15fd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009cf0 tx=0x7ff1b8011810 comp rx=0 tx=0).stop 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 -- 172.21.15.71:0/2512968028 shutdown_connections 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c18900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c15fd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2512968028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c0b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 -- 172.21.15.71:0/2512968028 >> 172.21.15.71:0/2512968028 conn(0x55f254c078c0 msgr2=0x55f254c07ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:30.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.672+0000 7ff1d0859740 1 -- 172.21.15.71:0/2512968028 shutdown_connections 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.673+0000 7ff1d0859740 1 -- 172.21.15.71:0/2512968028 wait complete. 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.673+0000 7ff1d0859740 1 -- start start 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.673+0000 7ff1d0859740 1 Processor -- start 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.673+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 0x55f254c135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.673+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0bc50 0x55f254c13af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c14030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1d0859740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f254c19060 con 0x55f254c16510 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1d0859740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f254af9920 con 0x55f254c0bc50 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1d0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f254c191e0 con 0x55f254c0b330 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1d0859740 10 client.0 _ll_register_callbacks cb 0x55f254b1d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 0x55f254c135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 0x55f254c135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45776/0 (socket says 172.21.15.71:45776) 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 -- 172.21.15.71:0/324482075 learned_addr learned my addr 172.21.15.71:0/324482075 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:30.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c7400640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c14030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6000640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0bc50 0x55f254c13af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 -- 172.21.15.71:0/324482075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0bc50 msgr2=0x55f254c13af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0bc50 0x55f254c13af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 -- 172.21.15.71:0/324482075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 msgr2=0x55f254c14030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c14030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c6a00640 1 -- 172.21.15.71:0/324482075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2548e22b0 con 0x55f254c0b330 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.674+0000 7ff1c7400640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 0x55f254c135b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009fa0 tx=0x7ff1b8005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=592d9cc61e6a46f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1b0c00640 1 -- 172.21.15.71:0/324482075 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8029070 con 0x55f254c0b330 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8004c80 con 0x55f254c0b330 2024-09-05T12:27:30.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1b0c00640 1 -- 172.21.15.71:0/324482075 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff1b8005240 con 0x55f254c0b330 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1b0c00640 1 -- 172.21.15.71:0/324482075 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1b8035050 con 0x55f254c0b330 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1d0859740 10 client.25135 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f254c24e80 con 0x55f254c0b330 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.675+0000 7ff1b0c00640 1 -- 172.21.15.71:0/324482075 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b8004c80 con 0x55f254c0b330 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1b0c00640 1 -- 172.21.15.71:0/324482075 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff1b80235f0 con 0x55f254c0b330 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1d0859740 10 client.25135 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1d0859740 10 client.25135 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1d0859740 20 client.25135 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1d0859740 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c27210 0x55f254c28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1d0859740 4 client.25135 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 --> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f254b7e620 con 0x55f254c27210 2024-09-05T12:27:30.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.676+0000 7ff1c6000640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c27210 0x55f254c28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.677+0000 7ff1c6000640 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c27210 0x55f254c28680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64212d4eebc7f324 server_cookie=3f9bca981bb55e21 in_seq=0 out_seq=0 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.677+0000 7ff1b0c00640 10 client.25135 ms_handle_connect on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.677+0000 7ff1b0c00640 1 -- 172.21.15.71:0/324482075 <== mds.3 v2:172.21.15.174:6836/870087433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f254b7e620 con 0x55f254c27210 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.677+0000 7ff1b0c00640 10 client.25135 handle_command_reply: tid=0 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.677+0000 7ff1d0859740 1 client.25135 shutdown 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 msgr2=0x55f254c135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 0x55f254c135b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009fa0 tx=0x7ff1b8005d30 comp rx=0 tx=0).stop 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 shutdown_connections 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c27210 0x55f254c28680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c14030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1b0c00640 0 client.25135 ms_handle_reset on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0bc50 0x55f254c13af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 --2- 172.21.15.71:0/324482075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0b330 0x55f254c135b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 >> 172.21.15.71:0/324482075 conn(0x55f254c078c0 msgr2=0x55f254c0d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:30.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 shutdown_connections 2024-09-05T12:27:30.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 1 -- 172.21.15.71:0/324482075 wait complete. 2024-09-05T12:27:30.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.678+0000 7ff1d0859740 20 client.25135 trim_cache size 0 max 16384 2024-09-05T12:27:30.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 Processor -- start 2024-09-05T12:27:30.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 -- start start 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0b330 0x55f254c0dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c13ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c10890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f254c269c0 con 0x55f254c16510 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f254c27090 con 0x55f254c0b330 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.697+0000 7ff1d0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f254c27760 con 0x55f254c0bc50 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c10890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c10890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55684/0 (socket says 172.21.15.71:55684) 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c7400640 1 -- 172.21.15.71:0/2729096782 learned_addr learned my addr 172.21.15.71:0/2729096782 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0b330 0x55f254c0dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6000640 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c13ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2729096782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 msgr2=0x55f254c13ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c13ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2729096782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 msgr2=0x55f254c10890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c10890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 -- 172.21.15.71:0/2729096782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f254c0ba80 con 0x55f254c0b330 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.698+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0b330 0x55f254c0dd90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800a010 tx=0x7ff1b80059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=715efa7cdfa41bab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1b3400640 1 -- 172.21.15.71:0/2729096782 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8029050 con 0x55f254c0b330 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1b3400640 1 -- 172.21.15.71:0/2729096782 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b8005040 con 0x55f254c0b330 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1b3400640 1 -- 172.21.15.71:0/2729096782 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8019a20 con 0x55f254c0b330 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 -- 172.21.15.71:0/2729096782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0b330 msgr2=0x55f254c0dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0b330 0x55f254c0dd90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800a010 tx=0x7ff1b80059b0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 -- 172.21.15.71:0/2729096782 shutdown_connections 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c16510 0x55f254c10890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c13ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 --2- 172.21.15.71:0/2729096782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c0b330 0x55f254c0dd90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 -- 172.21.15.71:0/2729096782 >> 172.21.15.71:0/2729096782 conn(0x55f254c078c0 msgr2=0x55f254c26230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 -- 172.21.15.71:0/2729096782 shutdown_connections 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.699+0000 7ff1d0859740 1 -- 172.21.15.71:0/2729096782 wait complete. 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.700+0000 7ff1d0859740 1 -- start start 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.700+0000 7ff1d0859740 1 Processor -- start 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.700+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c308e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c30e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1d0859740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c2b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1d0859740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f254c28500 con 0x55f254c0b330 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1d0859740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f254c27e30 con 0x55f254c16510 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1d0859740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f254c28bd0 con 0x55f254c0bc50 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1d0859740 10 client.0 _ll_register_callbacks cb 0x7ff1b8004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c308e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55700/0 (socket says 172.21.15.71:55700) 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 -- 172.21.15.71:0/4145628127 learned_addr learned my addr 172.21.15.71:0/4145628127 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c7400640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c2b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6000640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c30e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 -- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 msgr2=0x55f254c30e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c30e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 -- 172.21.15.71:0/4145628127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 msgr2=0x55f254c2b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c2b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6a00640 1 -- 172.21.15.71:0/4145628127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f254c14a20 con 0x55f254c0b330 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c7400640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.701+0000 7ff1c6000640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1c6a00640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c308e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80239f0 tx=0x7ff1b8004d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d541fa206816a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:30.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1b0c00640 1 -- 172.21.15.71:0/4145628127 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b803e070 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1b0c00640 1 -- 172.21.15.71:0/4145628127 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7ff1b8023630 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8005040 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1b0c00640 1 -- 172.21.15.71:0/4145628127 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff1b8047050 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1d0859740 10 client.15090 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f254c91cc0 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.702+0000 7ff1b0c00640 1 -- 172.21.15.71:0/4145628127 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1b8005040 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1b0c00640 1 -- 172.21.15.71:0/4145628127 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7ff1b8041020 con 0x55f254c0b330 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1d0859740 10 client.15090 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1d0859740 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1d0859740 20 client.15090 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1d0859740 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c964a0 0x55f254c951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:30.712 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1d0859740 4 client.15090 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 --> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] -- command(tid 0: {"prefix": "status"}) -- 0x55f254b78b30 con 0x55f254c964a0 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.703+0000 7ff1c6000640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c964a0 0x55f254c951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1c6000640 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c964a0 0x55f254c951b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c6f144f6780b6f4f server_cookie=eaf87caa93a30bf1 in_seq=0 out_seq=0 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1b0c00640 10 client.15090 ms_handle_connect on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1b0c00640 1 -- 172.21.15.71:0/4145628127 <== mds.3 v2:172.21.15.174:6836/870087433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f254b78b30 con 0x55f254c964a0 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1b0c00640 10 client.15090 handle_command_reply: tid=0 2024-09-05T12:27:30.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 client.15090 shutdown 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 msgr2=0x55f254c308e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c308e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff1b80239f0 tx=0x7ff1b8004d20 comp rx=0 tx=0).stop 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 shutdown_connections 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f254c964a0 0x55f254c951b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f254c16510 0x55f254c2b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f254c0bc50 0x55f254c30e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1b0c00640 0 client.15090 ms_handle_reset on v2:172.21.15.174:6836/870087433 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 --2- 172.21.15.71:0/4145628127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f254c0b330 0x55f254c308e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 >> 172.21.15.71:0/4145628127 conn(0x55f254c078c0 msgr2=0x55f254c06200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 shutdown_connections 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.704+0000 7ff1d0859740 1 -- 172.21.15.71:0/4145628127 wait complete. 2024-09-05T12:27:30.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1d0859740 20 client.15090 trim_cache size 0 max 16384 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 3, 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24490, 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 113.401055767, 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:30.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:30.716 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 189.35035100600001 2024-09-05T12:27:30.716 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:30.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff1ac0a2b80 msgr2=0x7ff1ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff1ac0a2b80 0x7ff1ac0a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4009fd0 tx=0x7ff1b4005850 comp rx=0 tx=0).stop 2024-09-05T12:27:30.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8106f20 msgr2=0x7ff1c819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8106f20 0x7ff1c819eec0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0005e30 tx=0x7ff1c0003f10 comp rx=0 tx=0).stop 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 shutdown_connections 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7ff1ac0a2b80 0x7ff1ac0a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c8106f20 0x7ff1c819eec0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1c81027a0 0x7ff1c819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 --2- 172.21.15.71:0/2304793860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c8101df0 0x7ff1c819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.705+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 >> 172.21.15.71:0/2304793860 conn(0x7ff1c80fd8b0 msgr2=0x7ff1c80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.706+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 shutdown_connections 2024-09-05T12:27:30.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:30.706+0000 7ff1cde00640 1 -- 172.21.15.71:0/2304793860 wait complete. 2024-09-05T12:27:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:30 smithi174 ceph-mon[34313]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-05T12:27:31.587 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T12:27:31.951 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:32.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:32 smithi071 ceph-mon[29845]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:32.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 -- 172.21.15.71:0/597986928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518106f20 msgr2=0x7f951810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 --2- 172.21.15.71:0/597986928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518106f20 0x7f951810eaa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9510009ca0 tx=0x7f951002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 -- 172.21.15.71:0/597986928 shutdown_connections 2024-09-05T12:27:32.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 --2- 172.21.15.71:0/597986928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518106f20 0x7f951810eaa0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 --2- 172.21.15.71:0/597986928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95181027a0 0x7f95181067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 --2- 172.21.15.71:0/597986928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9518101df0 0x7f95181021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 -- 172.21.15.71:0/597986928 >> 172.21.15.71:0/597986928 conn(0x7f95180fd8b0 msgr2=0x7f95180ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 -- 172.21.15.71:0/597986928 shutdown_connections 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.546+0000 7f951d200640 1 -- 172.21.15.71:0/597986928 wait complete. 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.547+0000 7f951d200640 1 Processor -- start 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 -- start start 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518101df0 0x7f95181983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95181027a0 0x7f9518198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9518106f20 0x7f951819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9518111fd0 con 0x7f9518101df0 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9518111e50 con 0x7f9518106f20 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f951d200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9518112150 con 0x7f95181027a0 2024-09-05T12:27:32.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518101df0 0x7f95181983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f9516800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9518106f20 0x7f951819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f9516800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9518106f20 0x7f951819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:38272/0 (socket says 172.21.15.71:38272) 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f9516800640 1 -- 172.21.15.71:0/1812057270 learned_addr learned my addr 172.21.15.71:0/1812057270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.548+0000 7f950d400640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95181027a0 0x7f9518198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f9515e00640 1 -- 172.21.15.71:0/1812057270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95181027a0 msgr2=0x7f9518198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f9515e00640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95181027a0 0x7f9518198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f9515e00640 1 -- 172.21.15.71:0/1812057270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9518106f20 msgr2=0x7f951819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f9515e00640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9518106f20 0x7f951819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f9515e00640 1 -- 172.21.15.71:0/1812057270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951819d380 con 0x7f9518101df0 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f950d400640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95181027a0 0x7f9518198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f9515e00640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518101df0 0x7f95181983d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9f0 tx=0x7f950800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f950f400640 1 -- 172.21.15.71:0/1812057270 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800ccc0 con 0x7f9518101df0 2024-09-05T12:27:32.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f950f400640 1 -- 172.21.15.71:0/1812057270 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9508005020 con 0x7f9518101df0 2024-09-05T12:27:32.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951819d670 con 0x7f9518101df0 2024-09-05T12:27:32.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f950f400640 1 -- 172.21.15.71:0/1812057270 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080049e0 con 0x7f9518101df0 2024-09-05T12:27:32.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.549+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181a4f50 con 0x7f9518101df0 2024-09-05T12:27:32.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 Processor -- start 2024-09-05T12:27:32.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f950f400640 1 -- 172.21.15.71:0/1812057270 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9508005560 con 0x7f9518101df0 2024-09-05T12:27:32.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 -- start start 2024-09-05T12:27:32.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8ea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56369f8500b0 con 0x56369f8eac50 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56369f7f7ab0 con 0x56369f8ea330 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f951fe5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56369f84c070 con 0x56369f8f5510 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.553+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55726/0 (socket says 172.21.15.71:55726) 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 -- 172.21.15.71:0/2513438007 learned_addr learned my addr 172.21.15.71:0/2513438007 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9516800640 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8ea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f950d400640 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 -- 172.21.15.71:0/2513438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 msgr2=0x56369f8f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 -- 172.21.15.71:0/2513438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 msgr2=0x56369f8ea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8ea710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 -- 172.21.15.71:0/2513438007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369f8044a0 con 0x56369f8eac50 2024-09-05T12:27:32.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f9515e00640 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f4fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f95080d74b0 tx=0x7f95080d75a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3ce7ff88538115 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:32.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.554+0000 7f950f400640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.555+0000 7f950f400640 1 -- 172.21.15.71:0/1812057270 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f95080cf2d0 con 0x7f9518101df0 2024-09-05T12:27:32.560 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.555+0000 7f9516800640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.555+0000 7f9507400640 1 -- 172.21.15.71:0/2513438007 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080e2070 con 0x56369f8eac50 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.555+0000 7f9507400640 1 -- 172.21.15.71:0/2513438007 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080051d0 con 0x56369f8eac50 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.555+0000 7f9507400640 1 -- 172.21.15.71:0/2513438007 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950809b0d0 con 0x56369f8eac50 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 -- 172.21.15.71:0/2513438007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 msgr2=0x56369f8f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f9516800640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9510009a60 tx=0x7f9510005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f4fd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f95080d74b0 tx=0x7f95080d75a0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 -- 172.21.15.71:0/2513438007 shutdown_connections 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f4fd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 --2- 172.21.15.71:0/2513438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8ea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 -- 172.21.15.71:0/2513438007 >> 172.21.15.71:0/2513438007 conn(0x56369f8e68c0 msgr2=0x56369f8e6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 -- 172.21.15.71:0/2513438007 shutdown_connections 2024-09-05T12:27:32.561 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 -- 172.21.15.71:0/2513438007 wait complete. 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.556+0000 7f951fe5b740 1 -- start start 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 Processor -- start 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f8f2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8eac50 0x56369f8f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8f5510 0x56369f8f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56369f8f8060 con 0x56369f8ea330 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56369f7d8920 con 0x56369f8f5510 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56369f8f81e0 con 0x56369f8eac50 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f951fe5b740 10 client.0 _ll_register_callbacks cb 0x56369f7fc8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f8f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f8f2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55742/0 (socket says 172.21.15.71:55742) 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f9515e00640 1 -- 172.21.15.71:0/159033817 learned_addr learned my addr 172.21.15.71:0/159033817 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.557+0000 7f950d400640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8eac50 0x56369f8f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9516800640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8f5510 0x56369f8f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9515e00640 1 -- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8eac50 msgr2=0x56369f8f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9515e00640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8eac50 0x56369f8f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9515e00640 1 -- 172.21.15.71:0/159033817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8f5510 msgr2=0x56369f8f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9515e00640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8f5510 0x56369f8f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9515e00640 1 -- 172.21.15.71:0/159033817 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369f5c12b0 con 0x56369f8ea330 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f950d400640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8eac50 0x56369f8f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9516800640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8f5510 0x56369f8f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f9515e00640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f8f2660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f95080a3eb0 tx=0x7f95080053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5487a009d64e07f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f94fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f94fbe00640 1 -- 172.21.15.71:0/159033817 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950801c070 con 0x56369f8ea330 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f94fbe00640 1 -- 172.21.15.71:0/159033817 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f95080860b0 con 0x56369f8ea330 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.558+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95080051d0 con 0x56369f8ea330 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f94fbe00640 1 -- 172.21.15.71:0/159033817 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95080d2e00 con 0x56369f8ea330 2024-09-05T12:27:32.563 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 10 client.15102 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:32.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56369f8f8060 con 0x56369f8ea330 2024-09-05T12:27:32.564 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f94fbe00640 1 -- 172.21.15.71:0/159033817 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080051d0 con 0x56369f8ea330 2024-09-05T12:27:32.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f94fbe00640 1 -- 172.21.15.71:0/159033817 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f9508097de0 con 0x56369f8ea330 2024-09-05T12:27:32.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 10 client.15102 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:32.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T12:27:32.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 20 client.15102 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:32.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f906000 0x56369f907470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 4 client.15102 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:32.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.559+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 --> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56369f85d620 con 0x56369f906000 2024-09-05T12:27:32.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.560+0000 7f950d400640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f906000 0x56369f907470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.560+0000 7f950d400640 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f906000 0x56369f907470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fcfe4bf142625df4 server_cookie=fe0ac78bf33e2930 in_seq=0 out_seq=0 2024-09-05T12:27:32.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.560+0000 7f94fbe00640 10 client.15102 ms_handle_connect on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:32.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.561+0000 7f94fbe00640 1 -- 172.21.15.71:0/159033817 <== mds.2 v2:172.21.15.174:6838/584492863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56369f85d620 con 0x56369f906000 2024-09-05T12:27:32.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.561+0000 7f94fbe00640 10 client.15102 handle_command_reply: tid=0 2024-09-05T12:27:32.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.561+0000 7f951fe5b740 1 client.15102 shutdown 2024-09-05T12:27:32.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 msgr2=0x56369f8f2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f8f2660 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f95080a3eb0 tx=0x7f95080053d0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 shutdown_connections 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f906000 0x56369f907470 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8f5510 0x56369f8f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8eac50 0x56369f8f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 --2- 172.21.15.71:0/159033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f8f2660 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 >> 172.21.15.71:0/159033817 conn(0x56369f8e68c0 msgr2=0x56369f8ec5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:32.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f94fbe00640 0 client.15102 ms_handle_reset on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:32.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 shutdown_connections 2024-09-05T12:27:32.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 1 -- 172.21.15.71:0/159033817 wait complete. 2024-09-05T12:27:32.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.562+0000 7f951fe5b740 20 client.15102 trim_cache size 0 max 16384 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.580+0000 7f951fe5b740 1 Processor -- start 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.580+0000 7f951fe5b740 1 -- start start 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8e6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f951fe5b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56369f9057b0 con 0x56369f8eac50 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f951fe5b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56369f905e80 con 0x56369f8ea330 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f951fe5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56369f906550 con 0x56369f8f5510 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f950d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f950d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55758/0 (socket says 172.21.15.71:55758) 2024-09-05T12:27:32.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f950d400640 1 -- 172.21.15.71:0/49349837 learned_addr learned my addr 172.21.15.71:0/49349837 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:32.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f9515e00640 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8e6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f9516800640 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f9515e00640 1 -- 172.21.15.71:0/49349837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 msgr2=0x56369f8f2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f9515e00640 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f9515e00640 1 -- 172.21.15.71:0/49349837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 msgr2=0x56369f8f2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.581+0000 7f9515e00640 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.582+0000 7f9515e00640 1 -- 172.21.15.71:0/49349837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369f8f4fd0 con 0x56369f8ea330 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.582+0000 7f950d400640 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.582+0000 7f9515e00640 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8e6210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9508098140 tx=0x7f95080096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab20f691d5b6b8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.582+0000 7f9506a00640 1 -- 172.21.15.71:0/49349837 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950801c070 con 0x56369f8ea330 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.582+0000 7f9506a00640 1 -- 172.21.15.71:0/49349837 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080b3070 con 0x56369f8ea330 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.582+0000 7f9506a00640 1 -- 172.21.15.71:0/49349837 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950809b0d0 con 0x56369f8ea330 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 -- 172.21.15.71:0/49349837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 msgr2=0x56369f8e6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8e6210 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9508098140 tx=0x7f95080096e0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 -- 172.21.15.71:0/49349837 shutdown_connections 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8f2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8eac50 0x56369f8f2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 --2- 172.21.15.71:0/49349837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8ea330 0x56369f8e6210 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 -- 172.21.15.71:0/49349837 >> 172.21.15.71:0/49349837 conn(0x56369f8e68c0 msgr2=0x56369f8ecec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 -- 172.21.15.71:0/49349837 shutdown_connections 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 -- 172.21.15.71:0/49349837 wait complete. 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 -- start start 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.583+0000 7f951fe5b740 1 Processor -- start 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f90fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8eac50 0x56369f8ef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8ef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56369f9072f0 con 0x56369f8ea330 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56369f906c20 con 0x56369f8eac50 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56369f9079c0 con 0x56369f8f5510 2024-09-05T12:27:32.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f951fe5b740 10 client.0 _ll_register_callbacks cb 0x7f9500005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:32.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f90fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f9515e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f90fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55768/0 (socket says 172.21.15.71:55768) 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f9515e00640 1 -- 172.21.15.71:0/3717405984 learned_addr learned my addr 172.21.15.71:0/3717405984 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f9516800640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8ef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.584+0000 7f950d400640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8eac50 0x56369f8ef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f9515e00640 1 -- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 msgr2=0x56369f8ef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f9515e00640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8ef830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f9515e00640 1 -- 172.21.15.71:0/3717405984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8eac50 msgr2=0x56369f8ef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f9515e00640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8eac50 0x56369f8ef2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f9515e00640 1 -- 172.21.15.71:0/3717405984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369f8efed0 con 0x56369f8ea330 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f9515e00640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f90fd00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9508099a20 tx=0x7f9508099c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b8c1273041741e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f94fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f94fbe00640 1 -- 172.21.15.71:0/3717405984 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95080ba060 con 0x56369f8ea330 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f94fbe00640 1 -- 172.21.15.71:0/3717405984 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f95080a3f50 con 0x56369f8ea330 2024-09-05T12:27:32.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95080b3070 con 0x56369f8ea330 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f94fbe00640 1 -- 172.21.15.71:0/3717405984 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9508081050 con 0x56369f8ea330 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.585+0000 7f951fe5b740 10 client.15108 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56369f8efed0 con 0x56369f8ea330 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f94fbe00640 1 -- 172.21.15.71:0/3717405984 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95080b3070 con 0x56369f8ea330 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f94fbe00640 1 -- 172.21.15.71:0/3717405984 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f950809b9d0 con 0x56369f8ea330 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 10 client.15108 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 20 client.15108 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f974a60 0x56369f973770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 4 client.15108 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.586+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 --> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] -- command(tid 0: {"prefix": "status"}) -- 0x56369f857b30 con 0x56369f974a60 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.587+0000 7f950d400640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f974a60 0x56369f973770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:32.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.587+0000 7f950d400640 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f974a60 0x56369f973770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=67d0bddb537fa509 server_cookie=ce300c7430465984 in_seq=0 out_seq=0 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.587+0000 7f94fbe00640 10 client.15108 ms_handle_connect on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.587+0000 7f94fbe00640 1 -- 172.21.15.71:0/3717405984 <== mds.2 v2:172.21.15.174:6838/584492863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56369f857b30 con 0x56369f974a60 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.587+0000 7f94fbe00640 10 client.15108 handle_command_reply: tid=0 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.587+0000 7f951fe5b740 1 client.15108 shutdown 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 msgr2=0x56369f90fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f90fd00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f9508099a20 tx=0x7f9508099c80 comp rx=0 tx=0).stop 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 shutdown_connections 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x56369f974a60 0x56369f973770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56369f8f5510 0x56369f8ef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56369f8eac50 0x56369f8ef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 --2- 172.21.15.71:0/3717405984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56369f8ea330 0x56369f90fd00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 >> 172.21.15.71:0/3717405984 conn(0x56369f8e68c0 msgr2=0x56369f8ed9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:32.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f94fbe00640 0 client.15108 ms_handle_reset on v2:172.21.15.174:6838/584492863 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 shutdown_connections 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 1 -- 172.21.15.71:0/3717405984 wait complete. 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.588+0000 7f951fe5b740 20 client.15108 trim_cache size 0 max 16384 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 2, 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24502, 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 116.297773654, 2024-09-05T12:27:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 185.77851582900001 2024-09-05T12:27:32.595 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f94fc0a2c00 msgr2=0x7f94fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9510009a60 tx=0x7f9510005d80 comp rx=0 tx=0).stop 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518101df0 msgr2=0x7f95181983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518101df0 0x7f95181983d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f950800e9f0 tx=0x7f950800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 shutdown_connections 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f94fc0a2c00 0x7f94fc0a50c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9518106f20 0x7f951819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95181027a0 0x7f9518198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 --2- 172.21.15.71:0/1812057270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9518101df0 0x7f95181983d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 >> 172.21.15.71:0/1812057270 conn(0x7f95180fd8b0 msgr2=0x7f95180fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:32.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 shutdown_connections 2024-09-05T12:27:32.597 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:32.589+0000 7f951d200640 1 -- 172.21.15.71:0/1812057270 wait complete. 2024-09-05T12:27:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:32 smithi123 ceph-mon[34321]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:32 smithi174 ceph-mon[34313]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-05T12:27:33.539 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.g status 2024-09-05T12:27:33.906 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:34.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 -- 172.21.15.71:0/208264295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 msgr2=0x7f7e18110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/208264295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e18110cc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009860 tx=0x7f7e1002a340 comp rx=0 tx=0).stop 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 -- 172.21.15.71:0/208264295 shutdown_connections 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/208264295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e18110cc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/208264295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e181049c0 0x7f7e18108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/208264295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e18104010 0x7f7e181043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 -- 172.21.15.71:0/208264295 >> 172.21.15.71:0/208264295 conn(0x7f7e180ffa50 msgr2=0x7f7e18101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 -- 172.21.15.71:0/208264295 shutdown_connections 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.479+0000 7f7e1fc00640 1 -- 172.21.15.71:0/208264295 wait complete. 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.480+0000 7f7e1fc00640 1 Processor -- start 2024-09-05T12:27:34.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.480+0000 7f7e1fc00640 1 -- start start 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1fc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e18104010 0x7f7e1819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1fc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e181049c0 0x7f7e1819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1fc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e1819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1fc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18114150 con 0x7f7e18109140 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1fc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e18113fd0 con 0x7f7e18104010 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1fc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e181142d0 con 0x7f7e181049c0 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e1819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e1819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59016/0 (socket says 172.21.15.71:59016) 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 -- 172.21.15.71:0/459531023 learned_addr learned my addr 172.21.15.71:0/459531023 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:34.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e17e00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e181049c0 0x7f7e1819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1d200640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e18104010 0x7f7e1819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 -- 172.21.15.71:0/459531023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e181049c0 msgr2=0x7f7e1819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e181049c0 0x7f7e1819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 -- 172.21.15.71:0/459531023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e18104010 msgr2=0x7f7e1819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e18104010 0x7f7e1819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.481+0000 7f7e1dc00640 1 -- 172.21.15.71:0/459531023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1819f5e0 con 0x7f7e18109140 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.482+0000 7f7e1dc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e1819ef00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e1002a930 tx=0x7f7e10033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.482+0000 7f7e15600640 1 -- 172.21.15.71:0/459531023 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e10037040 con 0x7f7e18109140 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.482+0000 7f7e15600640 1 -- 172.21.15.71:0/459531023 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e10032770 con 0x7f7e18109140 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.482+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1819f870 con 0x7f7e18109140 2024-09-05T12:27:34.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.482+0000 7f7e15600640 1 -- 172.21.15.71:0/459531023 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e100054d0 con 0x7f7e18109140 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.482+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e181a71b0 con 0x7f7e18109140 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.485+0000 7f7e15600640 1 -- 172.21.15.71:0/459531023 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e10004250 con 0x7f7e18109140 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.485+0000 7f7e2261c740 1 Processor -- start 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.485+0000 7f7e2261c740 1 -- start start 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374661710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x56137466bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x56137466e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e2261c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5613745c70b0 con 0x561374661c50 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e2261c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56137456eab0 con 0x56137466c510 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e2261c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5613745c3070 con 0x561374661330 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e1d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x56137466bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e17e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374661710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e1d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x56137466bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59030/0 (socket says 172.21.15.71:59030) 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e1d200640 1 -- 172.21.15.71:0/4008147953 learned_addr learned my addr 172.21.15.71:0/4008147953 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e17e00640 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x56137466e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.486+0000 7f7e15600640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7e000a2b80 0x7f7e000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e17e00640 1 -- 172.21.15.71:0/4008147953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 msgr2=0x56137466e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e17e00640 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x56137466e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e17e00640 1 -- 172.21.15.71:0/4008147953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 msgr2=0x56137466bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e17e00640 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x56137466bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e15600640 1 -- 172.21.15.71:0/459531023 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7e100ef1d0 con 0x7f7e18109140 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e17e00640 1 -- 172.21.15.71:0/4008147953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137457b4a0 con 0x561374661330 2024-09-05T12:27:34.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e1dc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7e000a2b80 0x7f7e000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e17e00640 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374661710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009690 tx=0x7f7e04009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3478edb20c978493 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:34.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.487+0000 7f7e0a000640 1 -- 172.21.15.71:0/4008147953 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04013cd0 con 0x561374661330 2024-09-05T12:27:34.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e0a000640 1 -- 172.21.15.71:0/4008147953 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e04005500 con 0x561374661330 2024-09-05T12:27:34.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e0a000640 1 -- 172.21.15.71:0/4008147953 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e040117b0 con 0x561374661330 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 -- 172.21.15.71:0/4008147953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 msgr2=0x561374661710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e1dc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7e000a2b80 0x7f7e000a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005820 tx=0x7f7e10005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374661710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009690 tx=0x7f7e04009620 comp rx=0 tx=0).stop 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 -- 172.21.15.71:0/4008147953 shutdown_connections 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x56137466e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x56137466bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 --2- 172.21.15.71:0/4008147953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374661710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 -- 172.21.15.71:0/4008147953 >> 172.21.15.71:0/4008147953 conn(0x56137465d8c0 msgr2=0x56137465dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:34.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.488+0000 7f7e2261c740 1 -- 172.21.15.71:0/4008147953 shutdown_connections 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.489+0000 7f7e2261c740 1 -- 172.21.15.71:0/4008147953 wait complete. 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.489+0000 7f7e2261c740 1 -- start start 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.489+0000 7f7e2261c740 1 Processor -- start 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374669640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661c50 0x561374669b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 0x56137466a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56137466f060 con 0x56137466c510 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56137454f920 con 0x561374661c50 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56137466f1e0 con 0x561374661330 2024-09-05T12:27:34.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e2261c740 10 client.0 _ll_register_callbacks cb 0x5613745738a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:34.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 0x56137466a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374669640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 0x56137466a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59032/0 (socket says 172.21.15.71:59032) 2024-09-05T12:27:34.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 -- 172.21.15.71:0/210127125 learned_addr learned my addr 172.21.15.71:0/210127125 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e1d200640 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661c50 0x561374669b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 -- 172.21.15.71:0/210127125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 msgr2=0x561374669640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374669640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 -- 172.21.15.71:0/210127125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661c50 msgr2=0x561374669b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661c50 0x561374669b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.490+0000 7f7e17e00640 1 -- 172.21.15.71:0/210127125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613743382b0 con 0x56137466c510 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7e17e00640 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 0x56137466a0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e040135a0 tx=0x7f7e04015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d679e02cdcf5407f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7dfea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7dfea00640 1 -- 172.21.15.71:0/210127125 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e04029040 con 0x56137466c510 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7dfea00640 1 -- 172.21.15.71:0/210127125 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7e04005100 con 0x56137466c510 2024-09-05T12:27:34.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e04005500 con 0x56137466c510 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7dfea00640 1 -- 172.21.15.71:0/210127125 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e04005300 con 0x56137466c510 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7e2261c740 10 client.15120 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.491+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56137467aed0 con 0x56137466c510 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7dfea00640 1 -- 172.21.15.71:0/210127125 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e04005500 con 0x56137466c510 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7dfea00640 1 -- 172.21.15.71:0/210127125 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7e040035f0 con 0x56137466c510 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e2261c740 10 client.15120 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e2261c740 10 client.15120 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e2261c740 20 client.15120 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e2261c740 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x56137467d170 0x56137467e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e2261c740 4 client.15120 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 --> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613745d4620 con 0x56137467d170 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.492+0000 7f7e1d200640 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x56137467d170 0x56137467e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.493+0000 7f7e1d200640 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x56137467d170 0x56137467e5e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cf78df44d3343e40 server_cookie=afdd6bcfa42f1a10 in_seq=0 out_seq=0 2024-09-05T12:27:34.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.493+0000 7f7dfea00640 10 client.15120 ms_handle_connect on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.493+0000 7f7dfea00640 1 -- 172.21.15.71:0/210127125 <== mds.1 v2:172.21.15.71:6838/2593934735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613745d4620 con 0x56137467d170 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.493+0000 7f7dfea00640 10 client.15120 handle_command_reply: tid=0 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 client.15120 shutdown 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 msgr2=0x56137466a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 0x56137466a0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7e040135a0 tx=0x7f7e04015040 comp rx=0 tx=0).stop 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 shutdown_connections 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x56137467d170 0x56137467e5e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56137466c510 0x56137466a0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661c50 0x561374669b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 --2- 172.21.15.71:0/210127125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x561374669640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 >> 172.21.15.71:0/210127125 conn(0x56137465d8c0 msgr2=0x5613746635f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:34.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7dfea00640 0 client.15120 ms_handle_reset on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:34.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 shutdown_connections 2024-09-05T12:27:34.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 1 -- 172.21.15.71:0/210127125 wait complete. 2024-09-05T12:27:34.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.494+0000 7f7e2261c740 20 client.15120 trim_cache size 0 max 16384 2024-09-05T12:27:34.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 Processor -- start 2024-09-05T12:27:34.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 -- start start 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661330 0x56137465d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x561374668f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137466c510 0x56137467c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56137467c920 con 0x561374661c50 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56137467cff0 con 0x561374661330 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e2261c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56137467d6c0 con 0x56137466c510 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e1d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x561374668f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e1d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x561374668f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59044/0 (socket says 172.21.15.71:59044) 2024-09-05T12:27:34.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e17e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661330 0x56137465d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e1d200640 1 -- 172.21.15.71:0/2549263081 learned_addr learned my addr 172.21.15.71:0/2549263081 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.513+0000 7f7e17e00640 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137466c510 0x56137467c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e1d200640 1 -- 172.21.15.71:0/2549263081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137466c510 msgr2=0x56137467c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e1d200640 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137466c510 0x56137467c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e1d200640 1 -- 172.21.15.71:0/2549263081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661330 msgr2=0x56137465d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e1d200640 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661330 0x56137465d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e1d200640 1 -- 172.21.15.71:0/2549263081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137466bfb0 con 0x561374661c50 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e17e00640 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137466c510 0x56137467c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e1d200640 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x561374668f00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x56137466ac80 tx=0x7f7e0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bd02eee1243292 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e09600640 1 -- 172.21.15.71:0/2549263081 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c004d20 con 0x561374661c50 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e09600640 1 -- 172.21.15.71:0/2549263081 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c00fd50 con 0x561374661c50 2024-09-05T12:27:34.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.514+0000 7f7e09600640 1 -- 172.21.15.71:0/2549263081 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c01d410 con 0x561374661c50 2024-09-05T12:27:34.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 -- 172.21.15.71:0/2549263081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 msgr2=0x561374668f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x561374668f00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x56137466ac80 tx=0x7f7e0c004a40 comp rx=0 tx=0).stop 2024-09-05T12:27:34.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 -- 172.21.15.71:0/2549263081 shutdown_connections 2024-09-05T12:27:34.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56137466c510 0x56137467c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x561374668f00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2549263081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561374661330 0x56137465d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 -- 172.21.15.71:0/2549263081 >> 172.21.15.71:0/2549263081 conn(0x56137465d8c0 msgr2=0x561374663ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 -- 172.21.15.71:0/2549263081 shutdown_connections 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.515+0000 7f7e2261c740 1 -- 172.21.15.71:0/2549263081 wait complete. 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 -- start start 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 Processor -- start 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x5613746672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x5613746658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x561374665ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56137467e460 con 0x561374661c50 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56137467dd90 con 0x56137466c510 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56137467eb30 con 0x561374661330 2024-09-05T12:27:34.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.516+0000 7f7e2261c740 10 client.0 _ll_register_callbacks cb 0x7f7e04004dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x5613746658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x5613746658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59046/0 (socket says 172.21.15.71:59046) 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 -- 172.21.15.71:0/2383054906 learned_addr learned my addr 172.21.15.71:0/2383054906 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e17e00640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x5613746672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e17e00640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x561374665ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 -- 172.21.15.71:0/2383054906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 msgr2=0x5613746672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x5613746672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 -- 172.21.15.71:0/2383054906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 msgr2=0x561374665ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x561374665ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 -- 172.21.15.71:0/2383054906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613746666f0 con 0x561374661c50 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7e1d200640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x5613746658b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009370 tx=0x7f7e0c0104c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6297d5803228b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.517+0000 7f7dfea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:34.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7dfea00640 1 -- 172.21.15.71:0/2383054906 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c005950 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7dfea00640 1 -- 172.21.15.71:0/2383054906 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f7e0c00f870 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e0c00fd50 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7dfea00640 1 -- 172.21.15.71:0/2383054906 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7e0c00fa70 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 10 client.15126 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561374686960 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7dfea00640 1 -- 172.21.15.71:0/2383054906 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e0c00fd50 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7dfea00640 1 -- 172.21.15.71:0/2383054906 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f7e0c003ae0 con 0x561374661c50 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 10 client.15126 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 10 client.15126 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 20 client.15126 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x5613746eb4c0 0x5613746ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:34.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 4 client.15126 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.518+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 --> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] -- command(tid 0: {"prefix": "status"}) -- 0x5613745ceb30 con 0x5613746eb4c0 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.519+0000 7f7e17e00640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x5613746eb4c0 0x5613746ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.519+0000 7f7e17e00640 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x5613746eb4c0 0x5613746ea1d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2a22b581f1153dc server_cookie=c0356d2ce652aa02 in_seq=0 out_seq=0 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.519+0000 7f7dfea00640 10 client.15126 ms_handle_connect on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.519+0000 7f7dfea00640 1 -- 172.21.15.71:0/2383054906 <== mds.1 v2:172.21.15.71:6838/2593934735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5613745ceb30 con 0x5613746eb4c0 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.519+0000 7f7dfea00640 10 client.15126 handle_command_reply: tid=0 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.519+0000 7f7e2261c740 1 client.15126 shutdown 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 msgr2=0x5613746658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x5613746658b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c009370 tx=0x7f7e0c0104c0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 shutdown_connections 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x5613746eb4c0 0x5613746ea1d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56137466c510 0x561374665ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561374661c50 0x5613746658b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 --2- 172.21.15.71:0/2383054906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561374661330 0x5613746672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 >> 172.21.15.71:0/2383054906 conn(0x56137465d8c0 msgr2=0x561374677430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7dfea00640 0 client.15126 ms_handle_reset on v2:172.21.15.71:6838/2593934735 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 shutdown_connections 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 1 -- 172.21.15.71:0/2383054906 wait complete. 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.520+0000 7f7e2261c740 20 client.15126 trim_cache size 0 max 16384 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:34.528 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 1, 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24514, 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 119.237620017, 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 183.768368197 2024-09-05T12:27:34.529 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:34.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7e000a2b80 msgr2=0x7f7e000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7e000a2b80 0x7f7e000a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7e10005820 tx=0x7f7e10005d60 comp rx=0 tx=0).stop 2024-09-05T12:27:34.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 msgr2=0x7f7e1819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:34.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e1819ef00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7e1002a930 tx=0x7f7e10033d10 comp rx=0 tx=0).stop 2024-09-05T12:27:34.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 shutdown_connections 2024-09-05T12:27:34.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f7e000a2b80 0x7f7e000a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7e18109140 0x7f7e1819ef00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e181049c0 0x7f7e1819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 --2- 172.21.15.71:0/459531023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e18104010 0x7f7e1819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:34.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 >> 172.21.15.71:0/459531023 conn(0x7f7e180ffa50 msgr2=0x7f7e18100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:34.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 shutdown_connections 2024-09-05T12:27:34.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:34.521+0000 7f7e1fc00640 1 -- 172.21.15.71:0/459531023 wait complete. 2024-09-05T12:27:34.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:34 smithi071 ceph-mon[29845]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:34 smithi123 ceph-mon[34321]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:34 smithi174 ceph-mon[34313]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:35.579 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T12:27:35.944 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 -- 172.21.15.71:0/110267922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 msgr2=0x7f4eb0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 --2- 172.21.15.71:0/110267922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 0x7f4eb0103120 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009900 tx=0x7f4ea002a4a0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 -- 172.21.15.71:0/110267922 shutdown_connections 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 --2- 172.21.15.71:0/110267922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eb010c170 0x7f4eb010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 --2- 172.21.15.71:0/110267922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 --2- 172.21.15.71:0/110267922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 0x7f4eb0103120 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.532+0000 7f4eb5600640 1 -- 172.21.15.71:0/110267922 >> 172.21.15.71:0/110267922 conn(0x7f4eb00fc7a0 msgr2=0x7f4eb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.533+0000 7f4eb5600640 1 -- 172.21.15.71:0/110267922 shutdown_connections 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.533+0000 7f4eb5600640 1 -- 172.21.15.71:0/110267922 wait complete. 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 Processor -- start 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 -- start start 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 0x7f4eb0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb01967b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eb010c170 0x7f4eb019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb010fdd0 con 0x7f4eb010c170 2024-09-05T12:27:36.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb010fc50 con 0x7f4eb0103660 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.534+0000 7f4eb5600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb010ff50 con 0x7f4eb0102d40 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb01967b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eb010c170 0x7f4eb019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb01967b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:59434/0 (socket says 172.21.15.71:59434) 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 -- 172.21.15.71:0/2474142296 learned_addr learned my addr 172.21.15.71:0/2474142296 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eaea00640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 0x7f4eb0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 -- 172.21.15.71:0/2474142296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 msgr2=0x7f4eb0196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 0x7f4eb0196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 -- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eb010c170 msgr2=0x7f4eb019ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eb010c170 0x7f4eb019ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 -- 172.21.15.71:0/2474142296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb019b220 con 0x7f4eb0103660 2024-09-05T12:27:36.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.535+0000 7f4eae000640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb01967b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00e9f0 tx=0x7f4e9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.536+0000 7f4ea7400640 1 -- 172.21.15.71:0/2474142296 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00ccc0 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.536+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb019b510 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.536+0000 7f4ea7400640 1 -- 172.21.15.71:0/2474142296 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c00ce60 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.536+0000 7f4ea7400640 1 -- 172.21.15.71:0/2474142296 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c010690 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.536+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb01a2df0 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.539+0000 7f4ea7400640 1 -- 172.21.15.71:0/2474142296 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e9c0046c0 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.540+0000 7f4ea7400640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e940a7010 0x7f4e940a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4ea7400640 1 -- 172.21.15.71:0/2474142296 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f4e9c0cf6e0 con 0x7f4eb0103660 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eaea00640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e940a7010 0x7f4e940a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 Processor -- start 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 -- start start 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227d8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227e3510 0x559b227e5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559b2273e0b0 con 0x559b227d8330 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b226e5ab0 con 0x559b227d8c50 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.541+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b2273a070 con 0x559b227e3510 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eaf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227e3510 0x559b227e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227d8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:59442/0 (socket says 172.21.15.71:59442) 2024-09-05T12:27:36.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 -- 172.21.15.71:0/35083692 learned_addr learned my addr 172.21.15.71:0/35083692 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eaf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227e3510 0x559b227e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45506/0 (socket says 172.21.15.71:45506) 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eaea00640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e940a7010 0x7f4e940a94d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000c00 tx=0x7f4ea0005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 -- 172.21.15.71:0/35083692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227e3510 msgr2=0x559b227e5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227e3510 0x559b227e5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 -- 172.21.15.71:0/35083692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 msgr2=0x559b227d8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227d8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 -- 172.21.15.71:0/35083692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b226f24a0 con 0x559b227d8c50 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eaea00640 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227d8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:36.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.542+0000 7f4eae000640 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e2fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c09bb10 tx=0x7f4e9c09c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b98b4b83ea01bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.543+0000 7f4e9b400640 1 -- 172.21.15.71:0/35083692 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a47f0 con 0x559b227d8c50 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.543+0000 7f4e9b400640 1 -- 172.21.15.71:0/35083692 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c099e80 con 0x559b227d8c50 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.543+0000 7f4e9b400640 1 -- 172.21.15.71:0/35083692 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c09afd0 con 0x559b227d8c50 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.543+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/35083692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 msgr2=0x559b227e2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.543+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e2fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c09bb10 tx=0x7f4e9c09c280 comp rx=0 tx=0).stop 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/35083692 shutdown_connections 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227e3510 0x559b227e5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e2fd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/35083692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227d8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/35083692 >> 172.21.15.71:0/35083692 conn(0x559b227d48c0 msgr2=0x559b227d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/35083692 shutdown_connections 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/35083692 wait complete. 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 -- start start 2024-09-05T12:27:36.574 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.544+0000 7f4eb7fcb740 1 Processor -- start 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 0x559b227e0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227e10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559b227e6060 con 0x559b227e3510 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b226c6920 con 0x559b227d8330 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b227e61e0 con 0x559b227d8c50 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eb7fcb740 10 client.0 _ll_register_callbacks cb 0x559b226ea8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 0x559b227e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eaf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227e10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eae000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 0x559b227e0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:59448/0 (socket says 172.21.15.71:59448) 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.545+0000 7f4eaea00640 1 -- 172.21.15.71:0/3477688061 learned_addr learned my addr 172.21.15.71:0/3477688061 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:36.575 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eae000640 1 -- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 msgr2=0x559b227e0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eae000640 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 0x559b227e0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eae000640 1 -- 172.21.15.71:0/3477688061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 msgr2=0x559b227e10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eae000640 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227e10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eae000640 1 -- 172.21.15.71:0/3477688061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b224af2b0 con 0x559b227d8c50 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eaea00640 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 0x559b227e0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eae000640 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227e0b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x559b227d9240 tx=0x7f4e9c0a48e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3222d47050fd495 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4e8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3477688061 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c07e020 con 0x559b227d8c50 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3477688061 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f4e9c0a4360 con 0x559b227d8c50 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.546+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c099e80 con 0x559b227d8c50 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3477688061 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e9c0ba050 con 0x559b227d8c50 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 10 client.25186 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:36.576 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b227e6060 con 0x559b227d8c50 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3477688061 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c099e80 con 0x559b227d8c50 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3477688061 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4e9c0ba310 con 0x559b227d8c50 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 10 client.25186 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 10 client.25186 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 20 client.25186 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b227f2fc0 0x559b227f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 4 client.25186 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.547+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b2274b620 con 0x559b227f2fc0 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.548+0000 7f4eaea00640 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b227f2fc0 0x559b227f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.549+0000 7f4e8fe00640 10 client.25186 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.549+0000 7f4eaea00640 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b227f2fc0 0x559b227f5180 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c049e351e4b1bfc9 server_cookie=e23fa1bcf62bf032 in_seq=0 out_seq=0 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.549+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3477688061 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559b2274b620 con 0x559b227f2fc0 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.549+0000 7f4e8fe00640 10 client.25186 handle_command_reply: tid=0 2024-09-05T12:27:36.577 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 client.25186 shutdown 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 msgr2=0x559b227e0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227e0b80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x559b227d9240 tx=0x7f4e9c0a48e0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 shutdown_connections 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b227f2fc0 0x559b227f5180 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227e10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227e0b80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3477688061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8330 0x559b227e0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4e8fe00640 0 client.25186 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 >> 172.21.15.71:0/3477688061 conn(0x559b227d48c0 msgr2=0x559b227da5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 shutdown_connections 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3477688061 wait complete. 2024-09-05T12:27:36.578 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.550+0000 7f4eb7fcb740 20 client.25186 trim_cache size 0 max 16384 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.569+0000 7f4eb7fcb740 1 Processor -- start 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.569+0000 7f4eb7fcb740 1 -- start start 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.569+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8330 0x559b227d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227dd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559b227f34c0 con 0x559b227e3510 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b227f3b90 con 0x559b227d8c50 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b227f4260 con 0x559b227d8330 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8330 0x559b227d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8330 0x559b227d4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45528/0 (socket says 172.21.15.71:45528) 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:59456/0 (socket says 172.21.15.71:59456) 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eaea00640 1 -- 172.21.15.71:0/1200248810 learned_addr learned my addr 172.21.15.71:0/1200248810 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eaf400640 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227dd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 -- 172.21.15.71:0/1200248810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8330 msgr2=0x559b227d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8330 0x559b227d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 -- 172.21.15.71:0/1200248810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 msgr2=0x559b227dd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227dd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eae000640 1 -- 172.21.15.71:0/1200248810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b227e6060 con 0x559b227d8c50 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.570+0000 7f4eaf400640 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227dd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.571+0000 7f4eae000640 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e0f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c020000 tx=0x7f4e9c09c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bd15f760e604e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:36.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.571+0000 7f4e9aa00640 1 -- 172.21.15.71:0/1200248810 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c07e070 con 0x559b227d8c50 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.571+0000 7f4e9aa00640 1 -- 172.21.15.71:0/1200248810 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c099e80 con 0x559b227d8c50 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.571+0000 7f4e9aa00640 1 -- 172.21.15.71:0/1200248810 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c0a45c0 con 0x559b227d8c50 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/1200248810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 msgr2=0x559b227e0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e0f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c020000 tx=0x7f4e9c09c280 comp rx=0 tx=0).stop 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/1200248810 shutdown_connections 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227e3510 0x559b227dd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227d8c50 0x559b227e0f10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/1200248810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8330 0x559b227d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/1200248810 >> 172.21.15.71:0/1200248810 conn(0x559b227d48c0 msgr2=0x559b227df770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/1200248810 shutdown_connections 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/1200248810 wait complete. 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.572+0000 7f4eb7fcb740 1 -- start start 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 Processor -- start 2024-09-05T12:27:36.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227fd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227e3510 0x559b227dd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559b227f5000 con 0x559b227d8330 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b227f4930 con 0x559b227e3510 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b227f56d0 con 0x559b227d8c50 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.573+0000 7f4eb7fcb740 10 client.0 _ll_register_callbacks cb 0x7f4e9c0a47f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227fd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227fd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59096/0 (socket says 172.21.15.71:59096) 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 -- 172.21.15.71:0/3183852117 learned_addr learned my addr 172.21.15.71:0/3183852117 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eae000640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaf400640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227e3510 0x559b227dd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 -- 172.21.15.71:0/3183852117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 msgr2=0x559b227dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227dd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 -- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227e3510 msgr2=0x559b227dd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227e3510 0x559b227dd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.574+0000 7f4eaea00640 1 -- 172.21.15.71:0/3183852117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b227ddd10 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4eaea00640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227fd930 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009900 tx=0x7f4ea002a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cdd92591a03e69 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4e8fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3183852117 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0035040 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3183852117 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f4ea0004ad0 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e9c099e80 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3183852117 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ea0065050 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4eb7fcb740 10 client.15144 fetch_fsmap learned FSMap version 29 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.575+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b227fc230 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3183852117 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e9c099e80 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3183852117 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f4ea0032550 con 0x559b227d8330 2024-09-05T12:27:36.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eb7fcb740 10 client.15144 fetch_fsmap finished waiting for FSMap version 29 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eb7fcb740 10 client.15144 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eb7fcb740 20 client.15144 populate_metadata read hostname 'smithi071' 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b22862da0 0x559b22861ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eb7fcb740 4 client.15144 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "status"}) -- 0x559b22745b30 con 0x559b22862da0 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.576+0000 7f4eae000640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b22862da0 0x559b22861ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eae000640 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b22862da0 0x559b22861ab0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe847b355d6593d server_cookie=9540d73e52e59835 in_seq=0 out_seq=0 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4e8fe00640 10 client.15144 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4e8fe00640 1 -- 172.21.15.71:0/3183852117 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559b22745b30 con 0x559b22862da0 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4e8fe00640 10 client.15144 handle_command_reply: tid=0 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 client.15144 shutdown 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 msgr2=0x559b227fd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227fd930 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009900 tx=0x7f4ea002a4a0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 shutdown_connections 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559b22862da0 0x559b22861ab0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b227e3510 0x559b227dd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b227d8c50 0x559b227dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 --2- 172.21.15.71:0/3183852117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b227d8330 0x559b227fd930 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 >> 172.21.15.71:0/3183852117 conn(0x559b227d48c0 msgr2=0x559b227d9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4e8fe00640 0 client.15144 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.577+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 shutdown_connections 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.578+0000 7f4eb7fcb740 1 -- 172.21.15.71:0/3183852117 wait complete. 2024-09-05T12:27:36.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.578+0000 7f4eb7fcb740 20 client.15144 trim_cache size 0 max 16384 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24524, 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 166.070476386, 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 29, 2024-09-05T12:27:36.590 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 94, 2024-09-05T12:27:36.591 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:27:36.591 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 179.64669565299999 2024-09-05T12:27:36.591 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.578+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e940a7010 msgr2=0x7f4e940a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.578+0000 7f4eb5600640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e940a7010 0x7f4e940a94d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0000c00 tx=0x7f4ea0005e30 comp rx=0 tx=0).stop 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.578+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 msgr2=0x7f4eb01967b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.578+0000 7f4eb5600640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb01967b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00e9f0 tx=0x7f4e9c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 shutdown_connections 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f4e940a7010 0x7f4e940a94d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4eb010c170 0x7f4eb019ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4eb0103660 0x7f4eb01967b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 --2- 172.21.15.71:0/2474142296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4eb0102d40 0x7f4eb0196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 >> 172.21.15.71:0/2474142296 conn(0x7f4eb00fc7a0 msgr2=0x7f4eb00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:36.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 shutdown_connections 2024-09-05T12:27:36.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:36.579+0000 7f4eb5600640 1 -- 172.21.15.71:0/2474142296 wait complete. 2024-09-05T12:27:36.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:36 smithi071 ceph-mon[29845]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:27:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:36 smithi123 ceph-mon[34321]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:27:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:36 smithi174 ceph-mon[34313]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-05T12:27:37.521 INFO:teuthology.run_tasks:Running task sequential... 2024-09-05T12:27:37.532 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-05T12:27:37.532 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-05T12:27:37.532 INFO:teuthology.task.print:Enabling mgr modules 2024-09-05T12:27:37.533 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-05T12:27:37.533 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T12:27:37.533 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:27:37.534 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 -- 172.21.15.71:0/1136587741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac104f50 msgr2=0x7f54ac1053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 --2- 172.21.15.71:0/1136587741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac104f50 0x7f54ac1053d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5498009900 tx=0x7f549802a410 comp rx=0 tx=0).stop 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 -- 172.21.15.71:0/1136587741 shutdown_connections 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 --2- 172.21.15.71:0/1136587741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54ac105910 0x7f54ac10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 --2- 172.21.15.71:0/1136587741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac104f50 0x7f54ac1053d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 --2- 172.21.15.71:0/1136587741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac103d60 0x7f54ac104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:37.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.685+0000 7f54b1a00640 1 -- 172.21.15.71:0/1136587741 >> 172.21.15.71:0/1136587741 conn(0x7f54ac0ff4e0 msgr2=0x7f54ac101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:37.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.686+0000 7f54b1a00640 1 -- 172.21.15.71:0/1136587741 shutdown_connections 2024-09-05T12:27:37.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.686+0000 7f54b1a00640 1 -- 172.21.15.71:0/1136587741 wait complete. 2024-09-05T12:27:37.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.686+0000 7f54b1a00640 1 Processor -- start 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 -- start start 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 0x7f54ac199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac104f50 0x7f54ac19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54ac105910 0x7f54ac1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac10fa30 con 0x7f54ac103d60 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac10f8b0 con 0x7f54ac104f50 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54b1a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac10fbb0 con 0x7f54ac105910 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54aaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 0x7f54ac199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54aaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 0x7f54ac199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59114/0 (socket says 172.21.15.71:59114) 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54aaa00640 1 -- 172.21.15.71:0/3534829411 learned_addr learned my addr 172.21.15.71:0/3534829411 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54aa000640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac104f50 0x7f54ac19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:37.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.687+0000 7f54ab400640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54ac105910 0x7f54ac1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aa000640 1 -- 172.21.15.71:0/3534829411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54ac105910 msgr2=0x7f54ac1a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aa000640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54ac105910 0x7f54ac1a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aa000640 1 -- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 msgr2=0x7f54ac199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aa000640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 0x7f54ac199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aa000640 1 -- 172.21.15.71:0/3534829411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54ac1a1b80 con 0x7f54ac104f50 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aaa00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 0x7f54ac199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54aa000640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac104f50 0x7f54ac19a4a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f549802a930 tx=0x7f5498032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549803f030 con 0x7f54ac104f50 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54ac1a1e10 con 0x7f54ac104f50 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5498003ce0 con 0x7f54ac104f50 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498035070 con 0x7f54ac104f50 2024-09-05T12:27:37.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.688+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54ac1a2280 con 0x7f54ac104f50 2024-09-05T12:27:37.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.689+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54ac10b020 con 0x7f54ac104f50 2024-09-05T12:27:37.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.691+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5498042030 con 0x7f54ac104f50 2024-09-05T12:27:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.692+0000 7f54a3400640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f54900a2c30 0x7f54900a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.693+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f54980efb40 con 0x7f54ac104f50 2024-09-05T12:27:37.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.693+0000 7f54aaa00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f54900a2c30 0x7f54900a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:37.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.693+0000 7f54aaa00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f54900a2c30 0x7f54900a50f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f549c0058c0 tx=0x7f549c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:37.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.697+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54980bc090 con 0x7f54ac104f50 2024-09-05T12:27:37.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:37.917+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f54ac02d7c0 con 0x7f54ac104f50 2024-09-05T12:27:38.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.361+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f549803b460 con 0x7f54ac104f50 2024-09-05T12:27:38.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.362+0000 7f54a3400640 1 -- 172.21.15.71:0/3534829411 <== mon.2 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f54980e4690 con 0x7f54ac104f50 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f54900a2c30 msgr2=0x7f54900a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f54900a2c30 0x7f54900a50f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f549c0058c0 tx=0x7f549c005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac104f50 msgr2=0x7f54ac19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac104f50 0x7f54ac19a4a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f549802a930 tx=0x7f5498032ce0 comp rx=0 tx=0).stop 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 shutdown_connections 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f54900a2c30 0x7f54900a50f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54ac105910 0x7f54ac1a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54ac104f50 0x7f54ac19a4a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:38.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 --2- 172.21.15.71:0/3534829411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54ac103d60 0x7f54ac199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:38.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 >> 172.21.15.71:0/3534829411 conn(0x7f54ac0ff4e0 msgr2=0x7f54ac1081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:38.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 shutdown_connections 2024-09-05T12:27:38.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:38.365+0000 7f54b1a00640 1 -- 172.21.15.71:0/3534829411 wait complete. 2024-09-05T12:27:38.386 INFO:teuthology.run_tasks:Running task kclient... 2024-09-05T12:27:38.400 INFO:tasks.kclient:Mounting kernel clients... 2024-09-05T12:27:38.400 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-05T12:27:38.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:38.400 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-05T12:27:38.436 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-05T12:27:38.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:38.437 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-05T12:27:38.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:38.537 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-09-05T12:27:38.612 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-mon[29845]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:38.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3534829411' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T12:27:38.613 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T12:27:38.613 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ignoring --setuser ceph since I am not root 2024-09-05T12:27:38.613 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ignoring --setgroup ceph since I am not root 2024-09-05T12:27:38.613 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:38.606+0000 7f0b3862d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:27:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-mon[34321]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3534829411' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T12:27:38.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T12:27:38.680 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: ignoring --setuser ceph since I am not root 2024-09-05T12:27:38.680 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: ignoring --setgroup ceph since I am not root 2024-09-05T12:27:38.680 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-mgr[35689]: -- 172.21.15.123:0/571740270 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fddc5b90e0 con 0x55fddc597000 2024-09-05T12:27:38.680 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:38.504+0000 7f8473a23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:27:38.681 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:38.562+0000 7f8473a23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:27:38.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-mon[34313]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 368 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:27:38.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3534829411' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T12:27:38.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-05T12:27:38.711 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: ignoring --setuser ceph since I am not root 2024-09-05T12:27:38.711 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: ignoring --setgroup ceph since I am not root 2024-09-05T12:27:38.711 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-mgr[36130]: -- 172.21.15.174:0/1698853576 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557fcf5cf0e0 con 0x557fcf5ad000 2024-09-05T12:27:38.712 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:38.503+0000 7f4f2b9d2040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-05T12:27:38.712 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:38.560+0000 7f4f2b9d2040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:27:38.712 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:38.709+0000 7f4f2b9d2040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:27:38.866 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:38.677+0000 7f0b3862d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-05T12:27:38.866 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:38.863+0000 7f0b3862d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:27:38.913 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-05T12:27:38.913 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-05T12:27:38.913 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T12:27:38.913 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs ls 2024-09-05T12:27:39.010 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:38.776+0000 7f4f2b9d2040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:27:39.010 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:38.921+0000 7f4f2b9d2040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:27:39.067 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:38.712+0000 7f8473a23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-05T12:27:39.067 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:38.780+0000 7f8473a23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:27:39.067 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:38.926+0000 7f8473a23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:27:39.068 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:39.066+0000 7f8473a23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:27:39.129 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:38.947+0000 7f0b3862d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-05T12:27:39.129 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:39.127+0000 7f0b3862d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:27:39.294 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:39.353 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:39.061+0000 7f4f2b9d2040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:27:39.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-05T12:27:39.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-mon[29845]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-05T12:27:39.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:39.304+0000 7f0b3862d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-05T12:27:39.616 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:39 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-05T12:27:39.616 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:39 smithi174 ceph-mon[34313]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-05T12:27:39.617 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:39.370+0000 7f4f2b9d2040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:27:39.617 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:39.431+0000 7f4f2b9d2040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:27:39.623 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:39 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-05T12:27:39.623 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:39 smithi123 ceph-mon[34321]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-05T12:27:39.623 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:39.378+0000 7f8473a23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:27:39.623 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:39.438+0000 7f8473a23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:27:39.623 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:39.622+0000 7f8473a23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:27:39.961 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:39.684+0000 7f0b3862d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-05T12:27:39.961 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:39.754+0000 7f0b3862d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-05T12:27:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.958+0000 7f42b0c00640 1 -- 172.21.15.71:0/2819052782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac103ff0 msgr2=0x7f42ac1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.958+0000 7f42b0c00640 1 --2- 172.21.15.71:0/2819052782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac103ff0 0x7f42ac1043d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f429c009900 tx=0x7f429c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:27:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 -- 172.21.15.71:0/2819052782 shutdown_connections 2024-09-05T12:27:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 --2- 172.21.15.71:0/2819052782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac109120 0x7f42ac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:39.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 --2- 172.21.15.71:0/2819052782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ac1049a0 0x7f42ac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 --2- 172.21.15.71:0/2819052782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac103ff0 0x7f42ac1043d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 -- 172.21.15.71:0/2819052782 >> 172.21.15.71:0/2819052782 conn(0x7f42ac0ffa90 msgr2=0x7f42ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 -- 172.21.15.71:0/2819052782 shutdown_connections 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.959+0000 7f42b0c00640 1 -- 172.21.15.71:0/2819052782 wait complete. 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.960+0000 7f42b0c00640 1 Processor -- start 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.960+0000 7f42b0c00640 1 -- start start 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42b0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac103ff0 0x7f42ac19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42b0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 0x7f42ac19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42b0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ac109120 0x7f42ac19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42b0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac114120 con 0x7f42ac103ff0 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42b0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac113fa0 con 0x7f42ac109120 2024-09-05T12:27:39.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42b0c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42ac1142a0 con 0x7f42ac1049a0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42a9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 0x7f42ac19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42a9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac103ff0 0x7f42ac19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42a9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 0x7f42ac19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45584/0 (socket says 172.21.15.71:45584) 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42a9400640 1 -- 172.21.15.71:0/3693018286 learned_addr learned my addr 172.21.15.71:0/3693018286 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42aa800640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ac109120 0x7f42ac19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42a9400640 1 -- 172.21.15.71:0/3693018286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ac109120 msgr2=0x7f42ac19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.961+0000 7f42a9400640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ac109120 0x7f42ac19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a9400640 1 -- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac103ff0 msgr2=0x7f42ac19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a9400640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac103ff0 0x7f42ac19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a9400640 1 -- 172.21.15.71:0/3693018286 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac19f5b0 con 0x7f42ac1049a0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a9e00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac103ff0 0x7f42ac19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a9400640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 0x7f42ac19ab40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f429800e9f0 tx=0x7f429800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f429800cd80 con 0x7f42ac1049a0 2024-09-05T12:27:39.966 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4298010910 con 0x7f42ac1049a0 2024-09-05T12:27:39.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42ac19f8a0 con 0x7f42ac1049a0 2024-09-05T12:27:39.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42ac1a7180 con 0x7f42ac1049a0 2024-09-05T12:27:39.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.962+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4298018990 con 0x7f42ac1049a0 2024-09-05T12:27:39.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.963+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c005290 con 0x7f42ac1049a0 2024-09-05T12:27:39.967 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.966+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f4298018b30 con 0x7f42ac1049a0 2024-09-05T12:27:39.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.968+0000 7f42a2a00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 0x7f42900a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:40.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.968+0000 7f42a9e00640 1 -- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 msgr2=0x7f42900a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2050574497 2024-09-05T12:27:40.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.969+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f42980cf820 con 0x7f42ac1049a0 2024-09-05T12:27:40.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.969+0000 7f42a9e00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 0x7f42900a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:27:40.144 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:39.615+0000 7f4f2b9d2040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:27:40.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:39.969+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f429809c3c0 con 0x7f42ac1049a0 2024-09-05T12:27:40.171 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.170+0000 7f42a9e00640 1 -- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 msgr2=0x7f42900a5110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2050574497 2024-09-05T12:27:40.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.170+0000 7f42a9e00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 0x7f42900a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T12:27:40.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.187+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f428c005850 con 0x7f42ac1049a0 2024-09-05T12:27:40.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.188+0000 7f42a2a00640 1 -- 172.21.15.71:0/3693018286 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f4298018e20 con 0x7f42ac1049a0 2024-09-05T12:27:40.190 INFO:teuthology.orchestra.run.smithi071.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 msgr2=0x7f42900a5110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 0x7f42900a5110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 msgr2=0x7f42ac19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 0x7f42ac19ab40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f429800e9f0 tx=0x7f429800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 shutdown_connections 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f42900a2c50 0x7f42900a5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42ac109120 0x7f42ac19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42ac1049a0 0x7f42ac19ab40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 --2- 172.21.15.71:0/3693018286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42ac103ff0 0x7f42ac19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:40.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 >> 172.21.15.71:0/3693018286 conn(0x7f42ac0ffa90 msgr2=0x7f42ac100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:40.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 shutdown_connections 2024-09-05T12:27:40.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:40.191+0000 7f42b0c00640 1 -- 172.21.15.71:0/3693018286 wait complete. 2024-09-05T12:27:40.259 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:39.975+0000 7f0b3862d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-05T12:27:40.637 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3693018286' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T12:27:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3693018286' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T12:27:40.679 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:40.501+0000 7f8473a23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:27:40.679 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:40.568+0000 7f8473a23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:27:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3693018286' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-05T12:27:40.760 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:40.493+0000 7f4f2b9d2040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:27:40.760 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:40.560+0000 7f4f2b9d2040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:27:41.087 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-05T12:27:41.087 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi071.front.sepia.ceph.com 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-05T12:27:41.088 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-05T12:27:41.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:41.088 DEBUG:teuthology.orchestra.run.smithi071:> ip addr 2024-09-05T12:27:41.128 INFO:teuthology.orchestra.run.smithi071.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-05T12:27:41.128 INFO:teuthology.orchestra.run.smithi071.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: inet 127.0.0.1/8 scope host lo 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 ::1/128 scope host 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4a brd ff:ff:ff:ff:ff:ff 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f0 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:d8:aa:4b brd ff:ff:ff:ff:ff:ff 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: altname enp7s0f1 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:64 brd ff:ff:ff:ff:ff:ff 2024-09-05T12:27:41.129 INFO:teuthology.orchestra.run.smithi071.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-05T12:27:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: link/ether 0c:c4:7a:8f:d7:65 brd ff:ff:ff:ff:ff:ff 2024-09-05T12:27:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: inet 172.21.15.71/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-05T12:27:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft 42079sec preferred_lft 42079sec 2024-09-05T12:27:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: inet6 fe80::ec4:7aff:fe8f:d765/64 scope link noprefixroute 2024-09-05T12:27:41.130 INFO:teuthology.orchestra.run.smithi071.stdout: valid_lft forever preferred_lft forever 2024-09-05T12:27:41.130 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-05T12:27:41.130 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:27:41.130 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:27:41.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add name ceph-brx type bridge 2024-09-05T12:27:41.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr flush dev ceph-brx 2024-09-05T12:27:41.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set ceph-brx up 2024-09-05T12:27:41.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-05T12:27:41.131 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:27:41.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:27:41.295 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:41.128+0000 7f0b3862d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-05T12:27:41.295 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:41.214+0000 7f0b3862d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-05T12:27:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:41.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:41.364 DEBUG:teuthology.orchestra.run.smithi071:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-05T12:27:41.421 INFO:teuthology.orchestra.run.smithi071.stdout:1 2024-09-05T12:27:41.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:41.424 DEBUG:teuthology.orchestra.run.smithi071:> route 2024-09-05T12:27:41.462 INFO:teuthology.orchestra.run.smithi071.stdout:Kernel IP routing table 2024-09-05T12:27:41.462 INFO:teuthology.orchestra.run.smithi071.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-05T12:27:41.462 INFO:teuthology.orchestra.run.smithi071.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-05T12:27:41.462 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-05T12:27:41.462 INFO:teuthology.orchestra.run.smithi071.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-05T12:27:41.463 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:27:41.463 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:27:41.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-05T12:27:41.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-05T12:27:41.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-05T12:27:41.463 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:27:41.525 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.265+0000 7f4f2b9d2040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:27:41.525 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.381+0000 7f4f2b9d2040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:27:41.539 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.275+0000 7f8473a23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:27:41.539 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.393+0000 7f8473a23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:27:41.582 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:27:41.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:41.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:41.664 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-05T12:27:41.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:41.700 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list-id 2024-09-05T12:27:41.776 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:27:41.776 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:27:41.776 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T12:27:41.777 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-05T12:27:41.777 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:27:41.799 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.524+0000 7f4f2b9d2040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:27:41.800 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.584+0000 7f4f2b9d2040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:27:41.800 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.731+0000 7f4f2b9d2040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:27:41.800 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.798+0000 7f4f2b9d2040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:27:41.815 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.537+0000 7f8473a23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:27:41.815 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.598+0000 7f8473a23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:27:41.816 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.746+0000 7f8473a23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:27:41.816 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.814+0000 7f8473a23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:27:41.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:27:41.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:41.957 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-05T12:27:41.957 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:27:41.957 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:27:41.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-05T12:27:41.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-05T12:27:41.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-05T12:27:41.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-05T12:27:41.958 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-05T12:27:41.958 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:27:42.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:27:42.080 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:41.858+0000 7f4f2b9d2040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:27:42.081 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:42.014+0000 7f4f2b9d2040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:27:42.081 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:42.079+0000 7f4f2b9d2040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:27:42.100 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:41.873+0000 7f8473a23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:27:42.101 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:42.030+0000 7f8473a23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:27:42.101 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:42.097+0000 7f8473a23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:27:42.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:42.182 DEBUG:teuthology.orchestra.run.smithi071:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-05T12:27:42.182 DEBUG:teuthology.orchestra.run.smithi071:> set -e 2024-09-05T12:27:42.182 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set brx.0 up 2024-09-05T12:27:42.182 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link set dev brx.0 master ceph-brx 2024-09-05T12:27:42.182 DEBUG:teuthology.orchestra.run.smithi071:> ') 2024-09-05T12:27:42.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:27:42.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:42.346 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:42.145+0000 7f4f2b9d2040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:27:42.346 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:42.226+0000 7f4f2b9d2040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:27:42.346 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:42.285+0000 7f4f2b9d2040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:27:42.346 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:42.344+0000 7f4f2b9d2040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:27:42.354 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T12:27:42.354 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-05T12:27:42.365 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:42.162+0000 7f8473a23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:27:42.365 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:42.244+0000 7f8473a23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:27:42.365 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:42.304+0000 7f8473a23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:27:42.366 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:42.364+0000 7f8473a23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:27:42.390 INFO:teuthology.orchestra.run.smithi071.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-05T12:27:42.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-05T12:27:42.391 DEBUG:teuthology.orchestra.run.smithi071:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-05T12:27:42.466 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-05T12:27:42.466 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-05T12:27:42.466 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/58aebd0b-d33c-4637-a6a0-e71d9cc8243f 2024-09-05T12:27:42.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:42.466 DEBUG:teuthology.orchestra.run.smithi071:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/58aebd0b-d33c-4637-a6a0-e71d9cc8243f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-05T12:27:42.513 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.190+0000 7f0b3862d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-05T12:27:42.513 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.335+0000 7f0b3862d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-05T12:27:42.597 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-05T12:27:42.597 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-05T12:27:42.767 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.508+0000 7f0b3862d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-05T12:27:42.768 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.585+0000 7f0b3862d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-05T12:27:42.768 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.765+0000 7f0b3862d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-05T12:27:43.112 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.849+0000 7f0b3862d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-05T12:27:43.112 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:42.921+0000 7f0b3862d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-05T12:27:43.112 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:43.109+0000 7f0b3862d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-05T12:27:43.236 INFO:teuthology.orchestra.run.smithi071.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,nowsync,nofallback 2024-09-05T12:27:43.236 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync". 2024-09-05T12:27:43.236 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: resolved to: "172.21.15.71:6789,172.21.15.123:6789,172.21.15.174:6789" 2024-09-05T12:27:43.236 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: trying mount with new device syntax: 0@8c9ae41c-6b80-11ef-bcd6-c7b262605968.cephfs=/volumes/qa/sv_1/58aebd0b-d33c-4637-a6a0-e71d9cc8243f 2024-09-05T12:27:43.236 INFO:teuthology.orchestra.run.smithi071.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,nowsync,name=0,key=0,mon_addr=172.21.15.71:6789/172.21.15.123:6789/172.21.15.174:6789" will pass to kernel 2024-09-05T12:27:43.238 INFO:teuthology.orchestra.run.smithi071.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-05T12:27:43.238 INFO:teuthology.orchestra.run.smithi071.stdout: You just mounted a file system that supports labels which does not 2024-09-05T12:27:43.238 INFO:teuthology.orchestra.run.smithi071.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-05T12:27:43.238 INFO:teuthology.orchestra.run.smithi071.stdout: applications will generate AVC messages and not be allowed access to 2024-09-05T12:27:43.238 INFO:teuthology.orchestra.run.smithi071.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-05T12:27:43.241 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-05T12:27:43.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:27:43.242 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-05T12:27:43.306 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-05T12:27:43.384 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:43.193+0000 7f0b3862d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-05T12:27:43.384 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:43.278+0000 7f0b3862d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-05T12:27:43.384 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:43.382+0000 7f0b3862d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-05T12:27:43.384 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: Standby manager daemon z restarted 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: Standby manager daemon z started 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: Standby manager daemon y restarted 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: Standby manager daemon y started 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T12:27:43.385 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-mon[29845]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:27:43.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T12:27:43.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:43.400 INFO:teuthology.orchestra.run.smithi071.stdout:client15174 2024-09-05T12:27:43.400 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:27:43.400 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:27:43.429 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y[35685]: 2024-09-05T12:27:43.151+0000 7f8473a23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: Standby manager daemon z restarted 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: Standby manager daemon z started 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: Standby manager daemon y restarted 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: Standby manager daemon y started 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T12:27:43.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:43 smithi123 ceph-mon[34321]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:27:43.437 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z[36126]: 2024-09-05T12:27:43.124+0000 7f4f2b9d2040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: Standby manager daemon z restarted 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: Standby manager daemon z started 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.174:0/310212018' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:27:43.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: Standby manager daemon y restarted 2024-09-05T12:27:43.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: Standby manager daemon y started 2024-09-05T12:27:43.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-05T12:27:43.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-05T12:27:43.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-05T12:27:43.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:43 smithi174 ceph-mon[34313]: from='mgr.? 172.21.15.123:0/3465108202' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-05T12:27:43.559 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T12:27:43.562 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:43.571 INFO:teuthology.orchestra.run.smithi071.stdout:client15174 2024-09-05T12:27:43.572 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-05T12:27:43.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-05T12:27:43.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:43.660 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:43.456+0000 7f0b3862d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-05T12:27:43.660 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:43.531+0000 7f0b3862d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-05T12:27:43.662 INFO:teuthology.orchestra.run.smithi071.stdout:client15174 2024-09-05T12:27:43.663 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968.client15174/status') 2024-09-05T12:27:43.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-05T12:27:43.763 INFO:teuthology.orchestra.run.smithi071.stdout:instance: client.15174 (3)192.168.0.1:0/3638947263 2024-09-05T12:27:43.764 INFO:teuthology.orchestra.run.smithi071.stdout:blocklisted: false 2024-09-05T12:27:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records in 2024-09-05T12:27:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:0+1 records out 2024-09-05T12:27:43.764 INFO:teuthology.orchestra.run.smithi071.stderr:70 bytes copied, 6.9877e-05 s, 1.0 MB/s 2024-09-05T12:27:43.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-05T12:27:43.775 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T12:27:43.787 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T12:27:43.787 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:27:43.788 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-05T12:27:43.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 -- 172.21.15.71:0/1517804707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8105910 msgr2=0x7f22f810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 --2- 172.21.15.71:0/1517804707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8105910 0x7f22f810c1a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009be0 tx=0x7f22f002a6c0 comp rx=0 tx=0).stop 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 -- 172.21.15.71:0/1517804707 shutdown_connections 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 --2- 172.21.15.71:0/1517804707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8105910 0x7f22f810c1a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 --2- 172.21.15.71:0/1517804707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 --2- 172.21.15.71:0/1517804707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22f8103d60 0x7f22f8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 -- 172.21.15.71:0/1517804707 >> 172.21.15.71:0/1517804707 conn(0x7f22f80ff4e0 msgr2=0x7f22f8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 -- 172.21.15.71:0/1517804707 shutdown_connections 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.939+0000 7f22ffe00640 1 -- 172.21.15.71:0/1517804707 wait complete. 2024-09-05T12:27:43.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 Processor -- start 2024-09-05T12:27:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 -- start start 2024-09-05T12:27:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8103d60 0x7f22f8199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:43.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f819a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22f8105910 0x7f22f81a1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810f9e0 con 0x7f22f8104f50 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810f860 con 0x7f22f8103d60 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.940+0000 7f22ffe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22f810fb60 con 0x7f22f8105910 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f819a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8103d60 0x7f22f8199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f819a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41866/0 (socket says 172.21.15.71:41866) 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 -- 172.21.15.71:0/275395609 learned_addr learned my addr 172.21.15.71:0/275395609 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fde00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22f8105910 0x7f22f81a1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 -- 172.21.15.71:0/275395609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22f8105910 msgr2=0x7f22f81a1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22f8105910 0x7f22f81a1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:43.945 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 -- 172.21.15.71:0/275395609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8103d60 msgr2=0x7f22f8199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8103d60 0x7f22f8199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 -- 172.21.15.71:0/275395609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f81a1ba0 con 0x7f22f8104f50 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.941+0000 7f22fca00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f819a4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f22e400e9c0 tx=0x7f22e400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.942+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e400ccc0 con 0x7f22f8104f50 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.942+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22e4005500 con 0x7f22f8104f50 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.942+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f81a1e90 con 0x7f22f8104f50 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.942+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e4010430 con 0x7f22f8104f50 2024-09-05T12:27:43.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.942+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f81a2350 con 0x7f22f8104f50 2024-09-05T12:27:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.943+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f810b020 con 0x7f22f8104f50 2024-09-05T12:27:43.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.945+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7f22e40046c0 con 0x7f22f8104f50 2024-09-05T12:27:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.946+0000 7f22f6000640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 0x7f22e00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.946+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7f22e40cf6d0 con 0x7f22f8104f50 2024-09-05T12:27:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.946+0000 7f22fd400640 1 -- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 msgr2=0x7f22e00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2050574497 2024-09-05T12:27:43.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.946+0000 7f22fd400640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 0x7f22e00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:27:43.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:43.950+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22e409c2d0 con 0x7f22f8104f50 2024-09-05T12:27:44.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.147+0000 7f22fd400640 1 -- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 msgr2=0x7f22e00a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2050574497 2024-09-05T12:27:44.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.147+0000 7f22fd400640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 0x7f22e00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-05T12:27:44.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.166+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f22f8062e90 con 0x7f22f8104f50 2024-09-05T12:27:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.197+0000 7f22f6000640 1 -- 172.21.15.71:0/275395609 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7f22e409bbd0 con 0x7f22f8104f50 2024-09-05T12:27:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.199+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 msgr2=0x7f22e00a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.199+0000 7f22ffe00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 0x7f22e00a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 msgr2=0x7f22f819a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f819a4c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f22e400e9c0 tx=0x7f22e400ee80 comp rx=0 tx=0).stop 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 shutdown_connections 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7f22e00a2c80 0x7f22e00a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22f8105910 0x7f22f81a1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f22f8104f50 0x7f22f819a4c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 --2- 172.21.15.71:0/275395609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22f8103d60 0x7f22f8199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 >> 172.21.15.71:0/275395609 conn(0x7f22f80ff4e0 msgr2=0x7f22f81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 shutdown_connections 2024-09-05T12:27:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.200+0000 7f22ffe00640 1 -- 172.21.15.71:0/275395609 wait complete. 2024-09-05T12:27:44.217 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-05T12:27:44.227 INFO:teuthology.run_tasks:Running task exec... 2024-09-05T12:27:44.236 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T12:27:44.236 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:27:44.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-05T12:27:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 -- 172.21.15.71:0/138001644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 msgr2=0x7fcb9c104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 --2- 172.21.15.71:0/138001644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c104160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c009860 tx=0x7fcb8c02a340 comp rx=0 tx=0).stop 2024-09-05T12:27:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 -- 172.21.15.71:0/138001644 shutdown_connections 2024-09-05T12:27:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 --2- 172.21.15.71:0/138001644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c10c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 --2- 172.21.15.71:0/138001644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c104f50 0x7fcb9c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 --2- 172.21.15.71:0/138001644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c104160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 -- 172.21.15.71:0/138001644 >> 172.21.15.71:0/138001644 conn(0x7fcb9c0ff4e0 msgr2=0x7fcb9c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:44.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 -- 172.21.15.71:0/138001644 shutdown_connections 2024-09-05T12:27:44.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.394+0000 7fcba0c00640 1 -- 172.21.15.71:0/138001644 wait complete. 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 Processor -- start 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 -- start start 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c104f50 0x7fcb9c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c1a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c10f9e0 con 0x7fcb9c105910 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c10f860 con 0x7fcb9c103d60 2024-09-05T12:27:44.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.395+0000 7fcba0c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9c10fb60 con 0x7fcb9c104f50 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c1a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c1a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41890/0 (socket says 172.21.15.71:41890) 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 -- 172.21.15.71:0/3364426526 learned_addr learned my addr 172.21.15.71:0/3364426526 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb99e00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb99400640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c104f50 0x7fcb9c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 -- 172.21.15.71:0/3364426526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c104f50 msgr2=0x7fcb9c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c104f50 0x7fcb9c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 -- 172.21.15.71:0/3364426526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 msgr2=0x7fcb9c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 -- 172.21.15.71:0/3364426526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9c1a1c20 con 0x7fcb9c105910 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb99e00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:44.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.396+0000 7fcb9a800640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c1a15b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009fd0 tx=0x7fcb9400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.397+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94004b20 con 0x7fcb9c105910 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.397+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb94003e50 con 0x7fcb9c105910 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.397+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9c1a1f10 con 0x7fcb9c105910 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.397+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400b630 con 0x7fcb9c105910 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.397+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb9c1a2380 con 0x7fcb9c105910 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.398+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb9c10b080 con 0x7fcb9c105910 2024-09-05T12:27:44.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.400+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 20) ==== 137850+0+0 (secure 0 0 0) 0x7fcb9401d020 con 0x7fcb9c105910 2024-09-05T12:27:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.401+0000 7fcb92a00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcb800a2c80 0x7fcb800a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.401+0000 7fcb99e00640 1 -- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcb800a2c80 msgr2=0x7fcb800a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.71:6800/2050574497 2024-09-05T12:27:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.401+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9092+0+0 (secure 0 0 0) 0x7fcb940cf310 con 0x7fcb9c105910 2024-09-05T12:27:44.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.401+0000 7fcb99e00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcb800a2c80 0x7fcb800a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-05T12:27:44.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.405+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb9409bfb0 con 0x7fcb9c105910 2024-09-05T12:27:44.497 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-mon[29845]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-05T12:27:44.497 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/275395609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T12:27:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:44 smithi174 ceph-mon[34313]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-05T12:27:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:44 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/275395609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T12:27:44.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.509+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7fcb94097f20 con 0x7fcb9c105910 2024-09-05T12:27:44.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.509+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcb800a2c80 msgr2=0x7fcb800a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:27:44.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.509+0000 7fcb92a00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcb800a2c80 0x7fcb800a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.616+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fcb9c062f00 con 0x7fcb9c105910 2024-09-05T12:27:44.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.622+0000 7fcb92a00640 1 -- 172.21.15.71:0/3364426526 <== mon.0 v2:172.21.15.71:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7fcb9408bf40 con 0x7fcb9c105910 2024-09-05T12:27:44.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 msgr2=0x7fcb9c1a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c1a15b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcb94009fd0 tx=0x7fcb9400c800 comp rx=0 tx=0).stop 2024-09-05T12:27:44.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 shutdown_connections 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:6800/2050574497,v1:172.21.15.71:6801/2050574497] conn(0x7fcb800a2c80 0x7fcb800a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb9c105910 0x7fcb9c1a15b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb9c104f50 0x7fcb9c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 --2- 172.21.15.71:0/3364426526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb9c103d60 0x7fcb9c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.628+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 >> 172.21.15.71:0/3364426526 conn(0x7fcb9c0ff4e0 msgr2=0x7fcb9c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.629+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 shutdown_connections 2024-09-05T12:27:44.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.629+0000 7fcba0c00640 1 -- 172.21.15.71:0/3364426526 wait complete. 2024-09-05T12:27:44.657 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-05T12:27:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:44 smithi123 ceph-mon[34321]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-05T12:27:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:44 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/275395609' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-05T12:27:44.823 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.495+0000 7f0b3862d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 -- 172.21.15.71:0/3843385418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 msgr2=0x7f34dc1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 --2- 172.21.15.71:0/3843385418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 0x7f34dc1053c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009860 tx=0x7f34c802a340 comp rx=0 tx=0).stop 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 -- 172.21.15.71:0/3843385418 shutdown_connections 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 --2- 172.21.15.71:0/3843385418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc105900 0x7f34dc10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 --2- 172.21.15.71:0/3843385418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 0x7f34dc1053c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 --2- 172.21.15.71:0/3843385418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc103d50 0x7f34dc104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 -- 172.21.15.71:0/3843385418 >> 172.21.15.71:0/3843385418 conn(0x7f34dc0ff4b0 msgr2=0x7f34dc1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 -- 172.21.15.71:0/3843385418 shutdown_connections 2024-09-05T12:27:44.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.898+0000 7f34e3200640 1 -- 172.21.15.71:0/3843385418 wait complete. 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 Processor -- start 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 -- start start 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 0x7f34dc199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 0x7f34dc19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc105900 0x7f34dc1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc10f9d0 con 0x7f34dc105900 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc10f850 con 0x7f34dc104f40 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.899+0000 7f34e3200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34dc10fb50 con 0x7f34dc103d50 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 0x7f34dc199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 0x7f34dc199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59538/0 (socket says 172.21.15.71:59538) 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34e1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc105900 0x7f34dc1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 -- 172.21.15.71:0/1762582630 learned_addr learned my addr 172.21.15.71:0/1762582630 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:44.902 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34db400640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 0x7f34dc19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:44.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 -- 172.21.15.71:0/1762582630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 msgr2=0x7f34dc19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 0x7f34dc19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 -- 172.21.15.71:0/1762582630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc105900 msgr2=0x7f34dc1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:44.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc105900 0x7f34dc1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:44.903 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34dbe00640 1 -- 172.21.15.71:0/1762582630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34dc1a1bc0 con 0x7f34dc103d50 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.900+0000 7f34e1200640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc105900 0x7f34dc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.901+0000 7f34dbe00640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 0x7f34dc199fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00e9f0 tx=0x7f34cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.901+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00ccc0 con 0x7f34dc103d50 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.901+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34cc0035f0 con 0x7f34dc103d50 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.901+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00ce60 con 0x7f34dc103d50 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.901+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34dc1a1eb0 con 0x7f34dc103d50 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.901+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34dc1a2320 con 0x7f34dc103d50 2024-09-05T12:27:44.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.902+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34dc10b070 con 0x7f34dc103d50 2024-09-05T12:27:44.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.904+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7f34cc0037a0 con 0x7f34dc103d50 2024-09-05T12:27:44.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.905+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f34cc0cf1a0 con 0x7f34dc103d50 2024-09-05T12:27:44.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:44.909+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34cc09c040 con 0x7f34dc103d50 2024-09-05T12:27:45.098 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.096+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f34dc02d7c0 con 0x7f34dc103d50 2024-09-05T12:27:45.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.103+0000 7f34d8c00640 1 -- 172.21.15.71:0/1762582630 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7f34cc09b940 con 0x7f34dc103d50 2024-09-05T12:27:45.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 msgr2=0x7f34dc199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 0x7f34dc199fa0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00e9f0 tx=0x7f34cc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 shutdown_connections 2024-09-05T12:27:45.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34dc105900 0x7f34dc1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34dc104f40 0x7f34dc19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 --2- 172.21.15.71:0/1762582630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34dc103d50 0x7f34dc199fa0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 >> 172.21.15.71:0/1762582630 conn(0x7f34dc0ff4b0 msgr2=0x7f34dc108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:45.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 shutdown_connections 2024-09-05T12:27:45.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.105+0000 7f34e3200640 1 -- 172.21.15.71:0/1762582630 wait complete. 2024-09-05T12:27:45.121 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.875+0000 7f0a91a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.121 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.875+0000 7f0a91a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.121 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.875+0000 7f0a91a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.121 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.875+0000 7f0a91a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.121 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.875+0000 7f0a91a00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.122 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.876+0000 7f0a94c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.122 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.876+0000 7f0a94c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.122 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.876+0000 7f0a94c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.122 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.876+0000 7f0a94c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.122 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:44.876+0000 7f0a94c00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:27:45.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-05T12:27:45.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 -- 172.21.15.71:0/2757147034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f81023c0 msgr2=0x7fe9f81047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 --2- 172.21.15.71:0/2757147034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f81023c0 0x7fe9f81047b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009860 tx=0x7fe9ec02a340 comp rx=0 tx=0).stop 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 -- 172.21.15.71:0/2757147034 shutdown_connections 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 --2- 172.21.15.71:0/2757147034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f8107620 0x7fe9f8109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 --2- 172.21.15.71:0/2757147034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f8104cf0 0x7fe9f81070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 --2- 172.21.15.71:0/2757147034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f81023c0 0x7fe9f81047b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 -- 172.21.15.71:0/2757147034 >> 172.21.15.71:0/2757147034 conn(0x7fe9f80fbfa0 msgr2=0x7fe9f80fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.284+0000 7fe9fea00640 1 -- 172.21.15.71:0/2757147034 shutdown_connections 2024-09-05T12:27:45.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.285+0000 7fe9fea00640 1 -- 172.21.15.71:0/2757147034 wait complete. 2024-09-05T12:27:45.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.285+0000 7fe9fea00640 1 Processor -- start 2024-09-05T12:27:45.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 -- start start 2024-09-05T12:27:45.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 0x7fe9f8199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f8104cf0 0x7fe9f819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 0x7fe9f81a14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f810d630 con 0x7fe9f8104cf0 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f810d4b0 con 0x7fe9f8107620 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fea00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f810d7b0 con 0x7fe9f81023c0 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 0x7fe9f8199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fcc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f8104cf0 0x7fe9f819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 0x7fe9f81a14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 0x7fe9f81a14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54224/0 (socket says 172.21.15.71:54224) 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 -- 172.21.15.71:0/925604758 learned_addr learned my addr 172.21.15.71:0/925604758 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 0x7fe9f8199ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59558/0 (socket says 172.21.15.71:59558) 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 -- 172.21.15.71:0/925604758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 msgr2=0x7fe9f8199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 0x7fe9f8199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.290 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 -- 172.21.15.71:0/925604758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f8104cf0 msgr2=0x7fe9f819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f8104cf0 0x7fe9f819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.286+0000 7fe9fe000640 1 -- 172.21.15.71:0/925604758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f81a1b10 con 0x7fe9f8107620 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9fd600640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 0x7fe9f8199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9fe000640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 0x7fe9f81a14a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000eb40 tx=0x7fe9f000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000ce40 con 0x7fe9f8107620 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f0005020 con 0x7fe9f8107620 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f81a1e00 con 0x7fe9f8107620 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0010690 con 0x7fe9f8107620 2024-09-05T12:27:45.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.287+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f81a2310 con 0x7fe9f8107620 2024-09-05T12:27:45.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.288+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f8109190 con 0x7fe9f8107620 2024-09-05T12:27:45.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.290+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 21) ==== 137518+0+0 (secure 0 0 0) 0x7fe9f00051d0 con 0x7fe9f8107620 2024-09-05T12:27:45.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.291+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe9f00cf5f0 con 0x7fe9f8107620 2024-09-05T12:27:45.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.295+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe9f009c490 con 0x7fe9f8107620 2024-09-05T12:27:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/275395609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T12:27:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:27:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: Active manager daemon x restarted 2024-09-05T12:27:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: Activating manager daemon x 2024-09-05T12:27:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T12:27:45.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: mgrmap e21: x(active, starting, since 0.0128501s), standbys: z, y 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:27:45.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T12:27:45.413 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:27:45.414 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: Manager daemon x is now available 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3364426526' entity='client.admin' 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T12:27:45.415 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' 2024-09-05T12:27:45.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.480+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fe9f802d7c0 con 0x7fe9f8107620 2024-09-05T12:27:45.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.486+0000 7fe9f6000640 1 -- 172.21.15.71:0/925604758 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fe9f009bd90 con 0x7fe9f8107620 2024-09-05T12:27:45.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 msgr2=0x7fe9f81a14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 0x7fe9f81a14a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000eb40 tx=0x7fe9f000ef40 comp rx=0 tx=0).stop 2024-09-05T12:27:45.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 shutdown_connections 2024-09-05T12:27:45.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9f8107620 0x7fe9f81a14a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9f8104cf0 0x7fe9f819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 --2- 172.21.15.71:0/925604758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9f81023c0 0x7fe9f8199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 >> 172.21.15.71:0/925604758 conn(0x7fe9f80fbfa0 msgr2=0x7fe9f8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:45.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.488+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 shutdown_connections 2024-09-05T12:27:45.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.489+0000 7fe9fea00640 1 -- 172.21.15.71:0/925604758 wait complete. 2024-09-05T12:27:45.510 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/275395609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: Active manager daemon x restarted 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: Activating manager daemon x 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: mgrmap e21: x(active, starting, since 0.0128501s), standbys: z, y 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:27:45.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:27:45.512 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:27:45.513 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: Manager daemon x is now available 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3364426526' entity='client.admin' 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T12:27:45.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:45 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' 2024-09-05T12:27:45.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/275395609' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: Active manager daemon x restarted 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: Activating manager daemon x 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: osdmap e95: 12 total, 12 up, 12 in 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: mgrmap e21: x(active, starting, since 0.0128501s), standbys: z, y 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-05T12:27:45.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-05T12:27:45.682 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-05T12:27:45.683 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: Manager daemon x is now available 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3364426526' entity='client.admin' 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-05T12:27:45.684 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:45 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 -- 172.21.15.71:0/1232820989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8105910 msgr2=0x7fb9c810c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 --2- 172.21.15.71:0/1232820989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8105910 0x7fb9c810c1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009be0 tx=0x7fb9c002a6c0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 -- 172.21.15.71:0/1232820989 shutdown_connections 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 --2- 172.21.15.71:0/1232820989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8105910 0x7fb9c810c1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 --2- 172.21.15.71:0/1232820989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8104f50 0x7fb9c81053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 --2- 172.21.15.71:0/1232820989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8103d60 0x7fb9c8104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 -- 172.21.15.71:0/1232820989 >> 172.21.15.71:0/1232820989 conn(0x7fb9c80ff4e0 msgr2=0x7fb9c8101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 -- 172.21.15.71:0/1232820989 shutdown_connections 2024-09-05T12:27:45.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.692+0000 7fb9cec00640 1 -- 172.21.15.71:0/1232820989 wait complete. 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 Processor -- start 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 -- start start 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8103d60 0x7fb9c8199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8104f50 0x7fb9c819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8105910 0x7fb9c81a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c810f9e0 con 0x7fb9c8105910 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c810f860 con 0x7fb9c8104f50 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.693+0000 7fb9cec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c810fb60 con 0x7fb9c8103d60 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9ccc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8105910 0x7fb9c81a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9ccc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8105910 0x7fb9c81a14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42020/0 (socket says 172.21.15.71:42020) 2024-09-05T12:27:45.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9ccc00640 1 -- 172.21.15.71:0/2342762652 learned_addr learned my addr 172.21.15.71:0/2342762652 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7400640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8104f50 0x7fb9c819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8103d60 0x7fb9c8199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 -- 172.21.15.71:0/2342762652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8104f50 msgr2=0x7fb9c819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8104f50 0x7fb9c819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 -- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8105910 msgr2=0x7fb9c81a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8105910 0x7fb9c81a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 -- 172.21.15.71:0/2342762652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c81a1b30 con 0x7fb9c8103d60 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7400640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8104f50 0x7fb9c819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.694+0000 7fb9c7e00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8103d60 0x7fb9c8199f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8104db0 tx=0x7fb9b800ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.695+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8004440 con 0x7fb9c8103d60 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.695+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b800cda0 con 0x7fb9c8103d60 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.695+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c81a1e20 con 0x7fb9c8103d60 2024-09-05T12:27:45.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.695+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b80196e0 con 0x7fb9c8103d60 2024-09-05T12:27:45.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.695+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c81a22e0 con 0x7fb9c8103d60 2024-09-05T12:27:45.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.696+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9c810b020 con 0x7fb9c8103d60 2024-09-05T12:27:45.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.698+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7fb9b8003780 con 0x7fb9c8103d60 2024-09-05T12:27:45.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.699+0000 7fb9c4c00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb9ac0a2ac0 0x7fb9ac0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:45.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.699+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb9b80cf3f0 con 0x7fb9c8103d60 2024-09-05T12:27:45.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.699+0000 7fb9c7400640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb9ac0a2ac0 0x7fb9ac0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:45.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.700+0000 7fb9c7400640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb9ac0a2ac0 0x7fb9ac0a4f80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb9c819b420 tx=0x7fb9b4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:45.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.703+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9b809c1a0 con 0x7fb9c8103d60 2024-09-05T12:27:45.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.915+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fb9c8062630 con 0x7fb9ac0a2ac0 2024-09-05T12:27:45.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.985+0000 7fb9c4c00640 1 -- 172.21.15.71:0/2342762652 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb9c8062630 con 0x7fb9ac0a2ac0 2024-09-05T12:27:45.987 INFO:teuthology.orchestra.run.smithi071.stdout:Schedule set for path / 2024-09-05T12:27:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb9ac0a2ac0 msgr2=0x7fb9ac0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb9ac0a2ac0 0x7fb9ac0a4f80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb9c819b420 tx=0x7fb9b4005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8103d60 msgr2=0x7fb9c8199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8103d60 0x7fb9c8199f10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8104db0 tx=0x7fb9b800ee50 comp rx=0 tx=0).stop 2024-09-05T12:27:45.989 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 shutdown_connections 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb9ac0a2ac0 0x7fb9ac0a4f80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9c8105910 0x7fb9c81a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9c8104f50 0x7fb9c819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 --2- 172.21.15.71:0/2342762652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9c8103d60 0x7fb9c8199f10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.987+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 >> 172.21.15.71:0/2342762652 conn(0x7fb9c80ff4e0 msgr2=0x7fb9c81081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.988+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 shutdown_connections 2024-09-05T12:27:45.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:45.988+0000 7fb9cec00640 1 -- 172.21.15.71:0/2342762652 wait complete. 2024-09-05T12:27:46.005 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-05T12:27:46.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:45.898+0000 7f0a72600640 -1 log_channel(cephadm) log [ERR] : [05/Sep/2024:12:27:45] ENGINE Error in HTTPServer.serve 2024-09-05T12:27:46.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: Traceback (most recent call last): 2024-09-05T12:27:46.161 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._connections.run(self.expiration_interval) 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._run(expiration_interval) 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: s = self.context.wrap_socket( 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: return self.sslsocket_class._create( 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self.do_handshake() 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:27:46.162 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: self._sslobj.do_handshake() 2024-09-05T12:27:46.163 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:27:46.163 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:27:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:27:46.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 -- 172.21.15.71:0/1818046773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678109c90 msgr2=0x7f967810c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 --2- 172.21.15.71:0/1818046773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678109c90 0x7f967810c080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9670009ca0 tx=0x7f967002a8b0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 -- 172.21.15.71:0/1818046773 shutdown_connections 2024-09-05T12:27:46.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 --2- 172.21.15.71:0/1818046773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678109c90 0x7f967810c080 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 --2- 172.21.15.71:0/1818046773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9678074fc0 0x7f9678109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 --2- 172.21.15.71:0/1818046773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9678074680 0x7f9678074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 -- 172.21.15.71:0/1818046773 >> 172.21.15.71:0/1818046773 conn(0x7f96780fd0d0 msgr2=0x7f96780ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:46.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 -- 172.21.15.71:0/1818046773 shutdown_connections 2024-09-05T12:27:46.173 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.170+0000 7f967e600640 1 -- 172.21.15.71:0/1818046773 wait complete. 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.171+0000 7f967e600640 1 Processor -- start 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.171+0000 7f967e600640 1 -- start start 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 0x7f9678199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9678074fc0 0x7f967819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9678109c90 0x7f96781a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f967810f8f0 con 0x7f9678109c90 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f967810f770 con 0x7f9678074680 2024-09-05T12:27:46.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967e600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f967810fa70 con 0x7f9678074fc0 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 0x7f9678199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9678109c90 0x7f96781a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 0x7f9678199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54256/0 (socket says 172.21.15.71:54256) 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 -- 172.21.15.71:0/4030568073 learned_addr learned my addr 172.21.15.71:0/4030568073 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f9677e00640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9678074fc0 0x7f967819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 -- 172.21.15.71:0/4030568073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9678074fc0 msgr2=0x7f967819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9678074fc0 0x7f967819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 -- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9678109c90 msgr2=0x7f96781a1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9678109c90 0x7f96781a1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.172+0000 7f967d200640 1 -- 172.21.15.71:0/4030568073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96781a1b90 con 0x7f9678074680 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.173+0000 7f967d200640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 0x7f9678199f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f966c00ea60 tx=0x7f966c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:46.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.173+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c00cde0 con 0x7f9678074680 2024-09-05T12:27:46.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.173+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f966c0035f0 con 0x7f9678074680 2024-09-05T12:27:46.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.173+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96781a1e80 con 0x7f9678074680 2024-09-05T12:27:46.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.173+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966c010670 con 0x7f9678074680 2024-09-05T12:27:46.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.173+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96781a22f0 con 0x7f9678074680 2024-09-05T12:27:46.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.174+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9678074a80 con 0x7f9678074680 2024-09-05T12:27:46.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.178+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f966c0037a0 con 0x7f9678074680 2024-09-05T12:27:46.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.179+0000 7f9675600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96600a29f0 0x7f96600a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.179+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f966c0cf400 con 0x7f9678074680 2024-09-05T12:27:46.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.179+0000 7f9677e00640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96600a29f0 0x7f96600a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.180+0000 7f9677e00640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96600a29f0 0x7f96600a4eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f967819b480 tx=0x7f9664005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:46.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.183+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f966c09c1b0 con 0x7f9678074680 2024-09-05T12:27:46.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.398+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f96780626d0 con 0x7f96600a29f0 2024-09-05T12:27:46.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.462+0000 7f9675600640 1 -- 172.21.15.71:0/4030568073 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f96780626d0 con 0x7f96600a29f0 2024-09-05T12:27:46.464 INFO:teuthology.orchestra.run.smithi071.stdout:Retention added to path / 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96600a29f0 msgr2=0x7f96600a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96600a29f0 0x7f96600a4eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f967819b480 tx=0x7f9664005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 msgr2=0x7f9678199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 0x7f9678199f70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f966c00ea60 tx=0x7f966c00ef20 comp rx=0 tx=0).stop 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 shutdown_connections 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96600a29f0 0x7f96600a4eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9678109c90 0x7f96781a1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9678074fc0 0x7f967819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 --2- 172.21.15.71:0/4030568073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9678074680 0x7f9678199f70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 >> 172.21.15.71:0/4030568073 conn(0x7f96780fd0d0 msgr2=0x7f9678105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:46.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 shutdown_connections 2024-09-05T12:27:46.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.464+0000 7f967e600640 1 -- 172.21.15.71:0/4030568073 wait complete. 2024-09-05T12:27:46.482 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-05T12:27:46.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 -- 172.21.15.71:0/1273240727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 msgr2=0x7f40d41025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 --2- 172.21.15.71:0/1273240727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 0x7f40d41025c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009900 tx=0x7f40c402a400 comp rx=0 tx=0).stop 2024-09-05T12:27:46.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 -- 172.21.15.71:0/1273240727 shutdown_connections 2024-09-05T12:27:46.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 --2- 172.21.15.71:0/1273240727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d41054b0 0x7f40d4105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 --2- 172.21.15.71:0/1273240727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d4104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.652 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 --2- 172.21.15.71:0/1273240727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 0x7f40d41025c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 -- 172.21.15.71:0/1273240727 >> 172.21.15.71:0/1273240727 conn(0x7f40d40fbfa0 msgr2=0x7f40d40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:46.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 -- 172.21.15.71:0/1273240727 shutdown_connections 2024-09-05T12:27:46.653 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.650+0000 7f40d8e00640 1 -- 172.21.15.71:0/1273240727 wait complete. 2024-09-05T12:27:46.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.651+0000 7f40d8e00640 1 Processor -- start 2024-09-05T12:27:46.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.651+0000 7f40d8e00640 1 -- start start 2024-09-05T12:27:46.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 0x7f40d419e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.654 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d41054b0 0x7f40d41a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d8e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4113c00 con 0x7f40d41054b0 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4113a80 con 0x7f40d41001b0 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d8e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40d4113d80 con 0x7f40d4102b00 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59638/0 (socket says 172.21.15.71:59638) 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d2a00640 1 -- 172.21.15.71:0/2674610782 learned_addr learned my addr 172.21.15.71:0/2674610782 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d3400640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 0x7f40d419e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.652+0000 7f40d3e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d41054b0 0x7f40d41a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40d2a00640 1 -- 172.21.15.71:0/2674610782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 msgr2=0x7f40d419e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40d2a00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 0x7f40d419e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40d2a00640 1 -- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d41054b0 msgr2=0x7f40d41a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.655 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40d2a00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d41054b0 0x7f40d41a57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40d2a00640 1 -- 172.21.15.71:0/2674610782 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40d41a5e30 con 0x7f40d4102b00 2024-09-05T12:27:46.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40d2a00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d419e750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c890 tx=0x7f40c000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:46.656 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.653+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0004220 con 0x7f40d4102b00 2024-09-05T12:27:46.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.654+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c00043c0 con 0x7f40d4102b00 2024-09-05T12:27:46.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.654+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c00049e0 con 0x7f40d4102b00 2024-09-05T12:27:46.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.654+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40d41a6120 con 0x7f40d4102b00 2024-09-05T12:27:46.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.654+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40d41a6620 con 0x7f40d4102b00 2024-09-05T12:27:46.657 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.655+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40d40fda90 con 0x7f40d4102b00 2024-09-05T12:27:46.659 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.658+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f40c00079b0 con 0x7f40d4102b00 2024-09-05T12:27:46.661 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.660+0000 7f40cbe00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40b80a29f0 0x7f40b80a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.660+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f40c00cf100 con 0x7f40d4102b00 2024-09-05T12:27:46.662 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.661+0000 7f40d3400640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40b80a29f0 0x7f40b80a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:46.663 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.662+0000 7f40d3400640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40b80a29f0 0x7f40b80a4eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40c4031440 tx=0x7f40c40315b0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:46.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.664+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f40c0014030 con 0x7f40d4102b00 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: mgrmap e22: x(active, since 1.02205s), standbys: z, y 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: [05/Sep/2024:12:27:45] ENGINE Bus STARTING 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: [05/Sep/2024:12:27:45] ENGINE Serving on http://172.21.15.71:8765 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: [05/Sep/2024:12:27:45] ENGINE Serving on https://172.21.15.71:7150 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: [05/Sep/2024:12:27:45] ENGINE Bus STARTED 2024-09-05T12:27:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: [05/Sep/2024:12:27:45] ENGINE Error in HTTPServer.serve 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: Traceback (most recent call last): 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._connections.run(self.expiration_interval) 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._run(expiration_interval) 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: s = self.context.wrap_socket( 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:27:46.761 INFO:journalctl@ceph.mon.c.smithi174.stdout: return self.sslsocket_class._create( 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout: self.do_handshake() 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout: self._sslobj.do_handshake() 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: from='client.25282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:46 smithi174 ceph-mon[34313]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: mgrmap e22: x(active, since 1.02205s), standbys: z, y 2024-09-05T12:27:46.762 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: [05/Sep/2024:12:27:45] ENGINE Bus STARTING 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: [05/Sep/2024:12:27:45] ENGINE Serving on http://172.21.15.71:8765 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: [05/Sep/2024:12:27:45] ENGINE Serving on https://172.21.15.71:7150 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: [05/Sep/2024:12:27:45] ENGINE Bus STARTED 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: [05/Sep/2024:12:27:45] ENGINE Error in HTTPServer.serve 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: Traceback (most recent call last): 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._connections.run(self.expiration_interval) 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._run(expiration_interval) 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:27:46.763 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: s = self.context.wrap_socket( 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: return self.sslsocket_class._create( 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: self.do_handshake() 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: self._sslobj.do_handshake() 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: from='client.25282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:46.764 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:46 smithi071 ceph-mon[29845]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:46.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.876+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f40d4062630 con 0x7f40b80a29f0 2024-09-05T12:27:46.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.884+0000 7f40cbe00640 1 -- 172.21.15.71:0/2674610782 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f40d4062630 con 0x7f40b80a29f0 2024-09-05T12:27:46.886 INFO:teuthology.orchestra.run.smithi071.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-05T00:00:00", "created": "2024-09-05T12:27:45", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-05T12:27:46.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40b80a29f0 msgr2=0x7f40b80a4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40b80a29f0 0x7f40b80a4eb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f40c4031440 tx=0x7f40c40315b0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 msgr2=0x7f40d419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d419e750 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c890 tx=0x7f40c000cd50 comp rx=0 tx=0).stop 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 shutdown_connections 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40b80a29f0 0x7f40b80a4eb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d41054b0 0x7f40d41a57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40d4102b00 0x7f40d419e750 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 --2- 172.21.15.71:0/2674610782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d41001b0 0x7f40d419e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.886+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 >> 172.21.15.71:0/2674610782 conn(0x7f40d40fbfa0 msgr2=0x7f40d4103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.887+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 shutdown_connections 2024-09-05T12:27:46.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:46.887+0000 7f40d8e00640 1 -- 172.21.15.71:0/2674610782 wait complete. 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: mgrmap e22: x(active, since 1.02205s), standbys: z, y 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: [05/Sep/2024:12:27:45] ENGINE Bus STARTING 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: [05/Sep/2024:12:27:45] ENGINE Serving on http://172.21.15.71:8765 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: [05/Sep/2024:12:27:45] ENGINE Serving on https://172.21.15.71:7150 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: [05/Sep/2024:12:27:45] ENGINE Bus STARTED 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: [05/Sep/2024:12:27:45] ENGINE Error in HTTPServer.serve 2024-09-05T12:27:46.895 INFO:journalctl@ceph.mon.b.smithi123.stdout: Traceback (most recent call last): 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: self._connections.run(self.expiration_interval) 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: self._run(expiration_interval) 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: s = self.context.wrap_socket( 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: return self.sslsocket_class._create( 2024-09-05T12:27:46.896 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-05T12:27:46.897 INFO:journalctl@ceph.mon.b.smithi123.stdout: self.do_handshake() 2024-09-05T12:27:46.897 INFO:journalctl@ceph.mon.b.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-05T12:27:46.897 INFO:journalctl@ceph.mon.b.smithi123.stdout: self._sslobj.do_handshake() 2024-09-05T12:27:46.897 INFO:journalctl@ceph.mon.b.smithi123.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-05T12:27:46.897 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: from='client.25282 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:46.897 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:46 smithi123 ceph-mon[34321]: from='client.25283 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:46.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-05T12:27:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 -- 172.21.15.71:0/4008891662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d81025d0 msgr2=0x7f06d81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 --2- 172.21.15.71:0/4008891662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d81025d0 0x7f06d81049c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009ca0 tx=0x7f06c802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 -- 172.21.15.71:0/4008891662 shutdown_connections 2024-09-05T12:27:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 --2- 172.21.15.71:0/4008891662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d8107830 0x7f06d8109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 --2- 172.21.15.71:0/4008891662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06d8104f00 0x7f06d81072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 --2- 172.21.15.71:0/4008891662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d81025d0 0x7f06d81049c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 -- 172.21.15.71:0/4008891662 >> 172.21.15.71:0/4008891662 conn(0x7f06d80fc1b0 msgr2=0x7f06d80fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 -- 172.21.15.71:0/4008891662 shutdown_connections 2024-09-05T12:27:47.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.064+0000 7f06dde00640 1 -- 172.21.15.71:0/4008891662 wait complete. 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.065+0000 7f06dde00640 1 Processor -- start 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.065+0000 7f06dde00640 1 -- start start 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06dde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 0x7f06d819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06dde00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d8104f00 0x7f06d819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06dde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06d8107830 0x7f06d81a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06dde00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810d890 con 0x7f06d8107830 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06dde00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810d710 con 0x7f06d81025d0 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06dde00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06d810da10 con 0x7f06d8104f00 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 0x7f06d819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 0x7f06d819a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54300/0 (socket says 172.21.15.71:54300) 2024-09-05T12:27:47.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 -- 172.21.15.71:0/3433469223 learned_addr learned my addr 172.21.15.71:0/3433469223 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06ce000640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d8104f00 0x7f06d819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d7400640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06d8107830 0x7f06d81a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 -- 172.21.15.71:0/3433469223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d8104f00 msgr2=0x7f06d819a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d8104f00 0x7f06d819a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 -- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06d8107830 msgr2=0x7f06d81a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06d8107830 0x7f06d81a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06d6a00640 1 -- 172.21.15.71:0/3433469223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d81a1c40 con 0x7f06d81025d0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.066+0000 7f06ce000640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d8104f00 0x7f06d819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.067+0000 7f06d6a00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 0x7f06d819a020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f06c8000f80 tx=0x7f06c80319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.067+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8003b70 con 0x7f06d81025d0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.067+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d81a1ed0 con 0x7f06d81025d0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.067+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c8003d10 con 0x7f06d81025d0 2024-09-05T12:27:47.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.067+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c802fc00 con 0x7f06d81025d0 2024-09-05T12:27:47.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.067+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d81a2390 con 0x7f06d81025d0 2024-09-05T12:27:47.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.068+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d81093a0 con 0x7f06d81025d0 2024-09-05T12:27:47.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.070+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 22) ==== 137645+0+0 (secure 0 0 0) 0x7f06c8004b40 con 0x7f06d81025d0 2024-09-05T12:27:47.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.071+0000 7f06dca00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06bc0a2ac0 0x7f06bc0a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:47.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.072+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f06c80f0600 con 0x7f06d81025d0 2024-09-05T12:27:47.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.072+0000 7f06ce000640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06bc0a2ac0 0x7f06bc0a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:47.074 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.072+0000 7f06ce000640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06bc0a2ac0 0x7f06bc0a4f80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06c00095f0 tx=0x7f06c0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:47.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.074+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f06c80bd3b0 con 0x7f06d81025d0 2024-09-05T12:27:47.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.302+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f06d8062630 con 0x7f06bc0a2ac0 2024-09-05T12:27:47.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.310+0000 7f06dca00640 1 -- 172.21.15.71:0/3433469223 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f06d8062630 con 0x7f06bc0a2ac0 2024-09-05T12:27:47.311 INFO:teuthology.orchestra.run.smithi071.stdout:/ 1m 6m3h 2024-09-05T12:27:47.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06bc0a2ac0 msgr2=0x7f06bc0a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:47.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06bc0a2ac0 0x7f06bc0a4f80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f06c00095f0 tx=0x7f06c0005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:47.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 msgr2=0x7f06d819a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 0x7f06d819a020 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f06c8000f80 tx=0x7f06c80319f0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 shutdown_connections 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06bc0a2ac0 0x7f06bc0a4f80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06d8107830 0x7f06d81a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06d8104f00 0x7f06d819a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 --2- 172.21.15.71:0/3433469223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06d81025d0 0x7f06d819a020 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 >> 172.21.15.71:0/3433469223 conn(0x7f06d80fc1b0 msgr2=0x7f06d8101ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 shutdown_connections 2024-09-05T12:27:47.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:47.312+0000 7f06dde00640 1 -- 172.21.15.71:0/3433469223 wait complete. 2024-09-05T12:27:47.332 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-05T12:27:47.404 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-05T12:27:47.414 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-05T12:27:47.423 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:27:47.424 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-05T12:27:47.428 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-05T12:27:47.441 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-05T12:27:47.441 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:27:47.881 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='client.25297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.890 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:27:47.891 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:47 smithi123 ceph-mon[34321]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-05T12:27:47.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='client.25297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:27:47.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:47 smithi071 ceph-mon[29845]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-05T12:27:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='client.25297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='client.25301 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:48.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:27:48.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:47 smithi174 ceph-mon[34313]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.546+0000 7f46bf200640 1 -- 172.21.15.71:0/2402664966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074510 msgr2=0x7f46b80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.546+0000 7f46bf200640 1 --2- 172.21.15.71:0/2402664966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074510 0x7f46b80748f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f46a8009900 tx=0x7f46a802a480 comp rx=0 tx=0).stop 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 -- 172.21.15.71:0/2402664966 shutdown_connections 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 --2- 172.21.15.71:0/2402664966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b810e450 0x7f46b8110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 --2- 172.21.15.71:0/2402664966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b8074e30 0x7f46b810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 --2- 172.21.15.71:0/2402664966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074510 0x7f46b80748f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 -- 172.21.15.71:0/2402664966 >> 172.21.15.71:0/2402664966 conn(0x7f46b80fd8b0 msgr2=0x7f46b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 -- 172.21.15.71:0/2402664966 shutdown_connections 2024-09-05T12:27:48.551 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.547+0000 7f46bf200640 1 -- 172.21.15.71:0/2402664966 wait complete. 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 Processor -- start 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 -- start start 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 0x7f46b819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074e30 0x7f46b819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b810e450 0x7f46b819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8114180 con 0x7f46b8074e30 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8114000 con 0x7f46b8074510 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.548+0000 7f46bf200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46b8114300 con 0x7f46b810e450 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 0x7f46b819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:48.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 0x7f46b819a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54324/0 (socket says 172.21.15.71:54324) 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 -- 172.21.15.71:0/2684221232 learned_addr learned my addr 172.21.15.71:0/2684221232 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46afe00640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074e30 0x7f46b819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46bd200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b810e450 0x7f46b819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 -- 172.21.15.71:0/2684221232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b810e450 msgr2=0x7f46b819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b810e450 0x7f46b819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 -- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074e30 msgr2=0x7f46b819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074e30 0x7f46b819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 -- 172.21.15.71:0/2684221232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b819f640 con 0x7f46b8074510 2024-09-05T12:27:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.549+0000 7f46b7e00640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 0x7f46b819a690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006010 tx=0x7f46a8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.550+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a803f030 con 0x7f46b8074510 2024-09-05T12:27:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.550+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46a8004440 con 0x7f46b8074510 2024-09-05T12:27:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.550+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b819f8d0 con 0x7f46b8074510 2024-09-05T12:27:48.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.550+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8035070 con 0x7f46b8074510 2024-09-05T12:27:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.550+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b81a7210 con 0x7f46b8074510 2024-09-05T12:27:48.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.551+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b8075c50 con 0x7f46b8074510 2024-09-05T12:27:48.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.554+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 23) ==== 137953+0+0 (secure 0 0 0) 0x7f46a8042030 con 0x7f46b8074510 2024-09-05T12:27:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.555+0000 7f46b5600640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f469c0a2cf0 0x7f469c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:48.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.556+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f46a80efe50 con 0x7f46b8074510 2024-09-05T12:27:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.556+0000 7f46afe00640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f469c0a2cf0 0x7f469c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.556+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46a80bc6d0 con 0x7f46b8074510 2024-09-05T12:27:48.558 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.557+0000 7f46afe00640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f469c0a2cf0 0x7f469c0a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46b80ffeb0 tx=0x7f46a0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:48.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.735+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46b8062630 con 0x7f469c0a2cf0 2024-09-05T12:27:48.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.749+0000 7f46b5600640 1 -- 172.21.15.71:0/2684221232 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339496 (secure 0 0 0) 0x7f46b8062630 con 0x7f469c0a2cf0 2024-09-05T12:27:48.756 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:48.758 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:27:48.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f469c0a2cf0 msgr2=0x7f469c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:48.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f469c0a2cf0 0x7f469c0a51b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46b80ffeb0 tx=0x7f46a0005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:48.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 msgr2=0x7f46b819a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:48.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 0x7f46b819a690 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f46a8006010 tx=0x7f46a8004040 comp rx=0 tx=0).stop 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 shutdown_connections 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b810e450 0x7f46b819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f469c0a2cf0 0x7f469c0a51b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b8074e30 0x7f46b819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 --2- 172.21.15.71:0/2684221232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46b8074510 0x7f46b819a690 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 >> 172.21.15.71:0/2684221232 conn(0x7f46b80fd8b0 msgr2=0x7f46b80ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 shutdown_connections 2024-09-05T12:27:48.761 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:48.754+0000 7f46bf200640 1 -- 172.21.15.71:0/2684221232 wait complete. 2024-09-05T12:27:49.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:48 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:49.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:48 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:49.325 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:48 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:49.325 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:48 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:48 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:49.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:48 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:49.989 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-05T12:27:48.524987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192848,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1547,"num_read_kb":943,"num_write":566,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378504,"kb_used_data":16512,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124563320,"statfs":{"total":1151940427776,"available":1151552839680,"internally_reserved":0,"allocated":16908288,"data_stored":10025537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.984508"},"pg_stats":[{"pgid":"2.1f","version":"95'14","reported_seq":182,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.883114+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:27:44.883114+0000","last_peered":"2024-09-05T12:27:44.883114+0000","last_clean":"2024-09-05T12:27:44.883114+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:27:44.883114+0000","last_undegraded":"2024-09-05T12:27:44.883114+0000","last_fullsized":"2024-09-05T12:27:44.883114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506128+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:27:44.506128+0000","last_peered":"2024-09-05T12:27:44.506128+0000","last_clean":"2024-09-05T12:27:44.506128+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:27:44.506128+0000","last_undegraded":"2024-09-05T12:27:44.506128+0000","last_fullsized":"2024-09-05T12:27:44.506128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553411+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:27:44.553411+0000","last_peered":"2024-09-05T12:27:44.553411+0000","last_clean":"2024-09-05T12:27:44.553411+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:27:44.553411+0000","last_undegraded":"2024-09-05T12:27:44.553411+0000","last_fullsized":"2024-09-05T12:27:44.553411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506439+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:27:44.506439+0000","last_peered":"2024-09-05T12:27:44.506439+0000","last_clean":"2024-09-05T12:27:44.506439+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:27:44.506439+0000","last_undegraded":"2024-09-05T12:27:44.506439+0000","last_fullsized":"2024-09-05T12:27:44.506439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884712+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:27:44.884712+0000","last_peered":"2024-09-05T12:27:44.884712+0000","last_clean":"2024-09-05T12:27:44.884712+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:27:44.884712+0000","last_undegraded":"2024-09-05T12:27:44.884712+0000","last_fullsized":"2024-09-05T12:27:44.884712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521451+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:27:44.521451+0000","last_peered":"2024-09-05T12:27:44.521451+0000","last_clean":"2024-09-05T12:27:44.521451+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:27:44.521451+0000","last_undegraded":"2024-09-05T12:27:44.521451+0000","last_fullsized":"2024-09-05T12:27:44.521451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'13","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509779+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:27:44.509779+0000","last_peered":"2024-09-05T12:27:44.509779+0000","last_clean":"2024-09-05T12:27:44.509779+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:27:44.509779+0000","last_undegraded":"2024-09-05T12:27:44.509779+0000","last_fullsized":"2024-09-05T12:27:44.509779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713202+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:27:44.713202+0000","last_peered":"2024-09-05T12:27:44.713202+0000","last_clean":"2024-09-05T12:27:44.713202+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:27:44.713202+0000","last_undegraded":"2024-09-05T12:27:44.713202+0000","last_fullsized":"2024-09-05T12:27:44.713202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521661+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:27:44.521661+0000","last_peered":"2024-09-05T12:27:44.521661+0000","last_clean":"2024-09-05T12:27:44.521661+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:27:44.521661+0000","last_undegraded":"2024-09-05T12:27:44.521661+0000","last_fullsized":"2024-09-05T12:27:44.521661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509541+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:27:44.509541+0000","last_peered":"2024-09-05T12:27:44.509541+0000","last_clean":"2024-09-05T12:27:44.509541+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:27:44.509541+0000","last_undegraded":"2024-09-05T12:27:44.509541+0000","last_fullsized":"2024-09-05T12:27:44.509541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520014+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:27:44.520014+0000","last_peered":"2024-09-05T12:27:44.520014+0000","last_clean":"2024-09-05T12:27:44.520014+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:27:44.520014+0000","last_undegraded":"2024-09-05T12:27:44.520014+0000","last_fullsized":"2024-09-05T12:27:44.520014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521095+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:27:44.521095+0000","last_peered":"2024-09-05T12:27:44.521095+0000","last_clean":"2024-09-05T12:27:44.521095+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:27:44.521095+0000","last_undegraded":"2024-09-05T12:27:44.521095+0000","last_fullsized":"2024-09-05T12:27:44.521095+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506770+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:27:44.506770+0000","last_peered":"2024-09-05T12:27:44.506770+0000","last_clean":"2024-09-05T12:27:44.506770+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:27:44.506770+0000","last_undegraded":"2024-09-05T12:27:44.506770+0000","last_fullsized":"2024-09-05T12:27:44.506770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509274+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:27:44.509274+0000","last_peered":"2024-09-05T12:27:44.509274+0000","last_clean":"2024-09-05T12:27:44.509274+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:27:44.509274+0000","last_undegraded":"2024-09-05T12:27:44.509274+0000","last_fullsized":"2024-09-05T12:27:44.509274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521721+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:27:44.521721+0000","last_peered":"2024-09-05T12:27:44.521721+0000","last_clean":"2024-09-05T12:27:44.521721+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:27:44.521721+0000","last_undegraded":"2024-09-05T12:27:44.521721+0000","last_fullsized":"2024-09-05T12:27:44.521721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505685+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:27:44.505685+0000","last_peered":"2024-09-05T12:27:44.505685+0000","last_clean":"2024-09-05T12:27:44.505685+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:27:44.505685+0000","last_undegraded":"2024-09-05T12:27:44.505685+0000","last_fullsized":"2024-09-05T12:27:44.505685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726946+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:27:44.726946+0000","last_peered":"2024-09-05T12:27:44.726946+0000","last_clean":"2024-09-05T12:27:44.726946+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:27:44.726946+0000","last_undegraded":"2024-09-05T12:27:44.726946+0000","last_fullsized":"2024-09-05T12:27:44.726946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521509+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:27:44.521509+0000","last_peered":"2024-09-05T12:27:44.521509+0000","last_clean":"2024-09-05T12:27:44.521509+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:27:44.521509+0000","last_undegraded":"2024-09-05T12:27:44.521509+0000","last_fullsized":"2024-09-05T12:27:44.521509+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726837+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:27:44.726837+0000","last_peered":"2024-09-05T12:27:44.726837+0000","last_clean":"2024-09-05T12:27:44.726837+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:27:44.726837+0000","last_undegraded":"2024-09-05T12:27:44.726837+0000","last_fullsized":"2024-09-05T12:27:44.726837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505891+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:27:44.505891+0000","last_peered":"2024-09-05T12:27:44.505891+0000","last_clean":"2024-09-05T12:27:44.505891+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:27:44.505891+0000","last_undegraded":"2024-09-05T12:27:44.505891+0000","last_fullsized":"2024-09-05T12:27:44.505891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523836+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:27:44.523836+0000","last_peered":"2024-09-05T12:27:44.523836+0000","last_clean":"2024-09-05T12:27:44.523836+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:27:44.523836+0000","last_undegraded":"2024-09-05T12:27:44.523836+0000","last_fullsized":"2024-09-05T12:27:44.523836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509025+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:27:44.509025+0000","last_peered":"2024-09-05T12:27:44.509025+0000","last_clean":"2024-09-05T12:27:44.509025+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:27:44.509025+0000","last_undegraded":"2024-09-05T12:27:44.509025+0000","last_fullsized":"2024-09-05T12:27:44.509025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713256+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:27:44.713256+0000","last_peered":"2024-09-05T12:27:44.713256+0000","last_clean":"2024-09-05T12:27:44.713256+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:27:44.713256+0000","last_undegraded":"2024-09-05T12:27:44.713256+0000","last_fullsized":"2024-09-05T12:27:44.713256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521832+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:27:44.521832+0000","last_peered":"2024-09-05T12:27:44.521832+0000","last_clean":"2024-09-05T12:27:44.521832+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:27:44.521832+0000","last_undegraded":"2024-09-05T12:27:44.521832+0000","last_fullsized":"2024-09-05T12:27:44.521832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884458+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:27:44.884458+0000","last_peered":"2024-09-05T12:27:44.884458+0000","last_clean":"2024-09-05T12:27:44.884458+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:27:44.884458+0000","last_undegraded":"2024-09-05T12:27:44.884458+0000","last_fullsized":"2024-09-05T12:27:44.884458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521673+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:27:44.521673+0000","last_peered":"2024-09-05T12:27:44.521673+0000","last_clean":"2024-09-05T12:27:44.521673+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:27:44.521673+0000","last_undegraded":"2024-09-05T12:27:44.521673+0000","last_fullsized":"2024-09-05T12:27:44.521673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553507+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:27:44.553507+0000","last_peered":"2024-09-05T12:27:44.553507+0000","last_clean":"2024-09-05T12:27:44.553507+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:27:44.553507+0000","last_undegraded":"2024-09-05T12:27:44.553507+0000","last_fullsized":"2024-09-05T12:27:44.553507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507103+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:27:44.507103+0000","last_peered":"2024-09-05T12:27:44.507103+0000","last_clean":"2024-09-05T12:27:44.507103+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:27:44.507103+0000","last_undegraded":"2024-09-05T12:27:44.507103+0000","last_fullsized":"2024-09-05T12:27:44.507103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'50","reported_seq":113,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.804109+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:27:44.804109+0000","last_peered":"2024-09-05T12:27:44.804109+0000","last_clean":"2024-09-05T12:27:44.804109+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:27:44.804109+0000","last_undegraded":"2024-09-05T12:27:44.804109+0000","last_fullsized":"2024-09-05T12:27:44.804109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505333+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:27:44.505333+0000","last_peered":"2024-09-05T12:27:44.505333+0000","last_clean":"2024-09-05T12:27:44.505333+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:27:44.505333+0000","last_undegraded":"2024-09-05T12:27:44.505333+0000","last_fullsized":"2024-09-05T12:27:44.505333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517516+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:27:44.517516+0000","last_peered":"2024-09-05T12:27:44.517516+0000","last_clean":"2024-09-05T12:27:44.517516+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:27:44.517516+0000","last_undegraded":"2024-09-05T12:27:44.517516+0000","last_fullsized":"2024-09-05T12:27:44.517516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509852+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:27:44.509852+0000","last_peered":"2024-09-05T12:27:44.509852+0000","last_clean":"2024-09-05T12:27:44.509852+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:27:44.509852+0000","last_undegraded":"2024-09-05T12:27:44.509852+0000","last_fullsized":"2024-09-05T12:27:44.509852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726921+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:27:44.726921+0000","last_peered":"2024-09-05T12:27:44.726921+0000","last_clean":"2024-09-05T12:27:44.726921+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:27:44.726921+0000","last_undegraded":"2024-09-05T12:27:44.726921+0000","last_fullsized":"2024-09-05T12:27:44.726921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515978+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:27:44.515978+0000","last_peered":"2024-09-05T12:27:44.515978+0000","last_clean":"2024-09-05T12:27:44.515978+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:27:44.515978+0000","last_undegraded":"2024-09-05T12:27:44.515978+0000","last_fullsized":"2024-09-05T12:27:44.515978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522745+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:27:44.522745+0000","last_peered":"2024-09-05T12:27:44.522745+0000","last_clean":"2024-09-05T12:27:44.522745+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:27:44.522745+0000","last_undegraded":"2024-09-05T12:27:44.522745+0000","last_fullsized":"2024-09-05T12:27:44.522745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509015+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:27:44.509015+0000","last_peered":"2024-09-05T12:27:44.509015+0000","last_clean":"2024-09-05T12:27:44.509015+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:27:44.509015+0000","last_undegraded":"2024-09-05T12:27:44.509015+0000","last_fullsized":"2024-09-05T12:27:44.509015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516026+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:27:44.516026+0000","last_peered":"2024-09-05T12:27:44.516026+0000","last_clean":"2024-09-05T12:27:44.516026+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:27:44.516026+0000","last_undegraded":"2024-09-05T12:27:44.516026+0000","last_fullsized":"2024-09-05T12:27:44.516026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.526494+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:27:45.526494+0000","last_peered":"2024-09-05T12:27:45.526494+0000","last_clean":"2024-09-05T12:27:45.526494+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:27:45.526494+0000","last_undegraded":"2024-09-05T12:27:45.526494+0000","last_fullsized":"2024-09-05T12:27:45.526494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517225+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:27:44.517225+0000","last_peered":"2024-09-05T12:27:44.517225+0000","last_clean":"2024-09-05T12:27:44.517225+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:27:44.517225+0000","last_undegraded":"2024-09-05T12:27:44.517225+0000","last_fullsized":"2024-09-05T12:27:44.517225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519362+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:27:44.519362+0000","last_peered":"2024-09-05T12:27:44.519362+0000","last_clean":"2024-09-05T12:27:44.519362+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:27:44.519362+0000","last_undegraded":"2024-09-05T12:27:44.519362+0000","last_fullsized":"2024-09-05T12:27:44.519362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.883725+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:27:44.883725+0000","last_peered":"2024-09-05T12:27:44.883725+0000","last_clean":"2024-09-05T12:27:44.883725+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:27:44.883725+0000","last_undegraded":"2024-09-05T12:27:44.883725+0000","last_fullsized":"2024-09-05T12:27:44.883725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516749+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:27:44.516749+0000","last_peered":"2024-09-05T12:27:44.516749+0000","last_clean":"2024-09-05T12:27:44.516749+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:27:44.516749+0000","last_undegraded":"2024-09-05T12:27:44.516749+0000","last_fullsized":"2024-09-05T12:27:44.516749+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516182+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:27:44.516182+0000","last_peered":"2024-09-05T12:27:44.516182+0000","last_clean":"2024-09-05T12:27:44.516182+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:27:44.516182+0000","last_undegraded":"2024-09-05T12:27:44.516182+0000","last_fullsized":"2024-09-05T12:27:44.516182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506602+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:27:44.506602+0000","last_peered":"2024-09-05T12:27:44.506602+0000","last_clean":"2024-09-05T12:27:44.506602+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:27:44.506602+0000","last_undegraded":"2024-09-05T12:27:44.506602+0000","last_fullsized":"2024-09-05T12:27:44.506602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520073+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:27:44.520073+0000","last_peered":"2024-09-05T12:27:44.520073+0000","last_clean":"2024-09-05T12:27:44.520073+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:27:44.520073+0000","last_undegraded":"2024-09-05T12:27:44.520073+0000","last_fullsized":"2024-09-05T12:27:44.520073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505827+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:27:44.505827+0000","last_peered":"2024-09-05T12:27:44.505827+0000","last_clean":"2024-09-05T12:27:44.505827+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:27:44.505827+0000","last_undegraded":"2024-09-05T12:27:44.505827+0000","last_fullsized":"2024-09-05T12:27:44.505827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522084+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:27:44.522084+0000","last_peered":"2024-09-05T12:27:44.522084+0000","last_clean":"2024-09-05T12:27:44.522084+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:27:44.522084+0000","last_undegraded":"2024-09-05T12:27:44.522084+0000","last_fullsized":"2024-09-05T12:27:44.522084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506892+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:27:44.506892+0000","last_peered":"2024-09-05T12:27:44.506892+0000","last_clean":"2024-09-05T12:27:44.506892+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:27:44.506892+0000","last_undegraded":"2024-09-05T12:27:44.506892+0000","last_fullsized":"2024-09-05T12:27:44.506892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506819+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:27:44.506819+0000","last_peered":"2024-09-05T12:27:44.506819+0000","last_clean":"2024-09-05T12:27:44.506819+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:27:44.506819+0000","last_undegraded":"2024-09-05T12:27:44.506819+0000","last_fullsized":"2024-09-05T12:27:44.506819+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507160+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:27:44.507160+0000","last_peered":"2024-09-05T12:27:44.507160+0000","last_clean":"2024-09-05T12:27:44.507160+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:27:44.507160+0000","last_undegraded":"2024-09-05T12:27:44.507160+0000","last_fullsized":"2024-09-05T12:27:44.507160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'9","reported_seq":181,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.507404+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:27:45.507404+0000","last_peered":"2024-09-05T12:27:45.507404+0000","last_clean":"2024-09-05T12:27:45.507404+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:27:45.507404+0000","last_undegraded":"2024-09-05T12:27:45.507404+0000","last_fullsized":"2024-09-05T12:27:45.507404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":5,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713421+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:27:44.713421+0000","last_peered":"2024-09-05T12:27:44.713421+0000","last_clean":"2024-09-05T12:27:44.713421+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:27:44.713421+0000","last_undegraded":"2024-09-05T12:27:44.713421+0000","last_fullsized":"2024-09-05T12:27:44.713421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506967+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:27:44.506967+0000","last_peered":"2024-09-05T12:27:44.506967+0000","last_clean":"2024-09-05T12:27:44.506967+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:27:44.506967+0000","last_undegraded":"2024-09-05T12:27:44.506967+0000","last_fullsized":"2024-09-05T12:27:44.506967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":90,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.800562+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:27:44.800562+0000","last_peered":"2024-09-05T12:27:44.800562+0000","last_clean":"2024-09-05T12:27:44.800562+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:27:44.800562+0000","last_undegraded":"2024-09-05T12:27:44.800562+0000","last_fullsized":"2024-09-05T12:27:44.800562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887282+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:27:44.887282+0000","last_peered":"2024-09-05T12:27:44.887282+0000","last_clean":"2024-09-05T12:27:44.887282+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:27:44.887282+0000","last_undegraded":"2024-09-05T12:27:44.887282+0000","last_fullsized":"2024-09-05T12:27:44.887282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507354+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:27:44.507354+0000","last_peered":"2024-09-05T12:27:44.507354+0000","last_clean":"2024-09-05T12:27:44.507354+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:27:44.507354+0000","last_undegraded":"2024-09-05T12:27:44.507354+0000","last_fullsized":"2024-09-05T12:27:44.507354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517307+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:27:44.517307+0000","last_peered":"2024-09-05T12:27:44.517307+0000","last_clean":"2024-09-05T12:27:44.517307+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:27:44.517307+0000","last_undegraded":"2024-09-05T12:27:44.517307+0000","last_fullsized":"2024-09-05T12:27:44.517307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519065+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:27:44.519065+0000","last_peered":"2024-09-05T12:27:44.519065+0000","last_clean":"2024-09-05T12:27:44.519065+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:27:44.519065+0000","last_undegraded":"2024-09-05T12:27:44.519065+0000","last_fullsized":"2024-09-05T12:27:44.519065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727189+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:27:44.727189+0000","last_peered":"2024-09-05T12:27:44.727189+0000","last_clean":"2024-09-05T12:27:44.727189+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:27:44.727189+0000","last_undegraded":"2024-09-05T12:27:44.727189+0000","last_fullsized":"2024-09-05T12:27:44.727189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727211+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:27:44.727211+0000","last_peered":"2024-09-05T12:27:44.727211+0000","last_clean":"2024-09-05T12:27:44.727211+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:27:44.727211+0000","last_undegraded":"2024-09-05T12:27:44.727211+0000","last_fullsized":"2024-09-05T12:27:44.727211+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519342+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:27:44.519342+0000","last_peered":"2024-09-05T12:27:44.519342+0000","last_clean":"2024-09-05T12:27:44.519342+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:27:44.519342+0000","last_undegraded":"2024-09-05T12:27:44.519342+0000","last_fullsized":"2024-09-05T12:27:44.519342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519536+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:27:44.519536+0000","last_peered":"2024-09-05T12:27:44.519536+0000","last_clean":"2024-09-05T12:27:44.519536+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:27:44.519536+0000","last_undegraded":"2024-09-05T12:27:44.519536+0000","last_fullsized":"2024-09-05T12:27:44.519536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509035+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:27:44.509035+0000","last_peered":"2024-09-05T12:27:44.509035+0000","last_clean":"2024-09-05T12:27:44.509035+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:27:44.509035+0000","last_undegraded":"2024-09-05T12:27:44.509035+0000","last_fullsized":"2024-09-05T12:27:44.509035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517190+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:27:44.517190+0000","last_peered":"2024-09-05T12:27:44.517190+0000","last_clean":"2024-09-05T12:27:44.517190+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:27:44.517190+0000","last_undegraded":"2024-09-05T12:27:44.517190+0000","last_fullsized":"2024-09-05T12:27:44.517190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506567+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:27:44.506567+0000","last_peered":"2024-09-05T12:27:44.506567+0000","last_clean":"2024-09-05T12:27:44.506567+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:27:44.506567+0000","last_undegraded":"2024-09-05T12:27:44.506567+0000","last_fullsized":"2024-09-05T12:27:44.506567+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'6","reported_seq":176,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.523301+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:27:45.523301+0000","last_peered":"2024-09-05T12:27:45.523301+0000","last_clean":"2024-09-05T12:27:45.523301+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:27:45.523301+0000","last_undegraded":"2024-09-05T12:27:45.523301+0000","last_fullsized":"2024-09-05T12:27:45.523301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523947+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:27:44.523947+0000","last_peered":"2024-09-05T12:27:44.523947+0000","last_clean":"2024-09-05T12:27:44.523947+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:27:44.523947+0000","last_undegraded":"2024-09-05T12:27:44.523947+0000","last_fullsized":"2024-09-05T12:27:44.523947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506890+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:27:44.506890+0000","last_peered":"2024-09-05T12:27:44.506890+0000","last_clean":"2024-09-05T12:27:44.506890+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:27:44.506890+0000","last_undegraded":"2024-09-05T12:27:44.506890+0000","last_fullsized":"2024-09-05T12:27:44.506890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519489+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:27:44.519489+0000","last_peered":"2024-09-05T12:27:44.519489+0000","last_clean":"2024-09-05T12:27:44.519489+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:27:44.519489+0000","last_undegraded":"2024-09-05T12:27:44.519489+0000","last_fullsized":"2024-09-05T12:27:44.519489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726953+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:27:44.726953+0000","last_peered":"2024-09-05T12:27:44.726953+0000","last_clean":"2024-09-05T12:27:44.726953+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:27:44.726953+0000","last_undegraded":"2024-09-05T12:27:44.726953+0000","last_fullsized":"2024-09-05T12:27:44.726953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517295+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:27:44.517295+0000","last_peered":"2024-09-05T12:27:44.517295+0000","last_clean":"2024-09-05T12:27:44.517295+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:27:44.517295+0000","last_undegraded":"2024-09-05T12:27:44.517295+0000","last_fullsized":"2024-09-05T12:27:44.517295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553650+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:27:44.553650+0000","last_peered":"2024-09-05T12:27:44.553650+0000","last_clean":"2024-09-05T12:27:44.553650+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:27:44.553650+0000","last_undegraded":"2024-09-05T12:27:44.553650+0000","last_fullsized":"2024-09-05T12:27:44.553650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520009+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:27:44.520009+0000","last_peered":"2024-09-05T12:27:44.520009+0000","last_clean":"2024-09-05T12:27:44.520009+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:27:44.520009+0000","last_undegraded":"2024-09-05T12:27:44.520009+0000","last_fullsized":"2024-09-05T12:27:44.520009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509567+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:27:44.509567+0000","last_peered":"2024-09-05T12:27:44.509567+0000","last_clean":"2024-09-05T12:27:44.509567+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:27:44.509567+0000","last_undegraded":"2024-09-05T12:27:44.509567+0000","last_fullsized":"2024-09-05T12:27:44.509567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":347,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.507482+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:27:45.507482+0000","last_peered":"2024-09-05T12:27:45.507482+0000","last_clean":"2024-09-05T12:27:45.507482+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:27:45.507482+0000","last_undegraded":"2024-09-05T12:27:45.507482+0000","last_fullsized":"2024-09-05T12:27:45.507482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68198,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":266,"num_read_kb":200,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522273+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:27:44.522273+0000","last_peered":"2024-09-05T12:27:44.522273+0000","last_clean":"2024-09-05T12:27:44.522273+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:27:44.522273+0000","last_undegraded":"2024-09-05T12:27:44.522273+0000","last_fullsized":"2024-09-05T12:27:44.522273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712959+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:27:44.712959+0000","last_peered":"2024-09-05T12:27:44.712959+0000","last_clean":"2024-09-05T12:27:44.712959+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:27:44.712959+0000","last_undegraded":"2024-09-05T12:27:44.712959+0000","last_fullsized":"2024-09-05T12:27:44.712959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520278+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:27:44.520278+0000","last_peered":"2024-09-05T12:27:44.520278+0000","last_clean":"2024-09-05T12:27:44.520278+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:27:44.520278+0000","last_undegraded":"2024-09-05T12:27:44.520278+0000","last_fullsized":"2024-09-05T12:27:44.520278+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522012+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:27:44.522012+0000","last_peered":"2024-09-05T12:27:44.522012+0000","last_clean":"2024-09-05T12:27:44.522012+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:27:44.522012+0000","last_undegraded":"2024-09-05T12:27:44.522012+0000","last_fullsized":"2024-09-05T12:27:44.522012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884499+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:27:44.884499+0000","last_peered":"2024-09-05T12:27:44.884499+0000","last_clean":"2024-09-05T12:27:44.884499+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:27:44.884499+0000","last_undegraded":"2024-09-05T12:27:44.884499+0000","last_fullsized":"2024-09-05T12:27:44.884499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507322+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:27:44.507322+0000","last_peered":"2024-09-05T12:27:44.507322+0000","last_clean":"2024-09-05T12:27:44.507322+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:27:44.507322+0000","last_undegraded":"2024-09-05T12:27:44.507322+0000","last_fullsized":"2024-09-05T12:27:44.507322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.508927+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:27:44.508927+0000","last_peered":"2024-09-05T12:27:44.508927+0000","last_clean":"2024-09-05T12:27:44.508927+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:27:44.508927+0000","last_undegraded":"2024-09-05T12:27:44.508927+0000","last_fullsized":"2024-09-05T12:27:44.508927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712976+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:27:44.712976+0000","last_peered":"2024-09-05T12:27:44.712976+0000","last_clean":"2024-09-05T12:27:44.712976+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:27:44.712976+0000","last_undegraded":"2024-09-05T12:27:44.712976+0000","last_fullsized":"2024-09-05T12:27:44.712976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'7","reported_seq":301,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.506888+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:27:45.506888+0000","last_peered":"2024-09-05T12:27:45.506888+0000","last_clean":"2024-09-05T12:27:45.506888+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:27:45.506888+0000","last_undegraded":"2024-09-05T12:27:45.506888+0000","last_fullsized":"2024-09-05T12:27:45.506888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505528+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:27:44.505528+0000","last_peered":"2024-09-05T12:27:44.505528+0000","last_clean":"2024-09-05T12:27:44.505528+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:27:44.505528+0000","last_undegraded":"2024-09-05T12:27:44.505528+0000","last_fullsized":"2024-09-05T12:27:44.505528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887297+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:27:44.887297+0000","last_peered":"2024-09-05T12:27:44.887297+0000","last_clean":"2024-09-05T12:27:44.887297+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:27:44.887297+0000","last_undegraded":"2024-09-05T12:27:44.887297+0000","last_fullsized":"2024-09-05T12:27:44.887297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522239+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:27:44.522239+0000","last_peered":"2024-09-05T12:27:44.522239+0000","last_clean":"2024-09-05T12:27:44.522239+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:27:44.522239+0000","last_undegraded":"2024-09-05T12:27:44.522239+0000","last_fullsized":"2024-09-05T12:27:44.522239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519527+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:27:44.519527+0000","last_peered":"2024-09-05T12:27:44.519527+0000","last_clean":"2024-09-05T12:27:44.519527+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:27:44.519527+0000","last_undegraded":"2024-09-05T12:27:44.519527+0000","last_fullsized":"2024-09-05T12:27:44.519527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727158+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:27:44.727158+0000","last_peered":"2024-09-05T12:27:44.727158+0000","last_clean":"2024-09-05T12:27:44.727158+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:27:44.727158+0000","last_undegraded":"2024-09-05T12:27:44.727158+0000","last_fullsized":"2024-09-05T12:27:44.727158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505490+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:27:44.505490+0000","last_peered":"2024-09-05T12:27:44.505490+0000","last_clean":"2024-09-05T12:27:44.505490+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:27:44.505490+0000","last_undegraded":"2024-09-05T12:27:44.505490+0000","last_fullsized":"2024-09-05T12:27:44.505490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516334+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:27:44.516334+0000","last_peered":"2024-09-05T12:27:44.516334+0000","last_clean":"2024-09-05T12:27:44.516334+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:27:44.516334+0000","last_undegraded":"2024-09-05T12:27:44.516334+0000","last_fullsized":"2024-09-05T12:27:44.516334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515852+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:27:44.515852+0000","last_peered":"2024-09-05T12:27:44.515852+0000","last_clean":"2024-09-05T12:27:44.515852+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:27:44.515852+0000","last_undegraded":"2024-09-05T12:27:44.515852+0000","last_fullsized":"2024-09-05T12:27:44.515852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.518959+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:27:44.518959+0000","last_peered":"2024-09-05T12:27:44.518959+0000","last_clean":"2024-09-05T12:27:44.518959+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:27:44.518959+0000","last_undegraded":"2024-09-05T12:27:44.518959+0000","last_fullsized":"2024-09-05T12:27:44.518959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553680+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:27:44.553680+0000","last_peered":"2024-09-05T12:27:44.553680+0000","last_clean":"2024-09-05T12:27:44.553680+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:27:44.553680+0000","last_undegraded":"2024-09-05T12:27:44.553680+0000","last_fullsized":"2024-09-05T12:27:44.553680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'7","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.521635+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:27:45.521635+0000","last_peered":"2024-09-05T12:27:45.521635+0000","last_clean":"2024-09-05T12:27:45.521635+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:27:45.521635+0000","last_undegraded":"2024-09-05T12:27:45.521635+0000","last_fullsized":"2024-09-05T12:27:45.521635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509141+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:27:44.509141+0000","last_peered":"2024-09-05T12:27:44.509141+0000","last_clean":"2024-09-05T12:27:44.509141+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:27:44.509141+0000","last_undegraded":"2024-09-05T12:27:44.509141+0000","last_fullsized":"2024-09-05T12:27:44.509141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553781+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:27:44.553781+0000","last_peered":"2024-09-05T12:27:44.553781+0000","last_clean":"2024-09-05T12:27:44.553781+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:27:44.553781+0000","last_undegraded":"2024-09-05T12:27:44.553781+0000","last_fullsized":"2024-09-05T12:27:44.553781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726440+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:27:44.726440+0000","last_peered":"2024-09-05T12:27:44.726440+0000","last_clean":"2024-09-05T12:27:44.726440+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:27:44.726440+0000","last_undegraded":"2024-09-05T12:27:44.726440+0000","last_fullsized":"2024-09-05T12:27:44.726440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553578+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:27:44.553578+0000","last_peered":"2024-09-05T12:27:44.553578+0000","last_clean":"2024-09-05T12:27:44.553578+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:27:44.553578+0000","last_undegraded":"2024-09-05T12:27:44.553578+0000","last_fullsized":"2024-09-05T12:27:44.553578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553078+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:27:44.553078+0000","last_peered":"2024-09-05T12:27:44.553078+0000","last_clean":"2024-09-05T12:27:44.553078+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:27:44.553078+0000","last_undegraded":"2024-09-05T12:27:44.553078+0000","last_fullsized":"2024-09-05T12:27:44.553078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887351+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:27:44.887351+0000","last_peered":"2024-09-05T12:27:44.887351+0000","last_clean":"2024-09-05T12:27:44.887351+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:27:44.887351+0000","last_undegraded":"2024-09-05T12:27:44.887351+0000","last_fullsized":"2024-09-05T12:27:44.887351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517395+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:27:44.517395+0000","last_peered":"2024-09-05T12:27:44.517395+0000","last_clean":"2024-09-05T12:27:44.517395+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:27:44.517395+0000","last_undegraded":"2024-09-05T12:27:44.517395+0000","last_fullsized":"2024-09-05T12:27:44.517395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726513+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:27:44.726513+0000","last_peered":"2024-09-05T12:27:44.726513+0000","last_clean":"2024-09-05T12:27:44.726513+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:27:44.726513+0000","last_undegraded":"2024-09-05T12:27:44.726513+0000","last_fullsized":"2024-09-05T12:27:44.726513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553154+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:27:44.553154+0000","last_peered":"2024-09-05T12:27:44.553154+0000","last_clean":"2024-09-05T12:27:44.553154+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:27:44.553154+0000","last_undegraded":"2024-09-05T12:27:44.553154+0000","last_fullsized":"2024-09-05T12:27:44.553154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506190+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:27:44.506190+0000","last_peered":"2024-09-05T12:27:44.506190+0000","last_clean":"2024-09-05T12:27:44.506190+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:27:44.506190+0000","last_undegraded":"2024-09-05T12:27:44.506190+0000","last_fullsized":"2024-09-05T12:27:44.506190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519213+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:27:44.519213+0000","last_peered":"2024-09-05T12:27:44.519213+0000","last_clean":"2024-09-05T12:27:44.519213+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:27:44.519213+0000","last_undegraded":"2024-09-05T12:27:44.519213+0000","last_fullsized":"2024-09-05T12:27:44.519213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505694+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:27:44.505694+0000","last_peered":"2024-09-05T12:27:44.505694+0000","last_clean":"2024-09-05T12:27:44.505694+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:27:44.505694+0000","last_undegraded":"2024-09-05T12:27:44.505694+0000","last_fullsized":"2024-09-05T12:27:44.505694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712927+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:27:44.712927+0000","last_peered":"2024-09-05T12:27:44.712927+0000","last_clean":"2024-09-05T12:27:44.712927+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:27:44.712927+0000","last_undegraded":"2024-09-05T12:27:44.712927+0000","last_fullsized":"2024-09-05T12:27:44.712927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516572+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:27:44.516572+0000","last_peered":"2024-09-05T12:27:44.516572+0000","last_clean":"2024-09-05T12:27:44.516572+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:27:44.516572+0000","last_undegraded":"2024-09-05T12:27:44.516572+0000","last_fullsized":"2024-09-05T12:27:44.516572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726524+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:27:44.726524+0000","last_peered":"2024-09-05T12:27:44.726524+0000","last_clean":"2024-09-05T12:27:44.726524+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:27:44.726524+0000","last_undegraded":"2024-09-05T12:27:44.726524+0000","last_fullsized":"2024-09-05T12:27:44.726524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553110+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:27:44.553110+0000","last_peered":"2024-09-05T12:27:44.553110+0000","last_clean":"2024-09-05T12:27:44.553110+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:27:44.553110+0000","last_undegraded":"2024-09-05T12:27:44.553110+0000","last_fullsized":"2024-09-05T12:27:44.553110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523087+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:27:44.523087+0000","last_peered":"2024-09-05T12:27:44.523087+0000","last_clean":"2024-09-05T12:27:44.523087+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:27:44.523087+0000","last_undegraded":"2024-09-05T12:27:44.523087+0000","last_fullsized":"2024-09-05T12:27:44.523087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727310+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:27:44.727310+0000","last_peered":"2024-09-05T12:27:44.727310+0000","last_clean":"2024-09-05T12:27:44.727310+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:27:44.727310+0000","last_undegraded":"2024-09-05T12:27:44.727310+0000","last_fullsized":"2024-09-05T12:27:44.727310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712915+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:27:44.712915+0000","last_peered":"2024-09-05T12:27:44.712915+0000","last_clean":"2024-09-05T12:27:44.712915+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:27:44.712915+0000","last_undegraded":"2024-09-05T12:27:44.712915+0000","last_fullsized":"2024-09-05T12:27:44.712915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884369+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:27:44.884369+0000","last_peered":"2024-09-05T12:27:44.884369+0000","last_clean":"2024-09-05T12:27:44.884369+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:27:44.884369+0000","last_undegraded":"2024-09-05T12:27:44.884369+0000","last_fullsized":"2024-09-05T12:27:44.884369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884201+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:27:44.884201+0000","last_peered":"2024-09-05T12:27:44.884201+0000","last_clean":"2024-09-05T12:27:44.884201+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:27:44.884201+0000","last_undegraded":"2024-09-05T12:27:44.884201+0000","last_fullsized":"2024-09-05T12:27:44.884201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516382+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:27:44.516382+0000","last_peered":"2024-09-05T12:27:44.516382+0000","last_clean":"2024-09-05T12:27:44.516382+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:27:44.516382+0000","last_undegraded":"2024-09-05T12:27:44.516382+0000","last_fullsized":"2024-09-05T12:27:44.516382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520915+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:27:44.520915+0000","last_peered":"2024-09-05T12:27:44.520915+0000","last_clean":"2024-09-05T12:27:44.520915+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:27:44.520915+0000","last_undegraded":"2024-09-05T12:27:44.520915+0000","last_fullsized":"2024-09-05T12:27:44.520915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507111+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:27:44.507111+0000","last_peered":"2024-09-05T12:27:44.507111+0000","last_clean":"2024-09-05T12:27:44.507111+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:27:44.507111+0000","last_undegraded":"2024-09-05T12:27:44.507111+0000","last_fullsized":"2024-09-05T12:27:44.507111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516166+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:27:44.516166+0000","last_peered":"2024-09-05T12:27:44.516166+0000","last_clean":"2024-09-05T12:27:44.516166+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:27:44.516166+0000","last_undegraded":"2024-09-05T12:27:44.516166+0000","last_fullsized":"2024-09-05T12:27:44.516166+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713259+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:27:44.713259+0000","last_peered":"2024-09-05T12:27:44.713259+0000","last_clean":"2024-09-05T12:27:44.713259+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:27:44.713259+0000","last_undegraded":"2024-09-05T12:27:44.713259+0000","last_fullsized":"2024-09-05T12:27:44.713259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505460+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:27:44.505460+0000","last_peered":"2024-09-05T12:27:44.505460+0000","last_clean":"2024-09-05T12:27:44.505460+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:27:44.505460+0000","last_undegraded":"2024-09-05T12:27:44.505460+0000","last_fullsized":"2024-09-05T12:27:44.505460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506619+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:27:44.506619+0000","last_peered":"2024-09-05T12:27:44.506619+0000","last_clean":"2024-09-05T12:27:44.506619+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:27:44.506619+0000","last_undegraded":"2024-09-05T12:27:44.506619+0000","last_fullsized":"2024-09-05T12:27:44.506619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515791+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:27:44.515791+0000","last_peered":"2024-09-05T12:27:44.515791+0000","last_clean":"2024-09-05T12:27:44.515791+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:27:44.515791+0000","last_undegraded":"2024-09-05T12:27:44.515791+0000","last_fullsized":"2024-09-05T12:27:44.515791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509128+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:27:44.509128+0000","last_peered":"2024-09-05T12:27:44.509128+0000","last_clean":"2024-09-05T12:27:44.509128+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:27:44.509128+0000","last_undegraded":"2024-09-05T12:27:44.509128+0000","last_fullsized":"2024-09-05T12:27:44.509128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521146+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:27:44.521146+0000","last_peered":"2024-09-05T12:27:44.521146+0000","last_clean":"2024-09-05T12:27:44.521146+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:27:44.521146+0000","last_undegraded":"2024-09-05T12:27:44.521146+0000","last_fullsized":"2024-09-05T12:27:44.521146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.885260+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:27:44.885260+0000","last_peered":"2024-09-05T12:27:44.885260+0000","last_clean":"2024-09-05T12:27:44.885260+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:27:44.885260+0000","last_undegraded":"2024-09-05T12:27:44.885260+0000","last_fullsized":"2024-09-05T12:27:44.885260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887150+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:27:44.887150+0000","last_peered":"2024-09-05T12:27:44.887150+0000","last_clean":"2024-09-05T12:27:44.887150+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:27:44.887150+0000","last_undegraded":"2024-09-05T12:27:44.887150+0000","last_fullsized":"2024-09-05T12:27:44.887150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727308+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:27:44.727308+0000","last_peered":"2024-09-05T12:27:44.727308+0000","last_clean":"2024-09-05T12:27:44.727308+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:27:44.727308+0000","last_undegraded":"2024-09-05T12:27:44.727308+0000","last_fullsized":"2024-09-05T12:27:44.727308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602246,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1411,"num_read_kb":824,"num_write":383,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1806738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449137,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":534079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678069,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28976,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716176,"statfs":{"total":95995035648,"available":95965364224,"internally_reserved":0,"allocated":1937408,"data_stored":1334434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841594,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1515520,"data_stored":936477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103232,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1507328,"data_stored":936833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1716224,"data_stored":1141235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528394,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":539158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29020,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716132,"statfs":{"total":95995035648,"available":95965319168,"internally_reserved":0,"allocated":1859584,"data_stored":1267238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":543232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1063289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1171456,"data_stored":594543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1150976,"data_stored":593478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":541541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:27:49.995 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:27:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='client.25307 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-05T12:27:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:50.261 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:49 smithi174 ceph-mon[34313]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:50.391 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:50.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:50.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='client.25307 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:50.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-05T12:27:50.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:50.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:50.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:50.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:49 smithi071 ceph-mon[29845]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:50.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:50.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='client.25307 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:27:50.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Updating smithi071:/etc/ceph/ceph.conf 2024-09-05T12:27:50.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-05T12:27:50.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-05T12:27:50.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:50.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:49 smithi123 ceph-mon[34321]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.011+0000 7febc0e00640 1 -- 172.21.15.71:0/2429418517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc1049c0 msgr2=0x7febbc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.011+0000 7febc0e00640 1 --2- 172.21.15.71:0/2429418517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc1049c0 0x7febbc108a10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7feba8009900 tx=0x7feba802a410 comp rx=0 tx=0).stop 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 -- 172.21.15.71:0/2429418517 shutdown_connections 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 --2- 172.21.15.71:0/2429418517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 0x7febbc110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 --2- 172.21.15.71:0/2429418517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc1049c0 0x7febbc108a10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 --2- 172.21.15.71:0/2429418517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febbc104010 0x7febbc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 -- 172.21.15.71:0/2429418517 >> 172.21.15.71:0/2429418517 conn(0x7febbc0ffa50 msgr2=0x7febbc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 -- 172.21.15.71:0/2429418517 shutdown_connections 2024-09-05T12:27:51.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.012+0000 7febc0e00640 1 -- 172.21.15.71:0/2429418517 wait complete. 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 Processor -- start 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 -- start start 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 0x7febbc1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febbc1049c0 0x7febbc1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 0x7febbc19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febbc1141b0 con 0x7febbc104010 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febbc114030 con 0x7febbc1049c0 2024-09-05T12:27:51.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febc0e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febbc114330 con 0x7febbc109140 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febb9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 0x7febbc1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.013+0000 7febb9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febbc1049c0 0x7febbc1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 0x7febbc1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42114/0 (socket says 172.21.15.71:42114) 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 -- 172.21.15.71:0/3663371532 learned_addr learned my addr 172.21.15.71:0/3663371532 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febba800640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 0x7febbc19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 -- 172.21.15.71:0/3663371532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 msgr2=0x7febbc19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 0x7febbc19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 -- 172.21.15.71:0/3663371532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febbc1049c0 msgr2=0x7febbc1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febbc1049c0 0x7febbc1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 -- 172.21.15.71:0/3663371532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc19acf0 con 0x7febbc104010 2024-09-05T12:27:51.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febba800640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 0x7febbc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.014+0000 7febb9e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 0x7febbc1a0400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febac00d950 tx=0x7febac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.015+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac017030 con 0x7febbc104010 2024-09-05T12:27:51.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.015+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febac004020 con 0x7febbc104010 2024-09-05T12:27:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.015+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febac0047a0 con 0x7febbc104010 2024-09-05T12:27:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.015+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc19afe0 con 0x7febbc104010 2024-09-05T12:27:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.015+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a72d0 con 0x7febbc104010 2024-09-05T12:27:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.016+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc1043f0 con 0x7febbc104010 2024-09-05T12:27:51.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.018+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7febac0041d0 con 0x7febbc104010 2024-09-05T12:27:51.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.020+0000 7febb2a00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feba00a2d80 0x7feba00a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:51.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.020+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7febac0cf2d0 con 0x7febbc104010 2024-09-05T12:27:51.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.020+0000 7febb9400640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feba00a2d80 0x7feba00a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:51.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.021+0000 7febb9400640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feba00a2d80 0x7feba00a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feba8000c00 tx=0x7feba8005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:51.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.025+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febac09b9e0 con 0x7febbc104010 2024-09-05T12:27:51.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:51.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:51.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:51.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:51 smithi071 ceph-mon[29845]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.227+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7febbc062630 con 0x7feba00a2d80 2024-09-05T12:27:51.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.235+0000 7febb2a00640 1 -- 172.21.15.71:0/3663371532 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339496 (secure 0 0 0) 0x7febbc062630 con 0x7feba00a2d80 2024-09-05T12:27:51.243 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:51.245 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.240+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feba00a2d80 msgr2=0x7feba00a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.240+0000 7febc0e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feba00a2d80 0x7feba00a5240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feba8000c00 tx=0x7feba8005b50 comp rx=0 tx=0).stop 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.240+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 msgr2=0x7febbc1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.240+0000 7febc0e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 0x7febbc1a0400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febac00d950 tx=0x7febac00de10 comp rx=0 tx=0).stop 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 shutdown_connections 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febbc109140 0x7febbc19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febbc1049c0 0x7febbc1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feba00a2d80 0x7feba00a5240 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 --2- 172.21.15.71:0/3663371532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febbc104010 0x7febbc1a0400 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:51.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 >> 172.21.15.71:0/3663371532 conn(0x7febbc0ffa50 msgr2=0x7febbc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:51.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 shutdown_connections 2024-09-05T12:27:51.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:51.241+0000 7febc0e00640 1 -- 172.21.15.71:0/3663371532 wait complete. 2024-09-05T12:27:51.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:51.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:51.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:51.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:51 smithi123 ceph-mon[34321]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.conf 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: Updating smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: Updating smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:51.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:51 smithi174 ceph-mon[34313]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-09-05T12:27:52.070 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-09-05T12:27:50.525433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192848,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1547,"num_read_kb":943,"num_write":566,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":946,"ondisk_log_size":946,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378504,"kb_used_data":16512,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124563320,"statfs":{"total":1151940427776,"available":1151552839680,"internally_reserved":0,"allocated":16908288,"data_stored":10025537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.984954"},"pg_stats":[{"pgid":"2.1f","version":"95'14","reported_seq":182,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.883114+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:27:44.883114+0000","last_peered":"2024-09-05T12:27:44.883114+0000","last_clean":"2024-09-05T12:27:44.883114+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:27:44.883114+0000","last_undegraded":"2024-09-05T12:27:44.883114+0000","last_fullsized":"2024-09-05T12:27:44.883114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506128+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:27:44.506128+0000","last_peered":"2024-09-05T12:27:44.506128+0000","last_clean":"2024-09-05T12:27:44.506128+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:27:44.506128+0000","last_undegraded":"2024-09-05T12:27:44.506128+0000","last_fullsized":"2024-09-05T12:27:44.506128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553411+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:27:44.553411+0000","last_peered":"2024-09-05T12:27:44.553411+0000","last_clean":"2024-09-05T12:27:44.553411+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:27:44.553411+0000","last_undegraded":"2024-09-05T12:27:44.553411+0000","last_fullsized":"2024-09-05T12:27:44.553411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506439+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:27:44.506439+0000","last_peered":"2024-09-05T12:27:44.506439+0000","last_clean":"2024-09-05T12:27:44.506439+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:27:44.506439+0000","last_undegraded":"2024-09-05T12:27:44.506439+0000","last_fullsized":"2024-09-05T12:27:44.506439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884712+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:27:44.884712+0000","last_peered":"2024-09-05T12:27:44.884712+0000","last_clean":"2024-09-05T12:27:44.884712+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:27:44.884712+0000","last_undegraded":"2024-09-05T12:27:44.884712+0000","last_fullsized":"2024-09-05T12:27:44.884712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521451+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:27:44.521451+0000","last_peered":"2024-09-05T12:27:44.521451+0000","last_clean":"2024-09-05T12:27:44.521451+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:27:44.521451+0000","last_undegraded":"2024-09-05T12:27:44.521451+0000","last_fullsized":"2024-09-05T12:27:44.521451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'13","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509779+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:27:44.509779+0000","last_peered":"2024-09-05T12:27:44.509779+0000","last_clean":"2024-09-05T12:27:44.509779+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:27:44.509779+0000","last_undegraded":"2024-09-05T12:27:44.509779+0000","last_fullsized":"2024-09-05T12:27:44.509779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713202+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:27:44.713202+0000","last_peered":"2024-09-05T12:27:44.713202+0000","last_clean":"2024-09-05T12:27:44.713202+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:27:44.713202+0000","last_undegraded":"2024-09-05T12:27:44.713202+0000","last_fullsized":"2024-09-05T12:27:44.713202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521661+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:27:44.521661+0000","last_peered":"2024-09-05T12:27:44.521661+0000","last_clean":"2024-09-05T12:27:44.521661+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:27:44.521661+0000","last_undegraded":"2024-09-05T12:27:44.521661+0000","last_fullsized":"2024-09-05T12:27:44.521661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509541+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:27:44.509541+0000","last_peered":"2024-09-05T12:27:44.509541+0000","last_clean":"2024-09-05T12:27:44.509541+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:27:44.509541+0000","last_undegraded":"2024-09-05T12:27:44.509541+0000","last_fullsized":"2024-09-05T12:27:44.509541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520014+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:27:44.520014+0000","last_peered":"2024-09-05T12:27:44.520014+0000","last_clean":"2024-09-05T12:27:44.520014+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:27:44.520014+0000","last_undegraded":"2024-09-05T12:27:44.520014+0000","last_fullsized":"2024-09-05T12:27:44.520014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521095+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:27:44.521095+0000","last_peered":"2024-09-05T12:27:44.521095+0000","last_clean":"2024-09-05T12:27:44.521095+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:27:44.521095+0000","last_undegraded":"2024-09-05T12:27:44.521095+0000","last_fullsized":"2024-09-05T12:27:44.521095+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506770+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:27:44.506770+0000","last_peered":"2024-09-05T12:27:44.506770+0000","last_clean":"2024-09-05T12:27:44.506770+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:27:44.506770+0000","last_undegraded":"2024-09-05T12:27:44.506770+0000","last_fullsized":"2024-09-05T12:27:44.506770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509274+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:27:44.509274+0000","last_peered":"2024-09-05T12:27:44.509274+0000","last_clean":"2024-09-05T12:27:44.509274+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:27:44.509274+0000","last_undegraded":"2024-09-05T12:27:44.509274+0000","last_fullsized":"2024-09-05T12:27:44.509274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521721+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:27:44.521721+0000","last_peered":"2024-09-05T12:27:44.521721+0000","last_clean":"2024-09-05T12:27:44.521721+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:27:44.521721+0000","last_undegraded":"2024-09-05T12:27:44.521721+0000","last_fullsized":"2024-09-05T12:27:44.521721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505685+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:27:44.505685+0000","last_peered":"2024-09-05T12:27:44.505685+0000","last_clean":"2024-09-05T12:27:44.505685+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:27:44.505685+0000","last_undegraded":"2024-09-05T12:27:44.505685+0000","last_fullsized":"2024-09-05T12:27:44.505685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726946+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:27:44.726946+0000","last_peered":"2024-09-05T12:27:44.726946+0000","last_clean":"2024-09-05T12:27:44.726946+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:27:44.726946+0000","last_undegraded":"2024-09-05T12:27:44.726946+0000","last_fullsized":"2024-09-05T12:27:44.726946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521509+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:27:44.521509+0000","last_peered":"2024-09-05T12:27:44.521509+0000","last_clean":"2024-09-05T12:27:44.521509+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:27:44.521509+0000","last_undegraded":"2024-09-05T12:27:44.521509+0000","last_fullsized":"2024-09-05T12:27:44.521509+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726837+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:27:44.726837+0000","last_peered":"2024-09-05T12:27:44.726837+0000","last_clean":"2024-09-05T12:27:44.726837+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:27:44.726837+0000","last_undegraded":"2024-09-05T12:27:44.726837+0000","last_fullsized":"2024-09-05T12:27:44.726837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505891+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:27:44.505891+0000","last_peered":"2024-09-05T12:27:44.505891+0000","last_clean":"2024-09-05T12:27:44.505891+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:27:44.505891+0000","last_undegraded":"2024-09-05T12:27:44.505891+0000","last_fullsized":"2024-09-05T12:27:44.505891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523836+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:27:44.523836+0000","last_peered":"2024-09-05T12:27:44.523836+0000","last_clean":"2024-09-05T12:27:44.523836+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:27:44.523836+0000","last_undegraded":"2024-09-05T12:27:44.523836+0000","last_fullsized":"2024-09-05T12:27:44.523836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509025+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:27:44.509025+0000","last_peered":"2024-09-05T12:27:44.509025+0000","last_clean":"2024-09-05T12:27:44.509025+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:27:44.509025+0000","last_undegraded":"2024-09-05T12:27:44.509025+0000","last_fullsized":"2024-09-05T12:27:44.509025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713256+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:27:44.713256+0000","last_peered":"2024-09-05T12:27:44.713256+0000","last_clean":"2024-09-05T12:27:44.713256+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:27:44.713256+0000","last_undegraded":"2024-09-05T12:27:44.713256+0000","last_fullsized":"2024-09-05T12:27:44.713256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521832+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:27:44.521832+0000","last_peered":"2024-09-05T12:27:44.521832+0000","last_clean":"2024-09-05T12:27:44.521832+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:27:44.521832+0000","last_undegraded":"2024-09-05T12:27:44.521832+0000","last_fullsized":"2024-09-05T12:27:44.521832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884458+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:27:44.884458+0000","last_peered":"2024-09-05T12:27:44.884458+0000","last_clean":"2024-09-05T12:27:44.884458+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:27:44.884458+0000","last_undegraded":"2024-09-05T12:27:44.884458+0000","last_fullsized":"2024-09-05T12:27:44.884458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521673+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:27:44.521673+0000","last_peered":"2024-09-05T12:27:44.521673+0000","last_clean":"2024-09-05T12:27:44.521673+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:27:44.521673+0000","last_undegraded":"2024-09-05T12:27:44.521673+0000","last_fullsized":"2024-09-05T12:27:44.521673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553507+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:27:44.553507+0000","last_peered":"2024-09-05T12:27:44.553507+0000","last_clean":"2024-09-05T12:27:44.553507+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:27:44.553507+0000","last_undegraded":"2024-09-05T12:27:44.553507+0000","last_fullsized":"2024-09-05T12:27:44.553507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507103+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:27:44.507103+0000","last_peered":"2024-09-05T12:27:44.507103+0000","last_clean":"2024-09-05T12:27:44.507103+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:27:44.507103+0000","last_undegraded":"2024-09-05T12:27:44.507103+0000","last_fullsized":"2024-09-05T12:27:44.507103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'50","reported_seq":113,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.804109+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:27:44.804109+0000","last_peered":"2024-09-05T12:27:44.804109+0000","last_clean":"2024-09-05T12:27:44.804109+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:27:44.804109+0000","last_undegraded":"2024-09-05T12:27:44.804109+0000","last_fullsized":"2024-09-05T12:27:44.804109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":90,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505333+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:27:44.505333+0000","last_peered":"2024-09-05T12:27:44.505333+0000","last_clean":"2024-09-05T12:27:44.505333+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:27:44.505333+0000","last_undegraded":"2024-09-05T12:27:44.505333+0000","last_fullsized":"2024-09-05T12:27:44.505333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517516+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:27:44.517516+0000","last_peered":"2024-09-05T12:27:44.517516+0000","last_clean":"2024-09-05T12:27:44.517516+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:27:44.517516+0000","last_undegraded":"2024-09-05T12:27:44.517516+0000","last_fullsized":"2024-09-05T12:27:44.517516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509852+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:27:44.509852+0000","last_peered":"2024-09-05T12:27:44.509852+0000","last_clean":"2024-09-05T12:27:44.509852+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:27:44.509852+0000","last_undegraded":"2024-09-05T12:27:44.509852+0000","last_fullsized":"2024-09-05T12:27:44.509852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726921+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:27:44.726921+0000","last_peered":"2024-09-05T12:27:44.726921+0000","last_clean":"2024-09-05T12:27:44.726921+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:27:44.726921+0000","last_undegraded":"2024-09-05T12:27:44.726921+0000","last_fullsized":"2024-09-05T12:27:44.726921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515978+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:27:44.515978+0000","last_peered":"2024-09-05T12:27:44.515978+0000","last_clean":"2024-09-05T12:27:44.515978+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:27:44.515978+0000","last_undegraded":"2024-09-05T12:27:44.515978+0000","last_fullsized":"2024-09-05T12:27:44.515978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522745+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:27:44.522745+0000","last_peered":"2024-09-05T12:27:44.522745+0000","last_clean":"2024-09-05T12:27:44.522745+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:27:44.522745+0000","last_undegraded":"2024-09-05T12:27:44.522745+0000","last_fullsized":"2024-09-05T12:27:44.522745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509015+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:27:44.509015+0000","last_peered":"2024-09-05T12:27:44.509015+0000","last_clean":"2024-09-05T12:27:44.509015+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:27:44.509015+0000","last_undegraded":"2024-09-05T12:27:44.509015+0000","last_fullsized":"2024-09-05T12:27:44.509015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516026+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:27:44.516026+0000","last_peered":"2024-09-05T12:27:44.516026+0000","last_clean":"2024-09-05T12:27:44.516026+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:27:44.516026+0000","last_undegraded":"2024-09-05T12:27:44.516026+0000","last_fullsized":"2024-09-05T12:27:44.516026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.526494+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:27:45.526494+0000","last_peered":"2024-09-05T12:27:45.526494+0000","last_clean":"2024-09-05T12:27:45.526494+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:27:45.526494+0000","last_undegraded":"2024-09-05T12:27:45.526494+0000","last_fullsized":"2024-09-05T12:27:45.526494+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517225+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:27:44.517225+0000","last_peered":"2024-09-05T12:27:44.517225+0000","last_clean":"2024-09-05T12:27:44.517225+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:27:44.517225+0000","last_undegraded":"2024-09-05T12:27:44.517225+0000","last_fullsized":"2024-09-05T12:27:44.517225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519362+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:27:44.519362+0000","last_peered":"2024-09-05T12:27:44.519362+0000","last_clean":"2024-09-05T12:27:44.519362+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:27:44.519362+0000","last_undegraded":"2024-09-05T12:27:44.519362+0000","last_fullsized":"2024-09-05T12:27:44.519362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":175,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.883725+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:27:44.883725+0000","last_peered":"2024-09-05T12:27:44.883725+0000","last_clean":"2024-09-05T12:27:44.883725+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:27:44.883725+0000","last_undegraded":"2024-09-05T12:27:44.883725+0000","last_fullsized":"2024-09-05T12:27:44.883725+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":128,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516749+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:27:44.516749+0000","last_peered":"2024-09-05T12:27:44.516749+0000","last_clean":"2024-09-05T12:27:44.516749+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:27:44.516749+0000","last_undegraded":"2024-09-05T12:27:44.516749+0000","last_fullsized":"2024-09-05T12:27:44.516749+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516182+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:27:44.516182+0000","last_peered":"2024-09-05T12:27:44.516182+0000","last_clean":"2024-09-05T12:27:44.516182+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:27:44.516182+0000","last_undegraded":"2024-09-05T12:27:44.516182+0000","last_fullsized":"2024-09-05T12:27:44.516182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506602+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:27:44.506602+0000","last_peered":"2024-09-05T12:27:44.506602+0000","last_clean":"2024-09-05T12:27:44.506602+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:27:44.506602+0000","last_undegraded":"2024-09-05T12:27:44.506602+0000","last_fullsized":"2024-09-05T12:27:44.506602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520073+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:27:44.520073+0000","last_peered":"2024-09-05T12:27:44.520073+0000","last_clean":"2024-09-05T12:27:44.520073+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:27:44.520073+0000","last_undegraded":"2024-09-05T12:27:44.520073+0000","last_fullsized":"2024-09-05T12:27:44.520073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505827+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:27:44.505827+0000","last_peered":"2024-09-05T12:27:44.505827+0000","last_clean":"2024-09-05T12:27:44.505827+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:27:44.505827+0000","last_undegraded":"2024-09-05T12:27:44.505827+0000","last_fullsized":"2024-09-05T12:27:44.505827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522084+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:27:44.522084+0000","last_peered":"2024-09-05T12:27:44.522084+0000","last_clean":"2024-09-05T12:27:44.522084+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:27:44.522084+0000","last_undegraded":"2024-09-05T12:27:44.522084+0000","last_fullsized":"2024-09-05T12:27:44.522084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":364,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506892+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:27:44.506892+0000","last_peered":"2024-09-05T12:27:44.506892+0000","last_clean":"2024-09-05T12:27:44.506892+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:27:44.506892+0000","last_undegraded":"2024-09-05T12:27:44.506892+0000","last_fullsized":"2024-09-05T12:27:44.506892+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506819+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:27:44.506819+0000","last_peered":"2024-09-05T12:27:44.506819+0000","last_clean":"2024-09-05T12:27:44.506819+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:27:44.506819+0000","last_undegraded":"2024-09-05T12:27:44.506819+0000","last_fullsized":"2024-09-05T12:27:44.506819+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507160+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:27:44.507160+0000","last_peered":"2024-09-05T12:27:44.507160+0000","last_clean":"2024-09-05T12:27:44.507160+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:27:44.507160+0000","last_undegraded":"2024-09-05T12:27:44.507160+0000","last_fullsized":"2024-09-05T12:27:44.507160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'9","reported_seq":181,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.507404+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:27:45.507404+0000","last_peered":"2024-09-05T12:27:45.507404+0000","last_clean":"2024-09-05T12:27:45.507404+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:27:45.507404+0000","last_undegraded":"2024-09-05T12:27:45.507404+0000","last_fullsized":"2024-09-05T12:27:45.507404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":5,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713421+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:27:44.713421+0000","last_peered":"2024-09-05T12:27:44.713421+0000","last_clean":"2024-09-05T12:27:44.713421+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:27:44.713421+0000","last_undegraded":"2024-09-05T12:27:44.713421+0000","last_fullsized":"2024-09-05T12:27:44.713421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506967+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:27:44.506967+0000","last_peered":"2024-09-05T12:27:44.506967+0000","last_clean":"2024-09-05T12:27:44.506967+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:27:44.506967+0000","last_undegraded":"2024-09-05T12:27:44.506967+0000","last_fullsized":"2024-09-05T12:27:44.506967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'41","reported_seq":90,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.800562+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:27:44.800562+0000","last_peered":"2024-09-05T12:27:44.800562+0000","last_clean":"2024-09-05T12:27:44.800562+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:27:44.800562+0000","last_undegraded":"2024-09-05T12:27:44.800562+0000","last_fullsized":"2024-09-05T12:27:44.800562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887282+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:27:44.887282+0000","last_peered":"2024-09-05T12:27:44.887282+0000","last_clean":"2024-09-05T12:27:44.887282+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:27:44.887282+0000","last_undegraded":"2024-09-05T12:27:44.887282+0000","last_fullsized":"2024-09-05T12:27:44.887282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507354+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:27:44.507354+0000","last_peered":"2024-09-05T12:27:44.507354+0000","last_clean":"2024-09-05T12:27:44.507354+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:27:44.507354+0000","last_undegraded":"2024-09-05T12:27:44.507354+0000","last_fullsized":"2024-09-05T12:27:44.507354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517307+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:27:44.517307+0000","last_peered":"2024-09-05T12:27:44.517307+0000","last_clean":"2024-09-05T12:27:44.517307+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:27:44.517307+0000","last_undegraded":"2024-09-05T12:27:44.517307+0000","last_fullsized":"2024-09-05T12:27:44.517307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519065+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:27:44.519065+0000","last_peered":"2024-09-05T12:27:44.519065+0000","last_clean":"2024-09-05T12:27:44.519065+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:27:44.519065+0000","last_undegraded":"2024-09-05T12:27:44.519065+0000","last_fullsized":"2024-09-05T12:27:44.519065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727189+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:27:44.727189+0000","last_peered":"2024-09-05T12:27:44.727189+0000","last_clean":"2024-09-05T12:27:44.727189+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:27:44.727189+0000","last_undegraded":"2024-09-05T12:27:44.727189+0000","last_fullsized":"2024-09-05T12:27:44.727189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727211+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:27:44.727211+0000","last_peered":"2024-09-05T12:27:44.727211+0000","last_clean":"2024-09-05T12:27:44.727211+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:27:44.727211+0000","last_undegraded":"2024-09-05T12:27:44.727211+0000","last_fullsized":"2024-09-05T12:27:44.727211+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519342+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:27:44.519342+0000","last_peered":"2024-09-05T12:27:44.519342+0000","last_clean":"2024-09-05T12:27:44.519342+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:27:44.519342+0000","last_undegraded":"2024-09-05T12:27:44.519342+0000","last_fullsized":"2024-09-05T12:27:44.519342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519536+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:27:44.519536+0000","last_peered":"2024-09-05T12:27:44.519536+0000","last_clean":"2024-09-05T12:27:44.519536+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:27:44.519536+0000","last_undegraded":"2024-09-05T12:27:44.519536+0000","last_fullsized":"2024-09-05T12:27:44.519536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509035+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:27:44.509035+0000","last_peered":"2024-09-05T12:27:44.509035+0000","last_clean":"2024-09-05T12:27:44.509035+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:27:44.509035+0000","last_undegraded":"2024-09-05T12:27:44.509035+0000","last_fullsized":"2024-09-05T12:27:44.509035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517190+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:27:44.517190+0000","last_peered":"2024-09-05T12:27:44.517190+0000","last_clean":"2024-09-05T12:27:44.517190+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:27:44.517190+0000","last_undegraded":"2024-09-05T12:27:44.517190+0000","last_fullsized":"2024-09-05T12:27:44.517190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506567+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:27:44.506567+0000","last_peered":"2024-09-05T12:27:44.506567+0000","last_clean":"2024-09-05T12:27:44.506567+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:27:44.506567+0000","last_undegraded":"2024-09-05T12:27:44.506567+0000","last_fullsized":"2024-09-05T12:27:44.506567+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'6","reported_seq":176,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.523301+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:27:45.523301+0000","last_peered":"2024-09-05T12:27:45.523301+0000","last_clean":"2024-09-05T12:27:45.523301+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:27:45.523301+0000","last_undegraded":"2024-09-05T12:27:45.523301+0000","last_fullsized":"2024-09-05T12:27:45.523301+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523947+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:27:44.523947+0000","last_peered":"2024-09-05T12:27:44.523947+0000","last_clean":"2024-09-05T12:27:44.523947+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:27:44.523947+0000","last_undegraded":"2024-09-05T12:27:44.523947+0000","last_fullsized":"2024-09-05T12:27:44.523947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506890+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:27:44.506890+0000","last_peered":"2024-09-05T12:27:44.506890+0000","last_clean":"2024-09-05T12:27:44.506890+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:27:44.506890+0000","last_undegraded":"2024-09-05T12:27:44.506890+0000","last_fullsized":"2024-09-05T12:27:44.506890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519489+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:27:44.519489+0000","last_peered":"2024-09-05T12:27:44.519489+0000","last_clean":"2024-09-05T12:27:44.519489+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:27:44.519489+0000","last_undegraded":"2024-09-05T12:27:44.519489+0000","last_fullsized":"2024-09-05T12:27:44.519489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726953+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:27:44.726953+0000","last_peered":"2024-09-05T12:27:44.726953+0000","last_clean":"2024-09-05T12:27:44.726953+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:27:44.726953+0000","last_undegraded":"2024-09-05T12:27:44.726953+0000","last_fullsized":"2024-09-05T12:27:44.726953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517295+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:27:44.517295+0000","last_peered":"2024-09-05T12:27:44.517295+0000","last_clean":"2024-09-05T12:27:44.517295+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:27:44.517295+0000","last_undegraded":"2024-09-05T12:27:44.517295+0000","last_fullsized":"2024-09-05T12:27:44.517295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553650+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:27:44.553650+0000","last_peered":"2024-09-05T12:27:44.553650+0000","last_clean":"2024-09-05T12:27:44.553650+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:27:44.553650+0000","last_undegraded":"2024-09-05T12:27:44.553650+0000","last_fullsized":"2024-09-05T12:27:44.553650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520009+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:27:44.520009+0000","last_peered":"2024-09-05T12:27:44.520009+0000","last_clean":"2024-09-05T12:27:44.520009+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:27:44.520009+0000","last_undegraded":"2024-09-05T12:27:44.520009+0000","last_fullsized":"2024-09-05T12:27:44.520009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509567+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:27:44.509567+0000","last_peered":"2024-09-05T12:27:44.509567+0000","last_clean":"2024-09-05T12:27:44.509567+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:27:44.509567+0000","last_undegraded":"2024-09-05T12:27:44.509567+0000","last_fullsized":"2024-09-05T12:27:44.509567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":347,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.507482+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:27:45.507482+0000","last_peered":"2024-09-05T12:27:45.507482+0000","last_clean":"2024-09-05T12:27:45.507482+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:27:45.507482+0000","last_undegraded":"2024-09-05T12:27:45.507482+0000","last_fullsized":"2024-09-05T12:27:45.507482+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68198,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":266,"num_read_kb":200,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522273+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:27:44.522273+0000","last_peered":"2024-09-05T12:27:44.522273+0000","last_clean":"2024-09-05T12:27:44.522273+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:27:44.522273+0000","last_undegraded":"2024-09-05T12:27:44.522273+0000","last_fullsized":"2024-09-05T12:27:44.522273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712959+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:27:44.712959+0000","last_peered":"2024-09-05T12:27:44.712959+0000","last_clean":"2024-09-05T12:27:44.712959+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:27:44.712959+0000","last_undegraded":"2024-09-05T12:27:44.712959+0000","last_fullsized":"2024-09-05T12:27:44.712959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520278+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:27:44.520278+0000","last_peered":"2024-09-05T12:27:44.520278+0000","last_clean":"2024-09-05T12:27:44.520278+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:27:44.520278+0000","last_undegraded":"2024-09-05T12:27:44.520278+0000","last_fullsized":"2024-09-05T12:27:44.520278+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522012+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:27:44.522012+0000","last_peered":"2024-09-05T12:27:44.522012+0000","last_clean":"2024-09-05T12:27:44.522012+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:27:44.522012+0000","last_undegraded":"2024-09-05T12:27:44.522012+0000","last_fullsized":"2024-09-05T12:27:44.522012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884499+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:27:44.884499+0000","last_peered":"2024-09-05T12:27:44.884499+0000","last_clean":"2024-09-05T12:27:44.884499+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:27:44.884499+0000","last_undegraded":"2024-09-05T12:27:44.884499+0000","last_fullsized":"2024-09-05T12:27:44.884499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507322+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:27:44.507322+0000","last_peered":"2024-09-05T12:27:44.507322+0000","last_clean":"2024-09-05T12:27:44.507322+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:27:44.507322+0000","last_undegraded":"2024-09-05T12:27:44.507322+0000","last_fullsized":"2024-09-05T12:27:44.507322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.508927+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:27:44.508927+0000","last_peered":"2024-09-05T12:27:44.508927+0000","last_clean":"2024-09-05T12:27:44.508927+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:27:44.508927+0000","last_undegraded":"2024-09-05T12:27:44.508927+0000","last_fullsized":"2024-09-05T12:27:44.508927+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712976+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:27:44.712976+0000","last_peered":"2024-09-05T12:27:44.712976+0000","last_clean":"2024-09-05T12:27:44.712976+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:27:44.712976+0000","last_undegraded":"2024-09-05T12:27:44.712976+0000","last_fullsized":"2024-09-05T12:27:44.712976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'7","reported_seq":301,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.506888+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:27:45.506888+0000","last_peered":"2024-09-05T12:27:45.506888+0000","last_clean":"2024-09-05T12:27:45.506888+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:27:45.506888+0000","last_undegraded":"2024-09-05T12:27:45.506888+0000","last_fullsized":"2024-09-05T12:27:45.506888+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":254,"num_read_kb":254,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505528+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:27:44.505528+0000","last_peered":"2024-09-05T12:27:44.505528+0000","last_clean":"2024-09-05T12:27:44.505528+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:27:44.505528+0000","last_undegraded":"2024-09-05T12:27:44.505528+0000","last_fullsized":"2024-09-05T12:27:44.505528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887297+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:27:44.887297+0000","last_peered":"2024-09-05T12:27:44.887297+0000","last_clean":"2024-09-05T12:27:44.887297+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:27:44.887297+0000","last_undegraded":"2024-09-05T12:27:44.887297+0000","last_fullsized":"2024-09-05T12:27:44.887297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522239+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:27:44.522239+0000","last_peered":"2024-09-05T12:27:44.522239+0000","last_clean":"2024-09-05T12:27:44.522239+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:27:44.522239+0000","last_undegraded":"2024-09-05T12:27:44.522239+0000","last_fullsized":"2024-09-05T12:27:44.522239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519527+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:27:44.519527+0000","last_peered":"2024-09-05T12:27:44.519527+0000","last_clean":"2024-09-05T12:27:44.519527+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:27:44.519527+0000","last_undegraded":"2024-09-05T12:27:44.519527+0000","last_fullsized":"2024-09-05T12:27:44.519527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727158+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:27:44.727158+0000","last_peered":"2024-09-05T12:27:44.727158+0000","last_clean":"2024-09-05T12:27:44.727158+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:27:44.727158+0000","last_undegraded":"2024-09-05T12:27:44.727158+0000","last_fullsized":"2024-09-05T12:27:44.727158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505490+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:27:44.505490+0000","last_peered":"2024-09-05T12:27:44.505490+0000","last_clean":"2024-09-05T12:27:44.505490+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:27:44.505490+0000","last_undegraded":"2024-09-05T12:27:44.505490+0000","last_fullsized":"2024-09-05T12:27:44.505490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516334+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:27:44.516334+0000","last_peered":"2024-09-05T12:27:44.516334+0000","last_clean":"2024-09-05T12:27:44.516334+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:27:44.516334+0000","last_undegraded":"2024-09-05T12:27:44.516334+0000","last_fullsized":"2024-09-05T12:27:44.516334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515852+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:27:44.515852+0000","last_peered":"2024-09-05T12:27:44.515852+0000","last_clean":"2024-09-05T12:27:44.515852+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:27:44.515852+0000","last_undegraded":"2024-09-05T12:27:44.515852+0000","last_fullsized":"2024-09-05T12:27:44.515852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.518959+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:27:44.518959+0000","last_peered":"2024-09-05T12:27:44.518959+0000","last_clean":"2024-09-05T12:27:44.518959+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:27:44.518959+0000","last_undegraded":"2024-09-05T12:27:44.518959+0000","last_fullsized":"2024-09-05T12:27:44.518959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553680+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:27:44.553680+0000","last_peered":"2024-09-05T12:27:44.553680+0000","last_clean":"2024-09-05T12:27:44.553680+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:27:44.553680+0000","last_undegraded":"2024-09-05T12:27:44.553680+0000","last_fullsized":"2024-09-05T12:27:44.553680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'7","reported_seq":179,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.521635+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:27:45.521635+0000","last_peered":"2024-09-05T12:27:45.521635+0000","last_clean":"2024-09-05T12:27:45.521635+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:27:45.521635+0000","last_undegraded":"2024-09-05T12:27:45.521635+0000","last_fullsized":"2024-09-05T12:27:45.521635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":131,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509141+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:27:44.509141+0000","last_peered":"2024-09-05T12:27:44.509141+0000","last_clean":"2024-09-05T12:27:44.509141+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:27:44.509141+0000","last_undegraded":"2024-09-05T12:27:44.509141+0000","last_fullsized":"2024-09-05T12:27:44.509141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553781+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:27:44.553781+0000","last_peered":"2024-09-05T12:27:44.553781+0000","last_clean":"2024-09-05T12:27:44.553781+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:27:44.553781+0000","last_undegraded":"2024-09-05T12:27:44.553781+0000","last_fullsized":"2024-09-05T12:27:44.553781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726440+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:27:44.726440+0000","last_peered":"2024-09-05T12:27:44.726440+0000","last_clean":"2024-09-05T12:27:44.726440+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:27:44.726440+0000","last_undegraded":"2024-09-05T12:27:44.726440+0000","last_fullsized":"2024-09-05T12:27:44.726440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553578+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:27:44.553578+0000","last_peered":"2024-09-05T12:27:44.553578+0000","last_clean":"2024-09-05T12:27:44.553578+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:27:44.553578+0000","last_undegraded":"2024-09-05T12:27:44.553578+0000","last_fullsized":"2024-09-05T12:27:44.553578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553078+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:27:44.553078+0000","last_peered":"2024-09-05T12:27:44.553078+0000","last_clean":"2024-09-05T12:27:44.553078+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:27:44.553078+0000","last_undegraded":"2024-09-05T12:27:44.553078+0000","last_fullsized":"2024-09-05T12:27:44.553078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887351+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:27:44.887351+0000","last_peered":"2024-09-05T12:27:44.887351+0000","last_clean":"2024-09-05T12:27:44.887351+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:27:44.887351+0000","last_undegraded":"2024-09-05T12:27:44.887351+0000","last_fullsized":"2024-09-05T12:27:44.887351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517395+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:27:44.517395+0000","last_peered":"2024-09-05T12:27:44.517395+0000","last_clean":"2024-09-05T12:27:44.517395+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:27:44.517395+0000","last_undegraded":"2024-09-05T12:27:44.517395+0000","last_fullsized":"2024-09-05T12:27:44.517395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726513+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:27:44.726513+0000","last_peered":"2024-09-05T12:27:44.726513+0000","last_clean":"2024-09-05T12:27:44.726513+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:27:44.726513+0000","last_undegraded":"2024-09-05T12:27:44.726513+0000","last_fullsized":"2024-09-05T12:27:44.726513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553154+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:27:44.553154+0000","last_peered":"2024-09-05T12:27:44.553154+0000","last_clean":"2024-09-05T12:27:44.553154+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:27:44.553154+0000","last_undegraded":"2024-09-05T12:27:44.553154+0000","last_fullsized":"2024-09-05T12:27:44.553154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506190+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:27:44.506190+0000","last_peered":"2024-09-05T12:27:44.506190+0000","last_clean":"2024-09-05T12:27:44.506190+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:27:44.506190+0000","last_undegraded":"2024-09-05T12:27:44.506190+0000","last_fullsized":"2024-09-05T12:27:44.506190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519213+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:27:44.519213+0000","last_peered":"2024-09-05T12:27:44.519213+0000","last_clean":"2024-09-05T12:27:44.519213+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:27:44.519213+0000","last_undegraded":"2024-09-05T12:27:44.519213+0000","last_fullsized":"2024-09-05T12:27:44.519213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505694+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:27:44.505694+0000","last_peered":"2024-09-05T12:27:44.505694+0000","last_clean":"2024-09-05T12:27:44.505694+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:27:44.505694+0000","last_undegraded":"2024-09-05T12:27:44.505694+0000","last_fullsized":"2024-09-05T12:27:44.505694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712927+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:27:44.712927+0000","last_peered":"2024-09-05T12:27:44.712927+0000","last_clean":"2024-09-05T12:27:44.712927+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:27:44.712927+0000","last_undegraded":"2024-09-05T12:27:44.712927+0000","last_fullsized":"2024-09-05T12:27:44.712927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516572+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:27:44.516572+0000","last_peered":"2024-09-05T12:27:44.516572+0000","last_clean":"2024-09-05T12:27:44.516572+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:27:44.516572+0000","last_undegraded":"2024-09-05T12:27:44.516572+0000","last_fullsized":"2024-09-05T12:27:44.516572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726524+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:27:44.726524+0000","last_peered":"2024-09-05T12:27:44.726524+0000","last_clean":"2024-09-05T12:27:44.726524+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:27:44.726524+0000","last_undegraded":"2024-09-05T12:27:44.726524+0000","last_fullsized":"2024-09-05T12:27:44.726524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553110+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:27:44.553110+0000","last_peered":"2024-09-05T12:27:44.553110+0000","last_clean":"2024-09-05T12:27:44.553110+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:27:44.553110+0000","last_undegraded":"2024-09-05T12:27:44.553110+0000","last_fullsized":"2024-09-05T12:27:44.553110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523087+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:27:44.523087+0000","last_peered":"2024-09-05T12:27:44.523087+0000","last_clean":"2024-09-05T12:27:44.523087+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:27:44.523087+0000","last_undegraded":"2024-09-05T12:27:44.523087+0000","last_fullsized":"2024-09-05T12:27:44.523087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727310+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:27:44.727310+0000","last_peered":"2024-09-05T12:27:44.727310+0000","last_clean":"2024-09-05T12:27:44.727310+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:27:44.727310+0000","last_undegraded":"2024-09-05T12:27:44.727310+0000","last_fullsized":"2024-09-05T12:27:44.727310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712915+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:27:44.712915+0000","last_peered":"2024-09-05T12:27:44.712915+0000","last_clean":"2024-09-05T12:27:44.712915+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:27:44.712915+0000","last_undegraded":"2024-09-05T12:27:44.712915+0000","last_fullsized":"2024-09-05T12:27:44.712915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884369+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:27:44.884369+0000","last_peered":"2024-09-05T12:27:44.884369+0000","last_clean":"2024-09-05T12:27:44.884369+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:27:44.884369+0000","last_undegraded":"2024-09-05T12:27:44.884369+0000","last_fullsized":"2024-09-05T12:27:44.884369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884201+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:27:44.884201+0000","last_peered":"2024-09-05T12:27:44.884201+0000","last_clean":"2024-09-05T12:27:44.884201+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:27:44.884201+0000","last_undegraded":"2024-09-05T12:27:44.884201+0000","last_fullsized":"2024-09-05T12:27:44.884201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516382+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:27:44.516382+0000","last_peered":"2024-09-05T12:27:44.516382+0000","last_clean":"2024-09-05T12:27:44.516382+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:27:44.516382+0000","last_undegraded":"2024-09-05T12:27:44.516382+0000","last_fullsized":"2024-09-05T12:27:44.516382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520915+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:27:44.520915+0000","last_peered":"2024-09-05T12:27:44.520915+0000","last_clean":"2024-09-05T12:27:44.520915+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:27:44.520915+0000","last_undegraded":"2024-09-05T12:27:44.520915+0000","last_fullsized":"2024-09-05T12:27:44.520915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507111+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:27:44.507111+0000","last_peered":"2024-09-05T12:27:44.507111+0000","last_clean":"2024-09-05T12:27:44.507111+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:27:44.507111+0000","last_undegraded":"2024-09-05T12:27:44.507111+0000","last_fullsized":"2024-09-05T12:27:44.507111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516166+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:27:44.516166+0000","last_peered":"2024-09-05T12:27:44.516166+0000","last_clean":"2024-09-05T12:27:44.516166+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:27:44.516166+0000","last_undegraded":"2024-09-05T12:27:44.516166+0000","last_fullsized":"2024-09-05T12:27:44.516166+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713259+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:27:44.713259+0000","last_peered":"2024-09-05T12:27:44.713259+0000","last_clean":"2024-09-05T12:27:44.713259+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:27:44.713259+0000","last_undegraded":"2024-09-05T12:27:44.713259+0000","last_fullsized":"2024-09-05T12:27:44.713259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505460+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:27:44.505460+0000","last_peered":"2024-09-05T12:27:44.505460+0000","last_clean":"2024-09-05T12:27:44.505460+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:27:44.505460+0000","last_undegraded":"2024-09-05T12:27:44.505460+0000","last_fullsized":"2024-09-05T12:27:44.505460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506619+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:27:44.506619+0000","last_peered":"2024-09-05T12:27:44.506619+0000","last_clean":"2024-09-05T12:27:44.506619+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:27:44.506619+0000","last_undegraded":"2024-09-05T12:27:44.506619+0000","last_fullsized":"2024-09-05T12:27:44.506619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515791+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:27:44.515791+0000","last_peered":"2024-09-05T12:27:44.515791+0000","last_clean":"2024-09-05T12:27:44.515791+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:27:44.515791+0000","last_undegraded":"2024-09-05T12:27:44.515791+0000","last_fullsized":"2024-09-05T12:27:44.515791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509128+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:27:44.509128+0000","last_peered":"2024-09-05T12:27:44.509128+0000","last_clean":"2024-09-05T12:27:44.509128+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:27:44.509128+0000","last_undegraded":"2024-09-05T12:27:44.509128+0000","last_fullsized":"2024-09-05T12:27:44.509128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521146+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:27:44.521146+0000","last_peered":"2024-09-05T12:27:44.521146+0000","last_clean":"2024-09-05T12:27:44.521146+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:27:44.521146+0000","last_undegraded":"2024-09-05T12:27:44.521146+0000","last_fullsized":"2024-09-05T12:27:44.521146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.885260+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:27:44.885260+0000","last_peered":"2024-09-05T12:27:44.885260+0000","last_clean":"2024-09-05T12:27:44.885260+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:27:44.885260+0000","last_undegraded":"2024-09-05T12:27:44.885260+0000","last_fullsized":"2024-09-05T12:27:44.885260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887150+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:27:44.887150+0000","last_peered":"2024-09-05T12:27:44.887150+0000","last_clean":"2024-09-05T12:27:44.887150+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:27:44.887150+0000","last_undegraded":"2024-09-05T12:27:44.887150+0000","last_fullsized":"2024-09-05T12:27:44.887150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727308+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:27:44.727308+0000","last_peered":"2024-09-05T12:27:44.727308+0000","last_clean":"2024-09-05T12:27:44.727308+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:27:44.727308+0000","last_undegraded":"2024-09-05T12:27:44.727308+0000","last_fullsized":"2024-09-05T12:27:44.727308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":602246,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1411,"num_read_kb":824,"num_write":383,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1806738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":307,"ondisk_log_size":307,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449137,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":534079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678069,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28976,"kb_used_data":1892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716176,"statfs":{"total":95995035648,"available":95965364224,"internally_reserved":0,"allocated":1937408,"data_stored":1334434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841594,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32668,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712484,"statfs":{"total":95995035648,"available":95961583616,"internally_reserved":0,"allocated":1515520,"data_stored":936477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103232,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36820,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708332,"statfs":{"total":95995035648,"available":95957331968,"internally_reserved":0,"allocated":1507328,"data_stored":936833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1716224,"data_stored":1141235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528394,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":539158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29020,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716132,"statfs":{"total":95995035648,"available":95965319168,"internally_reserved":0,"allocated":1859584,"data_stored":1267238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953557,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":543232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1638400,"data_stored":1063289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1171456,"data_stored":594543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574950,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1150976,"data_stored":593478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":541541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":201927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:27:52.076 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-05T12:27:52.076 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph pg dump --format=json 2024-09-05T12:27:52.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:27:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:27:52.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:27:52.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:27:52.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.445 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: Updating smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/config/ceph.client.admin.keyring 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:27:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:27:52.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:27:52.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:27:53.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 -- 172.21.15.71:0/915196229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 msgr2=0x7f21a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:53.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 --2- 172.21.15.71:0/915196229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a410ea90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f219c009ca0 tx=0x7f219c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 -- 172.21.15.71:0/915196229 shutdown_connections 2024-09-05T12:27:53.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 --2- 172.21.15.71:0/915196229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a410ea90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 --2- 172.21.15.71:0/915196229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4102790 0x7f21a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 --2- 172.21.15.71:0/915196229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a4101de0 0x7f21a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.011+0000 7f21ab200640 1 -- 172.21.15.71:0/915196229 >> 172.21.15.71:0/915196229 conn(0x7f21a40fd8c0 msgr2=0x7f21a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.012+0000 7f21ab200640 1 -- 172.21.15.71:0/915196229 shutdown_connections 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.012+0000 7f21ab200640 1 -- 172.21.15.71:0/915196229 wait complete. 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.012+0000 7f21ab200640 1 Processor -- start 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 -- start start 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4101de0 0x7f21a419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a4102790 0x7f21a419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f21a4111f70 con 0x7f21a4106f10 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f21a4111df0 con 0x7f21a4101de0 2024-09-05T12:27:53.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21ab200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41120f0 con 0x7f21a4102790 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21a9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4101de0 0x7f21a419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21a9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a419eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41774/0 (socket says 172.21.15.71:41774) 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21a9200640 1 -- 172.21.15.71:0/1995379942 learned_addr learned my addr 172.21.15.71:0/1995379942 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.013+0000 7f21a3400640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a4102790 0x7f21a419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a3e00640 1 -- 172.21.15.71:0/1995379942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a4102790 msgr2=0x7f21a419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a3e00640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a4102790 0x7f21a419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a3e00640 1 -- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 msgr2=0x7f21a419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a3e00640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a3e00640 1 -- 172.21.15.71:0/1995379942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a419f5a0 con 0x7f21a4101de0 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a3e00640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4101de0 0x7f21a419a5f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f219400e9f0 tx=0x7f219400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219400ccc0 con 0x7f21a4101de0 2024-09-05T12:27:53.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a419f890 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2194005020 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940049e0 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.014+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a41a7170 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.015+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a41021c0 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.018+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f2194005560 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.019+0000 7f21a0c00640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f21880a2d80 0x7f21880a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.019+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f21940cf9a0 con 0x7f21a4101de0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.019+0000 7f21a3400640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f21880a2d80 0x7f21880a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.020+0000 7f21a3400640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f21880a2d80 0x7f21880a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f21a4076480 tx=0x7f2190005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:53.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.021+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f21940051d0 con 0x7f21a4101de0 2024-09-05T12:27:53.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.195+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21a410de00 con 0x7f21880a2d80 2024-09-05T12:27:53.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.204+0000 7f21a0c00640 1 -- 172.21.15.71:0/1995379942 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339461 (secure 0 0 0) 0x7f21a410de00 con 0x7f21880a2d80 2024-09-05T12:27:53.210 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:53.212 INFO:teuthology.orchestra.run.smithi071.stderr:dumped all 2024-09-05T12:27:53.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f21880a2d80 msgr2=0x7f21880a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:53.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f21880a2d80 0x7f21880a5240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f21a4076480 tx=0x7f2190005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:53.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4101de0 msgr2=0x7f21a419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:53.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4101de0 0x7f21a419a5f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f219400e9f0 tx=0x7f219400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 shutdown_connections 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f21880a2d80 0x7f21880a5240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f21a4106f10 0x7f21a419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a4102790 0x7f21a419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 --2- 172.21.15.71:0/1995379942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21a4101de0 0x7f21a419a5f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 >> 172.21.15.71:0/1995379942 conn(0x7f21a40fd8c0 msgr2=0x7f21a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.210+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 shutdown_connections 2024-09-05T12:27:53.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:53.211+0000 7f21ab200640 1 -- 172.21.15.71:0/1995379942 wait complete. 2024-09-05T12:27:54.105 INFO:teuthology.orchestra.run.smithi071.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-09-05T12:27:52.525915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389480,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1950,"num_read_kb":1409,"num_write":712,"num_write_kb":5327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1036,"ondisk_log_size":1036,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379136,"kb_used_data":17144,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124562688,"statfs":{"total":1151940427776,"available":1151552192512,"internally_reserved":0,"allocated":17555456,"data_stored":10657022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":196632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":403,"num_read_kb":466,"num_write":146,"num_write_kb":914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.985436"},"pg_stats":[{"pgid":"2.1f","version":"95'14","reported_seq":187,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:49.532254+0000","last_change":"2024-09-05T12:24:46.036135+0000","last_active":"2024-09-05T12:27:49.532254+0000","last_peered":"2024-09-05T12:27:49.532254+0000","last_clean":"2024-09-05T12:27:49.532254+0000","last_became_active":"2024-09-05T12:24:46.035705+0000","last_became_peered":"2024-09-05T12:24:46.035705+0000","last_unstale":"2024-09-05T12:27:49.532254+0000","last_undegraded":"2024-09-05T12:27:49.532254+0000","last_fullsized":"2024-09-05T12:27:49.532254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:26:23.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506128+0000","last_change":"2024-09-05T12:24:48.512635+0000","last_active":"2024-09-05T12:27:44.506128+0000","last_peered":"2024-09-05T12:27:44.506128+0000","last_clean":"2024-09-05T12:27:44.506128+0000","last_became_active":"2024-09-05T12:24:48.512308+0000","last_became_peered":"2024-09-05T12:24:48.512308+0000","last_unstale":"2024-09-05T12:27:44.506128+0000","last_undegraded":"2024-09-05T12:27:44.506128+0000","last_fullsized":"2024-09-05T12:27:44.506128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:39:38.812164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553411+0000","last_change":"2024-09-05T12:24:58.573620+0000","last_active":"2024-09-05T12:27:44.553411+0000","last_peered":"2024-09-05T12:27:44.553411+0000","last_clean":"2024-09-05T12:27:44.553411+0000","last_became_active":"2024-09-05T12:24:58.573203+0000","last_became_peered":"2024-09-05T12:24:58.573203+0000","last_unstale":"2024-09-05T12:27:44.553411+0000","last_undegraded":"2024-09-05T12:27:44.553411+0000","last_fullsized":"2024-09-05T12:27:44.553411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:37:16.662134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506439+0000","last_change":"2024-09-05T12:24:46.065294+0000","last_active":"2024-09-05T12:27:44.506439+0000","last_peered":"2024-09-05T12:27:44.506439+0000","last_clean":"2024-09-05T12:27:44.506439+0000","last_became_active":"2024-09-05T12:24:46.064932+0000","last_became_peered":"2024-09-05T12:24:46.064932+0000","last_unstale":"2024-09-05T12:27:44.506439+0000","last_undegraded":"2024-09-05T12:27:44.506439+0000","last_fullsized":"2024-09-05T12:27:44.506439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:48:08.164258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884712+0000","last_change":"2024-09-05T12:24:48.536448+0000","last_active":"2024-09-05T12:27:44.884712+0000","last_peered":"2024-09-05T12:27:44.884712+0000","last_clean":"2024-09-05T12:27:44.884712+0000","last_became_active":"2024-09-05T12:24:48.535899+0000","last_became_peered":"2024-09-05T12:24:48.535899+0000","last_unstale":"2024-09-05T12:27:44.884712+0000","last_undegraded":"2024-09-05T12:27:44.884712+0000","last_fullsized":"2024-09-05T12:27:44.884712+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:48:13.018902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521451+0000","last_change":"2024-09-05T12:24:58.583213+0000","last_active":"2024-09-05T12:27:44.521451+0000","last_peered":"2024-09-05T12:27:44.521451+0000","last_clean":"2024-09-05T12:27:44.521451+0000","last_became_active":"2024-09-05T12:24:58.582817+0000","last_became_peered":"2024-09-05T12:24:58.582817+0000","last_unstale":"2024-09-05T12:27:44.521451+0000","last_undegraded":"2024-09-05T12:27:44.521451+0000","last_fullsized":"2024-09-05T12:27:44.521451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:48.930868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"95'13","reported_seq":55,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509779+0000","last_change":"2024-09-05T12:24:46.035640+0000","last_active":"2024-09-05T12:27:44.509779+0000","last_peered":"2024-09-05T12:27:44.509779+0000","last_clean":"2024-09-05T12:27:44.509779+0000","last_became_active":"2024-09-05T12:24:46.035149+0000","last_became_peered":"2024-09-05T12:24:46.035149+0000","last_unstale":"2024-09-05T12:27:44.509779+0000","last_undegraded":"2024-09-05T12:27:44.509779+0000","last_fullsized":"2024-09-05T12:27:44.509779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:09:30.890252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713202+0000","last_change":"2024-09-05T12:24:48.515743+0000","last_active":"2024-09-05T12:27:44.713202+0000","last_peered":"2024-09-05T12:27:44.713202+0000","last_clean":"2024-09-05T12:27:44.713202+0000","last_became_active":"2024-09-05T12:24:48.515371+0000","last_became_peered":"2024-09-05T12:24:48.515371+0000","last_unstale":"2024-09-05T12:27:44.713202+0000","last_undegraded":"2024-09-05T12:27:44.713202+0000","last_fullsized":"2024-09-05T12:27:44.713202+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:24:09.070824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521661+0000","last_change":"2024-09-05T12:24:58.579442+0000","last_active":"2024-09-05T12:27:44.521661+0000","last_peered":"2024-09-05T12:27:44.521661+0000","last_clean":"2024-09-05T12:27:44.521661+0000","last_became_active":"2024-09-05T12:24:58.579046+0000","last_became_peered":"2024-09-05T12:24:58.579046+0000","last_unstale":"2024-09-05T12:27:44.521661+0000","last_undegraded":"2024-09-05T12:27:44.521661+0000","last_fullsized":"2024-09-05T12:27:44.521661+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:41:31.362176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509541+0000","last_change":"2024-09-05T12:24:46.041576+0000","last_active":"2024-09-05T12:27:44.509541+0000","last_peered":"2024-09-05T12:27:44.509541+0000","last_clean":"2024-09-05T12:27:44.509541+0000","last_became_active":"2024-09-05T12:24:46.041236+0000","last_became_peered":"2024-09-05T12:24:46.041236+0000","last_unstale":"2024-09-05T12:27:44.509541+0000","last_undegraded":"2024-09-05T12:27:44.509541+0000","last_fullsized":"2024-09-05T12:27:44.509541+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:41:41.874499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":52,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520014+0000","last_change":"2024-09-05T12:24:48.520426+0000","last_active":"2024-09-05T12:27:44.520014+0000","last_peered":"2024-09-05T12:27:44.520014+0000","last_clean":"2024-09-05T12:27:44.520014+0000","last_became_active":"2024-09-05T12:24:48.519074+0000","last_became_peered":"2024-09-05T12:24:48.519074+0000","last_unstale":"2024-09-05T12:27:44.520014+0000","last_undegraded":"2024-09-05T12:27:44.520014+0000","last_fullsized":"2024-09-05T12:27:44.520014+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:13:02.463881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521095+0000","last_change":"2024-09-05T12:24:58.578986+0000","last_active":"2024-09-05T12:27:44.521095+0000","last_peered":"2024-09-05T12:27:44.521095+0000","last_clean":"2024-09-05T12:27:44.521095+0000","last_became_active":"2024-09-05T12:24:58.578545+0000","last_became_peered":"2024-09-05T12:24:58.578545+0000","last_unstale":"2024-09-05T12:27:44.521095+0000","last_undegraded":"2024-09-05T12:27:44.521095+0000","last_fullsized":"2024-09-05T12:27:44.521095+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:34:06.628403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506770+0000","last_change":"2024-09-05T12:24:48.519042+0000","last_active":"2024-09-05T12:27:44.506770+0000","last_peered":"2024-09-05T12:27:44.506770+0000","last_clean":"2024-09-05T12:27:44.506770+0000","last_became_active":"2024-09-05T12:24:48.518588+0000","last_became_peered":"2024-09-05T12:24:48.518588+0000","last_unstale":"2024-09-05T12:27:44.506770+0000","last_undegraded":"2024-09-05T12:27:44.506770+0000","last_fullsized":"2024-09-05T12:27:44.506770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:06:08.587312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509274+0000","last_change":"2024-09-05T12:24:46.038086+0000","last_active":"2024-09-05T12:27:44.509274+0000","last_peered":"2024-09-05T12:27:44.509274+0000","last_clean":"2024-09-05T12:27:44.509274+0000","last_became_active":"2024-09-05T12:24:46.037714+0000","last_became_peered":"2024-09-05T12:24:46.037714+0000","last_unstale":"2024-09-05T12:27:44.509274+0000","last_undegraded":"2024-09-05T12:27:44.509274+0000","last_fullsized":"2024-09-05T12:27:44.509274+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:52:42.550489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521721+0000","last_change":"2024-09-05T12:24:58.722822+0000","last_active":"2024-09-05T12:27:44.521721+0000","last_peered":"2024-09-05T12:27:44.521721+0000","last_clean":"2024-09-05T12:27:44.521721+0000","last_became_active":"2024-09-05T12:24:58.722413+0000","last_became_peered":"2024-09-05T12:24:58.722413+0000","last_unstale":"2024-09-05T12:27:44.521721+0000","last_undegraded":"2024-09-05T12:27:44.521721+0000","last_fullsized":"2024-09-05T12:27:44.521721+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:11:39.034047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505685+0000","last_change":"2024-09-05T12:24:48.528459+0000","last_active":"2024-09-05T12:27:44.505685+0000","last_peered":"2024-09-05T12:27:44.505685+0000","last_clean":"2024-09-05T12:27:44.505685+0000","last_became_active":"2024-09-05T12:24:48.527700+0000","last_became_peered":"2024-09-05T12:24:48.527700+0000","last_unstale":"2024-09-05T12:27:44.505685+0000","last_undegraded":"2024-09-05T12:27:44.505685+0000","last_fullsized":"2024-09-05T12:27:44.505685+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:45:14.794308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726946+0000","last_change":"2024-09-05T12:25:34.268734+0000","last_active":"2024-09-05T12:27:44.726946+0000","last_peered":"2024-09-05T12:27:44.726946+0000","last_clean":"2024-09-05T12:27:44.726946+0000","last_became_active":"2024-09-05T12:25:34.268372+0000","last_became_peered":"2024-09-05T12:25:34.268372+0000","last_unstale":"2024-09-05T12:27:44.726946+0000","last_undegraded":"2024-09-05T12:27:44.726946+0000","last_fullsized":"2024-09-05T12:27:44.726946+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:11:05.401998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521509+0000","last_change":"2024-09-05T12:25:34.270657+0000","last_active":"2024-09-05T12:27:44.521509+0000","last_peered":"2024-09-05T12:27:44.521509+0000","last_clean":"2024-09-05T12:27:44.521509+0000","last_became_active":"2024-09-05T12:25:34.270178+0000","last_became_peered":"2024-09-05T12:25:34.270178+0000","last_unstale":"2024-09-05T12:27:44.521509+0000","last_undegraded":"2024-09-05T12:27:44.521509+0000","last_fullsized":"2024-09-05T12:27:44.521509+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:28:08.711101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726837+0000","last_change":"2024-09-05T12:24:48.523514+0000","last_active":"2024-09-05T12:27:44.726837+0000","last_peered":"2024-09-05T12:27:44.726837+0000","last_clean":"2024-09-05T12:27:44.726837+0000","last_became_active":"2024-09-05T12:24:48.523163+0000","last_became_peered":"2024-09-05T12:24:48.523163+0000","last_unstale":"2024-09-05T12:27:44.726837+0000","last_undegraded":"2024-09-05T12:27:44.726837+0000","last_fullsized":"2024-09-05T12:27:44.726837+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:11:33.470288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505891+0000","last_change":"2024-09-05T12:24:46.035750+0000","last_active":"2024-09-05T12:27:44.505891+0000","last_peered":"2024-09-05T12:27:44.505891+0000","last_clean":"2024-09-05T12:27:44.505891+0000","last_became_active":"2024-09-05T12:24:46.035371+0000","last_became_peered":"2024-09-05T12:24:46.035371+0000","last_unstale":"2024-09-05T12:27:44.505891+0000","last_undegraded":"2024-09-05T12:27:44.505891+0000","last_fullsized":"2024-09-05T12:27:44.505891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:10:01.160492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523836+0000","last_change":"2024-09-05T12:24:58.574166+0000","last_active":"2024-09-05T12:27:44.523836+0000","last_peered":"2024-09-05T12:27:44.523836+0000","last_clean":"2024-09-05T12:27:44.523836+0000","last_became_active":"2024-09-05T12:24:58.573494+0000","last_became_peered":"2024-09-05T12:24:58.573494+0000","last_unstale":"2024-09-05T12:27:44.523836+0000","last_undegraded":"2024-09-05T12:27:44.523836+0000","last_fullsized":"2024-09-05T12:27:44.523836+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:24:45.248036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509025+0000","last_change":"2024-09-05T12:24:48.517954+0000","last_active":"2024-09-05T12:27:44.509025+0000","last_peered":"2024-09-05T12:27:44.509025+0000","last_clean":"2024-09-05T12:27:44.509025+0000","last_became_active":"2024-09-05T12:24:48.517579+0000","last_became_peered":"2024-09-05T12:24:48.517579+0000","last_unstale":"2024-09-05T12:27:44.509025+0000","last_undegraded":"2024-09-05T12:27:44.509025+0000","last_fullsized":"2024-09-05T12:27:44.509025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:24:46.517667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713256+0000","last_change":"2024-09-05T12:24:46.065567+0000","last_active":"2024-09-05T12:27:44.713256+0000","last_peered":"2024-09-05T12:27:44.713256+0000","last_clean":"2024-09-05T12:27:44.713256+0000","last_became_active":"2024-09-05T12:24:46.065336+0000","last_became_peered":"2024-09-05T12:24:46.065336+0000","last_unstale":"2024-09-05T12:27:44.713256+0000","last_undegraded":"2024-09-05T12:27:44.713256+0000","last_fullsized":"2024-09-05T12:27:44.713256+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:00:07.619714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521832+0000","last_change":"2024-09-05T12:24:58.723717+0000","last_active":"2024-09-05T12:27:44.521832+0000","last_peered":"2024-09-05T12:27:44.521832+0000","last_clean":"2024-09-05T12:27:44.521832+0000","last_became_active":"2024-09-05T12:24:58.722418+0000","last_became_peered":"2024-09-05T12:24:58.722418+0000","last_unstale":"2024-09-05T12:27:44.521832+0000","last_undegraded":"2024-09-05T12:27:44.521832+0000","last_fullsized":"2024-09-05T12:27:44.521832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:42:49.346295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884458+0000","last_change":"2024-09-05T12:24:48.534235+0000","last_active":"2024-09-05T12:27:44.884458+0000","last_peered":"2024-09-05T12:27:44.884458+0000","last_clean":"2024-09-05T12:27:44.884458+0000","last_became_active":"2024-09-05T12:24:48.533908+0000","last_became_peered":"2024-09-05T12:24:48.533908+0000","last_unstale":"2024-09-05T12:27:44.884458+0000","last_undegraded":"2024-09-05T12:27:44.884458+0000","last_fullsized":"2024-09-05T12:27:44.884458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:39.221474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521673+0000","last_change":"2024-09-05T12:24:46.034549+0000","last_active":"2024-09-05T12:27:44.521673+0000","last_peered":"2024-09-05T12:27:44.521673+0000","last_clean":"2024-09-05T12:27:44.521673+0000","last_became_active":"2024-09-05T12:24:46.034041+0000","last_became_peered":"2024-09-05T12:24:46.034041+0000","last_unstale":"2024-09-05T12:27:44.521673+0000","last_undegraded":"2024-09-05T12:27:44.521673+0000","last_fullsized":"2024-09-05T12:27:44.521673+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:33:27.731011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553507+0000","last_change":"2024-09-05T12:24:58.566862+0000","last_active":"2024-09-05T12:27:44.553507+0000","last_peered":"2024-09-05T12:27:44.553507+0000","last_clean":"2024-09-05T12:27:44.553507+0000","last_became_active":"2024-09-05T12:24:58.566471+0000","last_became_peered":"2024-09-05T12:24:58.566471+0000","last_unstale":"2024-09-05T12:27:44.553507+0000","last_undegraded":"2024-09-05T12:27:44.553507+0000","last_fullsized":"2024-09-05T12:27:44.553507+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:53:15.680957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507103+0000","last_change":"2024-09-05T12:24:48.515477+0000","last_active":"2024-09-05T12:27:44.507103+0000","last_peered":"2024-09-05T12:27:44.507103+0000","last_clean":"2024-09-05T12:27:44.507103+0000","last_became_active":"2024-09-05T12:24:48.512048+0000","last_became_peered":"2024-09-05T12:24:48.512048+0000","last_unstale":"2024-09-05T12:27:44.507103+0000","last_undegraded":"2024-09-05T12:27:44.507103+0000","last_fullsized":"2024-09-05T12:27:44.507103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:36:49.930207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"95'103","reported_seq":235,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:47.309024+0000","last_change":"2024-09-05T12:24:46.037176+0000","last_active":"2024-09-05T12:27:47.309024+0000","last_peered":"2024-09-05T12:27:47.309024+0000","last_clean":"2024-09-05T12:27:47.309024+0000","last_became_active":"2024-09-05T12:24:46.036852+0000","last_became_peered":"2024-09-05T12:24:46.036852+0000","last_unstale":"2024-09-05T12:27:47.309024+0000","last_undegraded":"2024-09-05T12:27:47.309024+0000","last_fullsized":"2024-09-05T12:27:47.309024+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:20:28.959952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505333+0000","last_change":"2024-09-05T12:24:58.721924+0000","last_active":"2024-09-05T12:27:44.505333+0000","last_peered":"2024-09-05T12:27:44.505333+0000","last_clean":"2024-09-05T12:27:44.505333+0000","last_became_active":"2024-09-05T12:24:58.721511+0000","last_became_peered":"2024-09-05T12:24:58.721511+0000","last_unstale":"2024-09-05T12:27:44.505333+0000","last_undegraded":"2024-09-05T12:27:44.505333+0000","last_fullsized":"2024-09-05T12:27:44.505333+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:27:37.952569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517516+0000","last_change":"2024-09-05T12:24:48.527772+0000","last_active":"2024-09-05T12:27:44.517516+0000","last_peered":"2024-09-05T12:27:44.517516+0000","last_clean":"2024-09-05T12:27:44.517516+0000","last_became_active":"2024-09-05T12:24:48.527427+0000","last_became_peered":"2024-09-05T12:24:48.527427+0000","last_unstale":"2024-09-05T12:27:44.517516+0000","last_undegraded":"2024-09-05T12:27:44.517516+0000","last_fullsized":"2024-09-05T12:27:44.517516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:39:31.066457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509852+0000","last_change":"2024-09-05T12:24:46.066686+0000","last_active":"2024-09-05T12:27:44.509852+0000","last_peered":"2024-09-05T12:27:44.509852+0000","last_clean":"2024-09-05T12:27:44.509852+0000","last_became_active":"2024-09-05T12:24:46.066215+0000","last_became_peered":"2024-09-05T12:24:46.066215+0000","last_unstale":"2024-09-05T12:27:44.509852+0000","last_undegraded":"2024-09-05T12:27:44.509852+0000","last_fullsized":"2024-09-05T12:27:44.509852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:12:14.488601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726921+0000","last_change":"2024-09-05T12:24:58.571944+0000","last_active":"2024-09-05T12:27:44.726921+0000","last_peered":"2024-09-05T12:27:44.726921+0000","last_clean":"2024-09-05T12:27:44.726921+0000","last_became_active":"2024-09-05T12:24:58.571493+0000","last_became_peered":"2024-09-05T12:24:58.571493+0000","last_unstale":"2024-09-05T12:27:44.726921+0000","last_undegraded":"2024-09-05T12:27:44.726921+0000","last_fullsized":"2024-09-05T12:27:44.726921+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:14:11.983971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"94'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515978+0000","last_change":"2024-09-05T12:24:48.532940+0000","last_active":"2024-09-05T12:27:44.515978+0000","last_peered":"2024-09-05T12:27:44.515978+0000","last_clean":"2024-09-05T12:27:44.515978+0000","last_became_active":"2024-09-05T12:24:48.532642+0000","last_became_peered":"2024-09-05T12:24:48.532642+0000","last_unstale":"2024-09-05T12:27:44.515978+0000","last_undegraded":"2024-09-05T12:27:44.515978+0000","last_fullsized":"2024-09-05T12:27:44.515978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:10:41.582750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522745+0000","last_change":"2024-09-05T12:24:46.067532+0000","last_active":"2024-09-05T12:27:44.522745+0000","last_peered":"2024-09-05T12:27:44.522745+0000","last_clean":"2024-09-05T12:27:44.522745+0000","last_became_active":"2024-09-05T12:24:46.066712+0000","last_became_peered":"2024-09-05T12:24:46.066712+0000","last_unstale":"2024-09-05T12:27:44.522745+0000","last_undegraded":"2024-09-05T12:27:44.522745+0000","last_fullsized":"2024-09-05T12:27:44.522745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:04.878997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509015+0000","last_change":"2024-09-05T12:24:58.721694+0000","last_active":"2024-09-05T12:27:44.509015+0000","last_peered":"2024-09-05T12:27:44.509015+0000","last_clean":"2024-09-05T12:27:44.509015+0000","last_became_active":"2024-09-05T12:24:58.721303+0000","last_became_peered":"2024-09-05T12:24:58.721303+0000","last_unstale":"2024-09-05T12:27:44.509015+0000","last_undegraded":"2024-09-05T12:27:44.509015+0000","last_fullsized":"2024-09-05T12:27:44.509015+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:02.003328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516026+0000","last_change":"2024-09-05T12:24:48.526812+0000","last_active":"2024-09-05T12:27:44.516026+0000","last_peered":"2024-09-05T12:27:44.516026+0000","last_clean":"2024-09-05T12:27:44.516026+0000","last_became_active":"2024-09-05T12:24:48.526422+0000","last_became_peered":"2024-09-05T12:24:48.526422+0000","last_unstale":"2024-09-05T12:27:44.516026+0000","last_undegraded":"2024-09-05T12:27:44.516026+0000","last_fullsized":"2024-09-05T12:27:44.516026+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:50:58.952467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"95'6","reported_seq":180,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:49.532626+0000","last_change":"2024-09-05T12:24:46.042036+0000","last_active":"2024-09-05T12:27:49.532626+0000","last_peered":"2024-09-05T12:27:49.532626+0000","last_clean":"2024-09-05T12:27:49.532626+0000","last_became_active":"2024-09-05T12:24:46.041563+0000","last_became_peered":"2024-09-05T12:24:46.041563+0000","last_unstale":"2024-09-05T12:27:49.532626+0000","last_undegraded":"2024-09-05T12:27:49.532626+0000","last_fullsized":"2024-09-05T12:27:49.532626+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:35:34.367792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517225+0000","last_change":"2024-09-05T12:24:58.579395+0000","last_active":"2024-09-05T12:27:44.517225+0000","last_peered":"2024-09-05T12:27:44.517225+0000","last_clean":"2024-09-05T12:27:44.517225+0000","last_became_active":"2024-09-05T12:24:58.575844+0000","last_became_peered":"2024-09-05T12:24:58.575844+0000","last_unstale":"2024-09-05T12:27:44.517225+0000","last_undegraded":"2024-09-05T12:27:44.517225+0000","last_fullsized":"2024-09-05T12:27:44.517225+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:25:25.020155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519362+0000","last_change":"2024-09-05T12:24:48.516929+0000","last_active":"2024-09-05T12:27:44.519362+0000","last_peered":"2024-09-05T12:27:44.519362+0000","last_clean":"2024-09-05T12:27:44.519362+0000","last_became_active":"2024-09-05T12:24:48.516614+0000","last_became_peered":"2024-09-05T12:24:48.516614+0000","last_unstale":"2024-09-05T12:27:44.519362+0000","last_undegraded":"2024-09-05T12:27:44.519362+0000","last_fullsized":"2024-09-05T12:27:44.519362+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:33:31.316991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"95'6","reported_seq":180,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:49.530297+0000","last_change":"2024-09-05T12:24:46.065463+0000","last_active":"2024-09-05T12:27:49.530297+0000","last_peered":"2024-09-05T12:27:49.530297+0000","last_clean":"2024-09-05T12:27:49.530297+0000","last_became_active":"2024-09-05T12:24:46.064587+0000","last_became_peered":"2024-09-05T12:24:46.064587+0000","last_unstale":"2024-09-05T12:27:49.530297+0000","last_undegraded":"2024-09-05T12:27:49.530297+0000","last_fullsized":"2024-09-05T12:27:49.530297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:06:09.628045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":133,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516749+0000","last_change":"2024-09-05T12:25:34.271660+0000","last_active":"2024-09-05T12:27:44.516749+0000","last_peered":"2024-09-05T12:27:44.516749+0000","last_clean":"2024-09-05T12:27:44.516749+0000","last_became_active":"2024-09-05T12:25:34.271095+0000","last_became_peered":"2024-09-05T12:25:34.271095+0000","last_unstale":"2024-09-05T12:27:44.516749+0000","last_undegraded":"2024-09-05T12:27:44.516749+0000","last_fullsized":"2024-09-05T12:27:44.516749+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:16:09.186585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,10,0],"acting":[6,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516182+0000","last_change":"2024-09-05T12:24:48.515438+0000","last_active":"2024-09-05T12:27:44.516182+0000","last_peered":"2024-09-05T12:27:44.516182+0000","last_clean":"2024-09-05T12:27:44.516182+0000","last_became_active":"2024-09-05T12:24:48.515086+0000","last_became_peered":"2024-09-05T12:24:48.515086+0000","last_unstale":"2024-09-05T12:27:44.516182+0000","last_undegraded":"2024-09-05T12:27:44.516182+0000","last_fullsized":"2024-09-05T12:27:44.516182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:01:25.495560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506602+0000","last_change":"2024-09-05T12:24:46.039134+0000","last_active":"2024-09-05T12:27:44.506602+0000","last_peered":"2024-09-05T12:27:44.506602+0000","last_clean":"2024-09-05T12:27:44.506602+0000","last_became_active":"2024-09-05T12:24:46.038772+0000","last_became_peered":"2024-09-05T12:24:46.038772+0000","last_unstale":"2024-09-05T12:27:44.506602+0000","last_undegraded":"2024-09-05T12:27:44.506602+0000","last_fullsized":"2024-09-05T12:27:44.506602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:54:03.752117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520073+0000","last_change":"2024-09-05T12:24:58.722778+0000","last_active":"2024-09-05T12:27:44.520073+0000","last_peered":"2024-09-05T12:27:44.520073+0000","last_clean":"2024-09-05T12:27:44.520073+0000","last_became_active":"2024-09-05T12:24:58.722440+0000","last_became_peered":"2024-09-05T12:24:58.722440+0000","last_unstale":"2024-09-05T12:27:44.520073+0000","last_undegraded":"2024-09-05T12:27:44.520073+0000","last_fullsized":"2024-09-05T12:27:44.520073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:57:40.820609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505827+0000","last_change":"2024-09-05T12:24:48.517831+0000","last_active":"2024-09-05T12:27:44.505827+0000","last_peered":"2024-09-05T12:27:44.505827+0000","last_clean":"2024-09-05T12:27:44.505827+0000","last_became_active":"2024-09-05T12:24:48.517498+0000","last_became_peered":"2024-09-05T12:24:48.517498+0000","last_unstale":"2024-09-05T12:27:44.505827+0000","last_undegraded":"2024-09-05T12:27:44.505827+0000","last_fullsized":"2024-09-05T12:27:44.505827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:39.963511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":49,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522084+0000","last_change":"2024-09-05T12:24:46.034879+0000","last_active":"2024-09-05T12:27:44.522084+0000","last_peered":"2024-09-05T12:27:44.522084+0000","last_clean":"2024-09-05T12:27:44.522084+0000","last_became_active":"2024-09-05T12:24:46.034559+0000","last_became_peered":"2024-09-05T12:24:46.034559+0000","last_unstale":"2024-09-05T12:27:44.522084+0000","last_undegraded":"2024-09-05T12:27:44.522084+0000","last_fullsized":"2024-09-05T12:27:44.522084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:35.116298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":408,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:46.655849+0000","last_change":"2024-09-05T12:23:32.155647+0000","last_active":"2024-09-05T12:27:46.655849+0000","last_peered":"2024-09-05T12:27:46.655849+0000","last_clean":"2024-09-05T12:27:46.655849+0000","last_became_active":"2024-09-05T12:23:32.155126+0000","last_became_peered":"2024-09-05T12:23:32.155126+0000","last_unstale":"2024-09-05T12:27:46.655849+0000","last_undegraded":"2024-09-05T12:27:46.655849+0000","last_fullsized":"2024-09-05T12:27:46.655849+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:19:53.849366+0000","last_clean_scrub_stamp":"2024-09-05T12:19:53.849366+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:54.969386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506819+0000","last_change":"2024-09-05T12:24:58.579844+0000","last_active":"2024-09-05T12:27:44.506819+0000","last_peered":"2024-09-05T12:27:44.506819+0000","last_clean":"2024-09-05T12:27:44.506819+0000","last_became_active":"2024-09-05T12:24:58.579391+0000","last_became_peered":"2024-09-05T12:24:58.579391+0000","last_unstale":"2024-09-05T12:27:44.506819+0000","last_undegraded":"2024-09-05T12:27:44.506819+0000","last_fullsized":"2024-09-05T12:27:44.506819+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:18:49.552585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507160+0000","last_change":"2024-09-05T12:24:48.519368+0000","last_active":"2024-09-05T12:27:44.507160+0000","last_peered":"2024-09-05T12:27:44.507160+0000","last_clean":"2024-09-05T12:27:44.507160+0000","last_became_active":"2024-09-05T12:24:48.518856+0000","last_became_peered":"2024-09-05T12:24:48.518856+0000","last_unstale":"2024-09-05T12:27:44.507160+0000","last_undegraded":"2024-09-05T12:27:44.507160+0000","last_fullsized":"2024-09-05T12:27:44.507160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:41:17.873720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"95'9","reported_seq":186,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:49.513051+0000","last_change":"2024-09-05T12:24:46.065014+0000","last_active":"2024-09-05T12:27:49.513051+0000","last_peered":"2024-09-05T12:27:49.513051+0000","last_clean":"2024-09-05T12:27:49.513051+0000","last_became_active":"2024-09-05T12:24:46.064616+0000","last_became_peered":"2024-09-05T12:24:46.064616+0000","last_unstale":"2024-09-05T12:27:49.513051+0000","last_undegraded":"2024-09-05T12:27:49.513051+0000","last_fullsized":"2024-09-05T12:27:49.513051+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:27:07.363685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":5,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713421+0000","last_change":"2024-09-05T12:24:58.571816+0000","last_active":"2024-09-05T12:27:44.713421+0000","last_peered":"2024-09-05T12:27:44.713421+0000","last_clean":"2024-09-05T12:27:44.713421+0000","last_became_active":"2024-09-05T12:24:58.571383+0000","last_became_peered":"2024-09-05T12:24:58.571383+0000","last_unstale":"2024-09-05T12:27:44.713421+0000","last_undegraded":"2024-09-05T12:27:44.713421+0000","last_fullsized":"2024-09-05T12:27:44.713421+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:21:04.000277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506967+0000","last_change":"2024-09-05T12:24:48.513027+0000","last_active":"2024-09-05T12:27:44.506967+0000","last_peered":"2024-09-05T12:27:44.506967+0000","last_clean":"2024-09-05T12:27:44.506967+0000","last_became_active":"2024-09-05T12:24:48.512684+0000","last_became_peered":"2024-09-05T12:24:48.512684+0000","last_unstale":"2024-09-05T12:27:44.506967+0000","last_undegraded":"2024-09-05T12:27:44.506967+0000","last_fullsized":"2024-09-05T12:27:44.506967+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:54:44.610448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'67","reported_seq":139,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:46.882901+0000","last_change":"2024-09-05T12:24:46.034172+0000","last_active":"2024-09-05T12:27:46.882901+0000","last_peered":"2024-09-05T12:27:46.882901+0000","last_clean":"2024-09-05T12:27:46.882901+0000","last_became_active":"2024-09-05T12:24:46.033783+0000","last_became_peered":"2024-09-05T12:24:46.033783+0000","last_unstale":"2024-09-05T12:27:46.882901+0000","last_undegraded":"2024-09-05T12:27:46.882901+0000","last_fullsized":"2024-09-05T12:27:46.882901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:17:18.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887282+0000","last_change":"2024-09-05T12:24:58.584228+0000","last_active":"2024-09-05T12:27:44.887282+0000","last_peered":"2024-09-05T12:27:44.887282+0000","last_clean":"2024-09-05T12:27:44.887282+0000","last_became_active":"2024-09-05T12:24:58.583854+0000","last_became_peered":"2024-09-05T12:24:58.583854+0000","last_unstale":"2024-09-05T12:27:44.887282+0000","last_undegraded":"2024-09-05T12:27:44.887282+0000","last_fullsized":"2024-09-05T12:27:44.887282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:03:01.183533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507354+0000","last_change":"2024-09-05T12:24:48.511733+0000","last_active":"2024-09-05T12:27:44.507354+0000","last_peered":"2024-09-05T12:27:44.507354+0000","last_clean":"2024-09-05T12:27:44.507354+0000","last_became_active":"2024-09-05T12:24:48.511410+0000","last_became_peered":"2024-09-05T12:24:48.511410+0000","last_unstale":"2024-09-05T12:27:44.507354+0000","last_undegraded":"2024-09-05T12:27:44.507354+0000","last_fullsized":"2024-09-05T12:27:44.507354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:19:35.261456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517307+0000","last_change":"2024-09-05T12:24:46.039341+0000","last_active":"2024-09-05T12:27:44.517307+0000","last_peered":"2024-09-05T12:27:44.517307+0000","last_clean":"2024-09-05T12:27:44.517307+0000","last_became_active":"2024-09-05T12:24:46.038941+0000","last_became_peered":"2024-09-05T12:24:46.038941+0000","last_unstale":"2024-09-05T12:27:44.517307+0000","last_undegraded":"2024-09-05T12:27:44.517307+0000","last_fullsized":"2024-09-05T12:27:44.517307+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:09:01.955339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519065+0000","last_change":"2024-09-05T12:24:58.571376+0000","last_active":"2024-09-05T12:27:44.519065+0000","last_peered":"2024-09-05T12:27:44.519065+0000","last_clean":"2024-09-05T12:27:44.519065+0000","last_became_active":"2024-09-05T12:24:58.571032+0000","last_became_peered":"2024-09-05T12:24:58.571032+0000","last_unstale":"2024-09-05T12:27:44.519065+0000","last_undegraded":"2024-09-05T12:27:44.519065+0000","last_fullsized":"2024-09-05T12:27:44.519065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:05:07.512861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727189+0000","last_change":"2024-09-05T12:24:48.514436+0000","last_active":"2024-09-05T12:27:44.727189+0000","last_peered":"2024-09-05T12:27:44.727189+0000","last_clean":"2024-09-05T12:27:44.727189+0000","last_became_active":"2024-09-05T12:24:48.514090+0000","last_became_peered":"2024-09-05T12:24:48.514090+0000","last_unstale":"2024-09-05T12:27:44.727189+0000","last_undegraded":"2024-09-05T12:27:44.727189+0000","last_fullsized":"2024-09-05T12:27:44.727189+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:37:55.201911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"95'3","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:45.313099+0000","last_change":"2024-09-05T12:25:34.276445+0000","last_active":"2024-09-05T12:27:45.313099+0000","last_peered":"2024-09-05T12:27:45.313099+0000","last_clean":"2024-09-05T12:27:45.313099+0000","last_became_active":"2024-09-05T12:25:34.273228+0000","last_became_peered":"2024-09-05T12:25:34.273228+0000","last_unstale":"2024-09-05T12:27:45.313099+0000","last_undegraded":"2024-09-05T12:27:45.313099+0000","last_fullsized":"2024-09-05T12:27:45.313099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:10:57.243323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519342+0000","last_change":"2024-09-05T12:24:58.577347+0000","last_active":"2024-09-05T12:27:44.519342+0000","last_peered":"2024-09-05T12:27:44.519342+0000","last_clean":"2024-09-05T12:27:44.519342+0000","last_became_active":"2024-09-05T12:24:58.576958+0000","last_became_peered":"2024-09-05T12:24:58.576958+0000","last_unstale":"2024-09-05T12:27:44.519342+0000","last_undegraded":"2024-09-05T12:27:44.519342+0000","last_fullsized":"2024-09-05T12:27:44.519342+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:57.007421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519536+0000","last_change":"2024-09-05T12:24:48.523160+0000","last_active":"2024-09-05T12:27:44.519536+0000","last_peered":"2024-09-05T12:27:44.519536+0000","last_clean":"2024-09-05T12:27:44.519536+0000","last_became_active":"2024-09-05T12:24:48.522812+0000","last_became_peered":"2024-09-05T12:24:48.522812+0000","last_unstale":"2024-09-05T12:27:44.519536+0000","last_undegraded":"2024-09-05T12:27:44.519536+0000","last_fullsized":"2024-09-05T12:27:44.519536+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:55:19.157934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":59,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509035+0000","last_change":"2024-09-05T12:24:46.065663+0000","last_active":"2024-09-05T12:27:44.509035+0000","last_peered":"2024-09-05T12:27:44.509035+0000","last_clean":"2024-09-05T12:27:44.509035+0000","last_became_active":"2024-09-05T12:24:46.064782+0000","last_became_peered":"2024-09-05T12:24:46.064782+0000","last_unstale":"2024-09-05T12:27:44.509035+0000","last_undegraded":"2024-09-05T12:27:44.509035+0000","last_fullsized":"2024-09-05T12:27:44.509035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:13:27.370954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517190+0000","last_change":"2024-09-05T12:24:58.580469+0000","last_active":"2024-09-05T12:27:44.517190+0000","last_peered":"2024-09-05T12:27:44.517190+0000","last_clean":"2024-09-05T12:27:44.517190+0000","last_became_active":"2024-09-05T12:24:58.579722+0000","last_became_peered":"2024-09-05T12:24:58.579722+0000","last_unstale":"2024-09-05T12:27:44.517190+0000","last_undegraded":"2024-09-05T12:27:44.517190+0000","last_fullsized":"2024-09-05T12:27:44.517190+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:47:14.007052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506567+0000","last_change":"2024-09-05T12:25:34.331448+0000","last_active":"2024-09-05T12:27:44.506567+0000","last_peered":"2024-09-05T12:27:44.506567+0000","last_clean":"2024-09-05T12:27:44.506567+0000","last_became_active":"2024-09-05T12:25:34.331035+0000","last_became_peered":"2024-09-05T12:25:34.331035+0000","last_unstale":"2024-09-05T12:27:44.506567+0000","last_undegraded":"2024-09-05T12:27:44.506567+0000","last_fullsized":"2024-09-05T12:27:44.506567+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:25.779407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"95'6","reported_seq":180,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:48.529601+0000","last_change":"2024-09-05T12:24:46.036145+0000","last_active":"2024-09-05T12:27:48.529601+0000","last_peered":"2024-09-05T12:27:48.529601+0000","last_clean":"2024-09-05T12:27:48.529601+0000","last_became_active":"2024-09-05T12:24:46.035763+0000","last_became_peered":"2024-09-05T12:24:46.035763+0000","last_unstale":"2024-09-05T12:27:48.529601+0000","last_undegraded":"2024-09-05T12:27:48.529601+0000","last_fullsized":"2024-09-05T12:27:48.529601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T13:53:11.205222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523947+0000","last_change":"2024-09-05T12:24:58.573853+0000","last_active":"2024-09-05T12:27:44.523947+0000","last_peered":"2024-09-05T12:27:44.523947+0000","last_clean":"2024-09-05T12:27:44.523947+0000","last_became_active":"2024-09-05T12:24:58.572801+0000","last_became_peered":"2024-09-05T12:24:58.572801+0000","last_unstale":"2024-09-05T12:27:44.523947+0000","last_undegraded":"2024-09-05T12:27:44.523947+0000","last_fullsized":"2024-09-05T12:27:44.523947+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:14:57.667655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506890+0000","last_change":"2024-09-05T12:24:48.514659+0000","last_active":"2024-09-05T12:27:44.506890+0000","last_peered":"2024-09-05T12:27:44.506890+0000","last_clean":"2024-09-05T12:27:44.506890+0000","last_became_active":"2024-09-05T12:24:48.514321+0000","last_became_peered":"2024-09-05T12:24:48.514321+0000","last_unstale":"2024-09-05T12:27:44.506890+0000","last_undegraded":"2024-09-05T12:27:44.506890+0000","last_fullsized":"2024-09-05T12:27:44.506890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:37:49.594175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519489+0000","last_change":"2024-09-05T12:24:46.036114+0000","last_active":"2024-09-05T12:27:44.519489+0000","last_peered":"2024-09-05T12:27:44.519489+0000","last_clean":"2024-09-05T12:27:44.519489+0000","last_became_active":"2024-09-05T12:24:46.035740+0000","last_became_peered":"2024-09-05T12:24:46.035740+0000","last_unstale":"2024-09-05T12:27:44.519489+0000","last_undegraded":"2024-09-05T12:27:44.519489+0000","last_fullsized":"2024-09-05T12:27:44.519489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:08.819298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726953+0000","last_change":"2024-09-05T12:25:34.271674+0000","last_active":"2024-09-05T12:27:44.726953+0000","last_peered":"2024-09-05T12:27:44.726953+0000","last_clean":"2024-09-05T12:27:44.726953+0000","last_became_active":"2024-09-05T12:25:34.271240+0000","last_became_peered":"2024-09-05T12:25:34.271240+0000","last_unstale":"2024-09-05T12:27:44.726953+0000","last_undegraded":"2024-09-05T12:27:44.726953+0000","last_fullsized":"2024-09-05T12:27:44.726953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:48:26.194530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517295+0000","last_change":"2024-09-05T12:24:48.515861+0000","last_active":"2024-09-05T12:27:44.517295+0000","last_peered":"2024-09-05T12:27:44.517295+0000","last_clean":"2024-09-05T12:27:44.517295+0000","last_became_active":"2024-09-05T12:24:48.513728+0000","last_became_peered":"2024-09-05T12:24:48.513728+0000","last_unstale":"2024-09-05T12:27:44.517295+0000","last_undegraded":"2024-09-05T12:27:44.517295+0000","last_fullsized":"2024-09-05T12:27:44.517295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-07T00:08:50.298064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":51,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553650+0000","last_change":"2024-09-05T12:24:46.036269+0000","last_active":"2024-09-05T12:27:44.553650+0000","last_peered":"2024-09-05T12:27:44.553650+0000","last_clean":"2024-09-05T12:27:44.553650+0000","last_became_active":"2024-09-05T12:24:46.035795+0000","last_became_peered":"2024-09-05T12:24:46.035795+0000","last_unstale":"2024-09-05T12:27:44.553650+0000","last_undegraded":"2024-09-05T12:27:44.553650+0000","last_fullsized":"2024-09-05T12:27:44.553650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:35:12.187802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520009+0000","last_change":"2024-09-05T12:24:58.581959+0000","last_active":"2024-09-05T12:27:44.520009+0000","last_peered":"2024-09-05T12:27:44.520009+0000","last_clean":"2024-09-05T12:27:44.520009+0000","last_became_active":"2024-09-05T12:24:58.581515+0000","last_became_peered":"2024-09-05T12:24:58.581515+0000","last_unstale":"2024-09-05T12:27:44.520009+0000","last_undegraded":"2024-09-05T12:27:44.520009+0000","last_fullsized":"2024-09-05T12:27:44.520009+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:57:05.450238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509567+0000","last_change":"2024-09-05T12:24:48.518431+0000","last_active":"2024-09-05T12:27:44.509567+0000","last_peered":"2024-09-05T12:27:44.509567+0000","last_clean":"2024-09-05T12:27:44.509567+0000","last_became_active":"2024-09-05T12:24:48.518138+0000","last_became_peered":"2024-09-05T12:24:48.518138+0000","last_unstale":"2024-09-05T12:27:44.509567+0000","last_undegraded":"2024-09-05T12:27:44.509567+0000","last_fullsized":"2024-09-05T12:27:44.509567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:32:00.785619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":357,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:50.513783+0000","last_change":"2024-09-05T12:24:46.040558+0000","last_active":"2024-09-05T12:27:50.513783+0000","last_peered":"2024-09-05T12:27:50.513783+0000","last_clean":"2024-09-05T12:27:50.513783+0000","last_became_active":"2024-09-05T12:24:46.040211+0000","last_became_peered":"2024-09-05T12:24:46.040211+0000","last_unstale":"2024-09-05T12:27:50.513783+0000","last_undegraded":"2024-09-05T12:27:50.513783+0000","last_fullsized":"2024-09-05T12:27:50.513783+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:03:34.442906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68198,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":276,"num_read_kb":205,"num_write":43,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522273+0000","last_change":"2024-09-05T12:24:58.574149+0000","last_active":"2024-09-05T12:27:44.522273+0000","last_peered":"2024-09-05T12:27:44.522273+0000","last_clean":"2024-09-05T12:27:44.522273+0000","last_became_active":"2024-09-05T12:24:58.573742+0000","last_became_peered":"2024-09-05T12:24:58.573742+0000","last_unstale":"2024-09-05T12:27:44.522273+0000","last_undegraded":"2024-09-05T12:27:44.522273+0000","last_fullsized":"2024-09-05T12:27:44.522273+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:22:22.785293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712959+0000","last_change":"2024-09-05T12:24:48.520427+0000","last_active":"2024-09-05T12:27:44.712959+0000","last_peered":"2024-09-05T12:27:44.712959+0000","last_clean":"2024-09-05T12:27:44.712959+0000","last_became_active":"2024-09-05T12:24:48.518262+0000","last_became_peered":"2024-09-05T12:24:48.518262+0000","last_unstale":"2024-09-05T12:27:44.712959+0000","last_undegraded":"2024-09-05T12:27:44.712959+0000","last_fullsized":"2024-09-05T12:27:44.712959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:21:59.521741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520278+0000","last_change":"2024-09-05T12:24:46.065105+0000","last_active":"2024-09-05T12:27:44.520278+0000","last_peered":"2024-09-05T12:27:44.520278+0000","last_clean":"2024-09-05T12:27:44.520278+0000","last_became_active":"2024-09-05T12:24:46.064701+0000","last_became_peered":"2024-09-05T12:24:46.064701+0000","last_unstale":"2024-09-05T12:27:44.520278+0000","last_undegraded":"2024-09-05T12:27:44.520278+0000","last_fullsized":"2024-09-05T12:27:44.520278+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:21:54.034783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522012+0000","last_change":"2024-09-05T12:24:58.583364+0000","last_active":"2024-09-05T12:27:44.522012+0000","last_peered":"2024-09-05T12:27:44.522012+0000","last_clean":"2024-09-05T12:27:44.522012+0000","last_became_active":"2024-09-05T12:24:58.582841+0000","last_became_peered":"2024-09-05T12:24:58.582841+0000","last_unstale":"2024-09-05T12:27:44.522012+0000","last_undegraded":"2024-09-05T12:27:44.522012+0000","last_fullsized":"2024-09-05T12:27:44.522012+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:44:10.536631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884499+0000","last_change":"2024-09-05T12:24:48.533862+0000","last_active":"2024-09-05T12:27:44.884499+0000","last_peered":"2024-09-05T12:27:44.884499+0000","last_clean":"2024-09-05T12:27:44.884499+0000","last_became_active":"2024-09-05T12:24:48.531902+0000","last_became_peered":"2024-09-05T12:24:48.531902+0000","last_unstale":"2024-09-05T12:27:44.884499+0000","last_undegraded":"2024-09-05T12:27:44.884499+0000","last_fullsized":"2024-09-05T12:27:44.884499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:34.084874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507322+0000","last_change":"2024-09-05T12:24:46.040077+0000","last_active":"2024-09-05T12:27:44.507322+0000","last_peered":"2024-09-05T12:27:44.507322+0000","last_clean":"2024-09-05T12:27:44.507322+0000","last_became_active":"2024-09-05T12:24:46.039585+0000","last_became_peered":"2024-09-05T12:24:46.039585+0000","last_unstale":"2024-09-05T12:27:44.507322+0000","last_undegraded":"2024-09-05T12:27:44.507322+0000","last_fullsized":"2024-09-05T12:27:44.507322+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:48:57.061515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509000+0000","last_change":"2024-09-05T12:25:34.271523+0000","last_active":"2024-09-05T12:27:44.509000+0000","last_peered":"2024-09-05T12:27:44.509000+0000","last_clean":"2024-09-05T12:27:44.509000+0000","last_became_active":"2024-09-05T12:25:34.270541+0000","last_became_peered":"2024-09-05T12:25:34.270541+0000","last_unstale":"2024-09-05T12:27:44.509000+0000","last_undegraded":"2024-09-05T12:27:44.509000+0000","last_fullsized":"2024-09-05T12:27:44.509000+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:08:20.989002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2,4],"acting":[8,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712976+0000","last_change":"2024-09-05T12:24:48.528427+0000","last_active":"2024-09-05T12:27:44.712976+0000","last_peered":"2024-09-05T12:27:44.712976+0000","last_clean":"2024-09-05T12:27:44.712976+0000","last_became_active":"2024-09-05T12:24:48.528048+0000","last_became_peered":"2024-09-05T12:24:48.528048+0000","last_unstale":"2024-09-05T12:27:44.712976+0000","last_undegraded":"2024-09-05T12:27:44.712976+0000","last_fullsized":"2024-09-05T12:27:44.712976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:14:46.286990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"95'7","reported_seq":311,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:50.513782+0000","last_change":"2024-09-05T12:24:46.036240+0000","last_active":"2024-09-05T12:27:50.513782+0000","last_peered":"2024-09-05T12:27:50.513782+0000","last_clean":"2024-09-05T12:27:50.513782+0000","last_became_active":"2024-09-05T12:24:46.035863+0000","last_became_peered":"2024-09-05T12:24:46.035863+0000","last_unstale":"2024-09-05T12:27:50.513782+0000","last_undegraded":"2024-09-05T12:27:50.513782+0000","last_fullsized":"2024-09-05T12:27:50.513782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:16:56.480708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":264,"num_read_kb":264,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505528+0000","last_change":"2024-09-05T12:24:58.574341+0000","last_active":"2024-09-05T12:27:44.505528+0000","last_peered":"2024-09-05T12:27:44.505528+0000","last_clean":"2024-09-05T12:27:44.505528+0000","last_became_active":"2024-09-05T12:24:58.573987+0000","last_became_peered":"2024-09-05T12:24:58.573987+0000","last_unstale":"2024-09-05T12:27:44.505528+0000","last_undegraded":"2024-09-05T12:27:44.505528+0000","last_fullsized":"2024-09-05T12:27:44.505528+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:05:10.695278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887297+0000","last_change":"2024-09-05T12:24:58.582910+0000","last_active":"2024-09-05T12:27:44.887297+0000","last_peered":"2024-09-05T12:27:44.887297+0000","last_clean":"2024-09-05T12:27:44.887297+0000","last_became_active":"2024-09-05T12:24:58.582489+0000","last_became_peered":"2024-09-05T12:24:58.582489+0000","last_unstale":"2024-09-05T12:27:44.887297+0000","last_undegraded":"2024-09-05T12:27:44.887297+0000","last_fullsized":"2024-09-05T12:27:44.887297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:44:05.679042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.522239+0000","last_change":"2024-09-05T12:24:48.527725+0000","last_active":"2024-09-05T12:27:44.522239+0000","last_peered":"2024-09-05T12:27:44.522239+0000","last_clean":"2024-09-05T12:27:44.522239+0000","last_became_active":"2024-09-05T12:24:48.526932+0000","last_became_peered":"2024-09-05T12:24:48.526932+0000","last_unstale":"2024-09-05T12:27:44.522239+0000","last_undegraded":"2024-09-05T12:27:44.522239+0000","last_fullsized":"2024-09-05T12:27:44.522239+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:49:07.881286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519527+0000","last_change":"2024-09-05T12:24:46.038809+0000","last_active":"2024-09-05T12:27:44.519527+0000","last_peered":"2024-09-05T12:27:44.519527+0000","last_clean":"2024-09-05T12:27:44.519527+0000","last_became_active":"2024-09-05T12:24:46.038485+0000","last_became_peered":"2024-09-05T12:24:46.038485+0000","last_unstale":"2024-09-05T12:27:44.519527+0000","last_undegraded":"2024-09-05T12:27:44.519527+0000","last_fullsized":"2024-09-05T12:27:44.519527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:34:43.544061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727158+0000","last_change":"2024-09-05T12:24:58.583074+0000","last_active":"2024-09-05T12:27:44.727158+0000","last_peered":"2024-09-05T12:27:44.727158+0000","last_clean":"2024-09-05T12:27:44.727158+0000","last_became_active":"2024-09-05T12:24:58.582643+0000","last_became_peered":"2024-09-05T12:24:58.582643+0000","last_unstale":"2024-09-05T12:27:44.727158+0000","last_undegraded":"2024-09-05T12:27:44.727158+0000","last_fullsized":"2024-09-05T12:27:44.727158+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:25:51.594320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505490+0000","last_change":"2024-09-05T12:24:48.517407+0000","last_active":"2024-09-05T12:27:44.505490+0000","last_peered":"2024-09-05T12:27:44.505490+0000","last_clean":"2024-09-05T12:27:44.505490+0000","last_became_active":"2024-09-05T12:24:48.515660+0000","last_became_peered":"2024-09-05T12:24:48.515660+0000","last_unstale":"2024-09-05T12:27:44.505490+0000","last_undegraded":"2024-09-05T12:27:44.505490+0000","last_fullsized":"2024-09-05T12:27:44.505490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:10:44.234302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516334+0000","last_change":"2024-09-05T12:24:46.036147+0000","last_active":"2024-09-05T12:27:44.516334+0000","last_peered":"2024-09-05T12:27:44.516334+0000","last_clean":"2024-09-05T12:27:44.516334+0000","last_became_active":"2024-09-05T12:24:46.035810+0000","last_became_peered":"2024-09-05T12:24:46.035810+0000","last_unstale":"2024-09-05T12:27:44.516334+0000","last_undegraded":"2024-09-05T12:27:44.516334+0000","last_fullsized":"2024-09-05T12:27:44.516334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:48:12.837318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515852+0000","last_change":"2024-09-05T12:24:46.034541+0000","last_active":"2024-09-05T12:27:44.515852+0000","last_peered":"2024-09-05T12:27:44.515852+0000","last_clean":"2024-09-05T12:27:44.515852+0000","last_became_active":"2024-09-05T12:24:46.034008+0000","last_became_peered":"2024-09-05T12:24:46.034008+0000","last_unstale":"2024-09-05T12:27:44.515852+0000","last_undegraded":"2024-09-05T12:27:44.515852+0000","last_fullsized":"2024-09-05T12:27:44.515852+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:59:53.088142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.518959+0000","last_change":"2024-09-05T12:24:48.512874+0000","last_active":"2024-09-05T12:27:44.518959+0000","last_peered":"2024-09-05T12:27:44.518959+0000","last_clean":"2024-09-05T12:27:44.518959+0000","last_became_active":"2024-09-05T12:24:48.512521+0000","last_became_peered":"2024-09-05T12:24:48.512521+0000","last_unstale":"2024-09-05T12:27:44.518959+0000","last_undegraded":"2024-09-05T12:27:44.518959+0000","last_fullsized":"2024-09-05T12:27:44.518959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:09:11.555307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553680+0000","last_change":"2024-09-05T12:24:58.573560+0000","last_active":"2024-09-05T12:27:44.553680+0000","last_peered":"2024-09-05T12:27:44.553680+0000","last_clean":"2024-09-05T12:27:44.553680+0000","last_became_active":"2024-09-05T12:24:58.573183+0000","last_became_peered":"2024-09-05T12:24:58.573183+0000","last_unstale":"2024-09-05T12:27:44.553680+0000","last_undegraded":"2024-09-05T12:27:44.553680+0000","last_fullsized":"2024-09-05T12:27:44.553680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:12:43.189696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"95'7","reported_seq":184,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:49.526696+0000","last_change":"2024-09-05T12:24:46.041039+0000","last_active":"2024-09-05T12:27:49.526696+0000","last_peered":"2024-09-05T12:27:49.526696+0000","last_clean":"2024-09-05T12:27:49.526696+0000","last_became_active":"2024-09-05T12:24:46.040263+0000","last_became_peered":"2024-09-05T12:24:46.040263+0000","last_unstale":"2024-09-05T12:27:49.526696+0000","last_undegraded":"2024-09-05T12:27:49.526696+0000","last_fullsized":"2024-09-05T12:27:49.526696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:45.013119+0000","last_clean_scrub_stamp":"2024-09-05T12:24:45.013119+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:49:04.428167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1326,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509141+0000","last_change":"2024-09-05T12:24:48.523380+0000","last_active":"2024-09-05T12:27:44.509141+0000","last_peered":"2024-09-05T12:27:44.509141+0000","last_clean":"2024-09-05T12:27:44.509141+0000","last_became_active":"2024-09-05T12:24:48.523056+0000","last_became_peered":"2024-09-05T12:24:48.523056+0000","last_unstale":"2024-09-05T12:27:44.509141+0000","last_undegraded":"2024-09-05T12:27:44.509141+0000","last_fullsized":"2024-09-05T12:27:44.509141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:30:54.366798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553781+0000","last_change":"2024-09-05T12:24:58.570007+0000","last_active":"2024-09-05T12:27:44.553781+0000","last_peered":"2024-09-05T12:27:44.553781+0000","last_clean":"2024-09-05T12:27:44.553781+0000","last_became_active":"2024-09-05T12:24:58.569594+0000","last_became_peered":"2024-09-05T12:24:58.569594+0000","last_unstale":"2024-09-05T12:27:44.553781+0000","last_undegraded":"2024-09-05T12:27:44.553781+0000","last_fullsized":"2024-09-05T12:27:44.553781+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:57.547398+0000","last_clean_scrub_stamp":"2024-09-05T12:24:57.547398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:03:45.078373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726440+0000","last_change":"2024-09-05T12:24:48.518739+0000","last_active":"2024-09-05T12:27:44.726440+0000","last_peered":"2024-09-05T12:27:44.726440+0000","last_clean":"2024-09-05T12:27:44.726440+0000","last_became_active":"2024-09-05T12:24:48.518416+0000","last_became_peered":"2024-09-05T12:24:48.518416+0000","last_unstale":"2024-09-05T12:27:44.726440+0000","last_undegraded":"2024-09-05T12:27:44.726440+0000","last_fullsized":"2024-09-05T12:27:44.726440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:14:45.067387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553578+0000","last_change":"2024-09-05T12:24:48.513106+0000","last_active":"2024-09-05T12:27:44.553578+0000","last_peered":"2024-09-05T12:27:44.553578+0000","last_clean":"2024-09-05T12:27:44.553578+0000","last_became_active":"2024-09-05T12:24:48.512759+0000","last_became_peered":"2024-09-05T12:24:48.512759+0000","last_unstale":"2024-09-05T12:27:44.553578+0000","last_undegraded":"2024-09-05T12:27:44.553578+0000","last_fullsized":"2024-09-05T12:27:44.553578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:26:30.710938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553078+0000","last_change":"2024-09-05T12:24:48.524258+0000","last_active":"2024-09-05T12:27:44.553078+0000","last_peered":"2024-09-05T12:27:44.553078+0000","last_clean":"2024-09-05T12:27:44.553078+0000","last_became_active":"2024-09-05T12:24:48.523883+0000","last_became_peered":"2024-09-05T12:24:48.523883+0000","last_unstale":"2024-09-05T12:27:44.553078+0000","last_undegraded":"2024-09-05T12:27:44.553078+0000","last_fullsized":"2024-09-05T12:27:44.553078+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:13:31.267292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887351+0000","last_change":"2024-09-05T12:25:34.270400+0000","last_active":"2024-09-05T12:27:44.887351+0000","last_peered":"2024-09-05T12:27:44.887351+0000","last_clean":"2024-09-05T12:27:44.887351+0000","last_became_active":"2024-09-05T12:25:34.269998+0000","last_became_peered":"2024-09-05T12:25:34.269998+0000","last_unstale":"2024-09-05T12:27:44.887351+0000","last_undegraded":"2024-09-05T12:27:44.887351+0000","last_fullsized":"2024-09-05T12:27:44.887351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:23:08.292195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,8],"acting":[0,6,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.517395+0000","last_change":"2024-09-05T12:25:34.331799+0000","last_active":"2024-09-05T12:27:44.517395+0000","last_peered":"2024-09-05T12:27:44.517395+0000","last_clean":"2024-09-05T12:27:44.517395+0000","last_became_active":"2024-09-05T12:25:34.331387+0000","last_became_peered":"2024-09-05T12:25:34.331387+0000","last_unstale":"2024-09-05T12:27:44.517395+0000","last_undegraded":"2024-09-05T12:27:44.517395+0000","last_fullsized":"2024-09-05T12:27:44.517395+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:03:01.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726513+0000","last_change":"2024-09-05T12:24:48.516603+0000","last_active":"2024-09-05T12:27:44.726513+0000","last_peered":"2024-09-05T12:27:44.726513+0000","last_clean":"2024-09-05T12:27:44.726513+0000","last_became_active":"2024-09-05T12:24:48.516208+0000","last_became_peered":"2024-09-05T12:24:48.516208+0000","last_unstale":"2024-09-05T12:27:44.726513+0000","last_undegraded":"2024-09-05T12:27:44.726513+0000","last_fullsized":"2024-09-05T12:27:44.726513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:19:10.858058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":50,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553154+0000","last_change":"2024-09-05T12:24:48.520327+0000","last_active":"2024-09-05T12:27:44.553154+0000","last_peered":"2024-09-05T12:27:44.553154+0000","last_clean":"2024-09-05T12:27:44.553154+0000","last_became_active":"2024-09-05T12:24:48.519833+0000","last_became_peered":"2024-09-05T12:24:48.519833+0000","last_unstale":"2024-09-05T12:27:44.553154+0000","last_undegraded":"2024-09-05T12:27:44.553154+0000","last_fullsized":"2024-09-05T12:27:44.553154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:54:40.437107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506190+0000","last_change":"2024-09-05T12:24:48.513455+0000","last_active":"2024-09-05T12:27:44.506190+0000","last_peered":"2024-09-05T12:27:44.506190+0000","last_clean":"2024-09-05T12:27:44.506190+0000","last_became_active":"2024-09-05T12:24:48.511523+0000","last_became_peered":"2024-09-05T12:24:48.511523+0000","last_unstale":"2024-09-05T12:27:44.506190+0000","last_undegraded":"2024-09-05T12:27:44.506190+0000","last_fullsized":"2024-09-05T12:27:44.506190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:29:05.457594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.519213+0000","last_change":"2024-09-05T12:24:48.524775+0000","last_active":"2024-09-05T12:27:44.519213+0000","last_peered":"2024-09-05T12:27:44.519213+0000","last_clean":"2024-09-05T12:27:44.519213+0000","last_became_active":"2024-09-05T12:24:48.524472+0000","last_became_peered":"2024-09-05T12:24:48.524472+0000","last_unstale":"2024-09-05T12:27:44.519213+0000","last_undegraded":"2024-09-05T12:27:44.519213+0000","last_fullsized":"2024-09-05T12:27:44.519213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:32:51.202423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505694+0000","last_change":"2024-09-05T12:24:48.518228+0000","last_active":"2024-09-05T12:27:44.505694+0000","last_peered":"2024-09-05T12:27:44.505694+0000","last_clean":"2024-09-05T12:27:44.505694+0000","last_became_active":"2024-09-05T12:24:48.512027+0000","last_became_peered":"2024-09-05T12:24:48.512027+0000","last_unstale":"2024-09-05T12:27:44.505694+0000","last_undegraded":"2024-09-05T12:27:44.505694+0000","last_fullsized":"2024-09-05T12:27:44.505694+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:08:45.722655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712927+0000","last_change":"2024-09-05T12:24:48.517736+0000","last_active":"2024-09-05T12:27:44.712927+0000","last_peered":"2024-09-05T12:27:44.712927+0000","last_clean":"2024-09-05T12:27:44.712927+0000","last_became_active":"2024-09-05T12:24:48.517235+0000","last_became_peered":"2024-09-05T12:24:48.517235+0000","last_unstale":"2024-09-05T12:27:44.712927+0000","last_undegraded":"2024-09-05T12:27:44.712927+0000","last_fullsized":"2024-09-05T12:27:44.712927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:52:12.860505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":48,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516572+0000","last_change":"2024-09-05T12:24:48.529737+0000","last_active":"2024-09-05T12:27:44.516572+0000","last_peered":"2024-09-05T12:27:44.516572+0000","last_clean":"2024-09-05T12:27:44.516572+0000","last_became_active":"2024-09-05T12:24:48.529368+0000","last_became_peered":"2024-09-05T12:24:48.529368+0000","last_unstale":"2024-09-05T12:27:44.516572+0000","last_undegraded":"2024-09-05T12:27:44.516572+0000","last_fullsized":"2024-09-05T12:27:44.516572+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T21:43:30.602105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.726524+0000","last_change":"2024-09-05T12:24:48.516049+0000","last_active":"2024-09-05T12:27:44.726524+0000","last_peered":"2024-09-05T12:27:44.726524+0000","last_clean":"2024-09-05T12:27:44.726524+0000","last_became_active":"2024-09-05T12:24:48.515676+0000","last_became_peered":"2024-09-05T12:24:48.515676+0000","last_unstale":"2024-09-05T12:27:44.726524+0000","last_undegraded":"2024-09-05T12:27:44.726524+0000","last_fullsized":"2024-09-05T12:27:44.726524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T17:38:45.340231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.553110+0000","last_change":"2024-09-05T12:24:48.510749+0000","last_active":"2024-09-05T12:27:44.553110+0000","last_peered":"2024-09-05T12:27:44.553110+0000","last_clean":"2024-09-05T12:27:44.553110+0000","last_became_active":"2024-09-05T12:24:48.510333+0000","last_became_peered":"2024-09-05T12:24:48.510333+0000","last_unstale":"2024-09-05T12:27:44.553110+0000","last_undegraded":"2024-09-05T12:27:44.553110+0000","last_fullsized":"2024-09-05T12:27:44.553110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T23:38:13.032973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.523087+0000","last_change":"2024-09-05T12:24:48.526440+0000","last_active":"2024-09-05T12:27:44.523087+0000","last_peered":"2024-09-05T12:27:44.523087+0000","last_clean":"2024-09-05T12:27:44.523087+0000","last_became_active":"2024-09-05T12:24:48.525986+0000","last_became_peered":"2024-09-05T12:24:48.525986+0000","last_unstale":"2024-09-05T12:27:44.523087+0000","last_undegraded":"2024-09-05T12:27:44.523087+0000","last_fullsized":"2024-09-05T12:27:44.523087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:24:09.536983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727310+0000","last_change":"2024-09-05T12:24:48.537290+0000","last_active":"2024-09-05T12:27:44.727310+0000","last_peered":"2024-09-05T12:27:44.727310+0000","last_clean":"2024-09-05T12:27:44.727310+0000","last_became_active":"2024-09-05T12:24:48.537037+0000","last_became_peered":"2024-09-05T12:24:48.537037+0000","last_unstale":"2024-09-05T12:27:44.727310+0000","last_undegraded":"2024-09-05T12:27:44.727310+0000","last_fullsized":"2024-09-05T12:27:44.727310+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T22:30:21.580483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":44,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.712915+0000","last_change":"2024-09-05T12:24:48.524380+0000","last_active":"2024-09-05T12:27:44.712915+0000","last_peered":"2024-09-05T12:27:44.712915+0000","last_clean":"2024-09-05T12:27:44.712915+0000","last_became_active":"2024-09-05T12:24:48.523960+0000","last_became_peered":"2024-09-05T12:24:48.523960+0000","last_unstale":"2024-09-05T12:27:44.712915+0000","last_undegraded":"2024-09-05T12:27:44.712915+0000","last_fullsized":"2024-09-05T12:27:44.712915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:46:58.164845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884369+0000","last_change":"2024-09-05T12:24:48.517223+0000","last_active":"2024-09-05T12:27:44.884369+0000","last_peered":"2024-09-05T12:27:44.884369+0000","last_clean":"2024-09-05T12:27:44.884369+0000","last_became_active":"2024-09-05T12:24:48.516916+0000","last_became_peered":"2024-09-05T12:24:48.516916+0000","last_unstale":"2024-09-05T12:27:44.884369+0000","last_undegraded":"2024-09-05T12:27:44.884369+0000","last_fullsized":"2024-09-05T12:27:44.884369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:52:51.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.884201+0000","last_change":"2024-09-05T12:24:48.528256+0000","last_active":"2024-09-05T12:27:44.884201+0000","last_peered":"2024-09-05T12:27:44.884201+0000","last_clean":"2024-09-05T12:27:44.884201+0000","last_became_active":"2024-09-05T12:24:48.524611+0000","last_became_peered":"2024-09-05T12:24:48.524611+0000","last_unstale":"2024-09-05T12:27:44.884201+0000","last_undegraded":"2024-09-05T12:27:44.884201+0000","last_fullsized":"2024-09-05T12:27:44.884201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:10:50.939452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":47,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516382+0000","last_change":"2024-09-05T12:24:48.517778+0000","last_active":"2024-09-05T12:27:44.516382+0000","last_peered":"2024-09-05T12:27:44.516382+0000","last_clean":"2024-09-05T12:27:44.516382+0000","last_became_active":"2024-09-05T12:24:48.517451+0000","last_became_peered":"2024-09-05T12:24:48.517451+0000","last_unstale":"2024-09-05T12:27:44.516382+0000","last_undegraded":"2024-09-05T12:27:44.516382+0000","last_fullsized":"2024-09-05T12:27:44.516382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T15:49:20.327437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.520915+0000","last_change":"2024-09-05T12:24:48.513534+0000","last_active":"2024-09-05T12:27:44.520915+0000","last_peered":"2024-09-05T12:27:44.520915+0000","last_clean":"2024-09-05T12:27:44.520915+0000","last_became_active":"2024-09-05T12:24:48.513232+0000","last_became_peered":"2024-09-05T12:24:48.513232+0000","last_unstale":"2024-09-05T12:27:44.520915+0000","last_undegraded":"2024-09-05T12:27:44.520915+0000","last_fullsized":"2024-09-05T12:27:44.520915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T18:54:39.820418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.507111+0000","last_change":"2024-09-05T12:24:48.520081+0000","last_active":"2024-09-05T12:27:44.507111+0000","last_peered":"2024-09-05T12:27:44.507111+0000","last_clean":"2024-09-05T12:27:44.507111+0000","last_became_active":"2024-09-05T12:24:48.519539+0000","last_became_peered":"2024-09-05T12:24:48.519539+0000","last_unstale":"2024-09-05T12:27:44.507111+0000","last_undegraded":"2024-09-05T12:27:44.507111+0000","last_fullsized":"2024-09-05T12:27:44.507111+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:08:59.099322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":57,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.516166+0000","last_change":"2024-09-05T12:25:34.331174+0000","last_active":"2024-09-05T12:27:44.516166+0000","last_peered":"2024-09-05T12:27:44.516166+0000","last_clean":"2024-09-05T12:27:44.516166+0000","last_became_active":"2024-09-05T12:25:34.330811+0000","last_became_peered":"2024-09-05T12:25:34.330811+0000","last_unstale":"2024-09-05T12:27:44.516166+0000","last_undegraded":"2024-09-05T12:27:44.516166+0000","last_fullsized":"2024-09-05T12:27:44.516166+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:32:56.347514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.713259+0000","last_change":"2024-09-05T12:24:48.522013+0000","last_active":"2024-09-05T12:27:44.713259+0000","last_peered":"2024-09-05T12:27:44.713259+0000","last_clean":"2024-09-05T12:27:44.713259+0000","last_became_active":"2024-09-05T12:24:48.521372+0000","last_became_peered":"2024-09-05T12:24:48.521372+0000","last_unstale":"2024-09-05T12:27:44.713259+0000","last_undegraded":"2024-09-05T12:27:44.713259+0000","last_fullsized":"2024-09-05T12:27:44.713259+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:00:37.762138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":46,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.505460+0000","last_change":"2024-09-05T12:24:48.519214+0000","last_active":"2024-09-05T12:27:44.505460+0000","last_peered":"2024-09-05T12:27:44.505460+0000","last_clean":"2024-09-05T12:27:44.505460+0000","last_became_active":"2024-09-05T12:24:48.513595+0000","last_became_peered":"2024-09-05T12:24:48.513595+0000","last_unstale":"2024-09-05T12:27:44.505460+0000","last_undegraded":"2024-09-05T12:27:44.505460+0000","last_fullsized":"2024-09-05T12:27:44.505460+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:00:00.098241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.506619+0000","last_change":"2024-09-05T12:24:48.528552+0000","last_active":"2024-09-05T12:27:44.506619+0000","last_peered":"2024-09-05T12:27:44.506619+0000","last_clean":"2024-09-05T12:27:44.506619+0000","last_became_active":"2024-09-05T12:24:48.528187+0000","last_became_peered":"2024-09-05T12:24:48.528187+0000","last_unstale":"2024-09-05T12:27:44.506619+0000","last_undegraded":"2024-09-05T12:27:44.506619+0000","last_fullsized":"2024-09-05T12:27:44.506619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T12:29:13.877467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.515791+0000","last_change":"2024-09-05T12:24:48.528679+0000","last_active":"2024-09-05T12:27:44.515791+0000","last_peered":"2024-09-05T12:27:44.515791+0000","last_clean":"2024-09-05T12:27:44.515791+0000","last_became_active":"2024-09-05T12:24:48.528358+0000","last_became_peered":"2024-09-05T12:24:48.528358+0000","last_unstale":"2024-09-05T12:27:44.515791+0000","last_undegraded":"2024-09-05T12:27:44.515791+0000","last_fullsized":"2024-09-05T12:27:44.515791+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T19:39:56.727791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.509128+0000","last_change":"2024-09-05T12:24:48.514840+0000","last_active":"2024-09-05T12:27:44.509128+0000","last_peered":"2024-09-05T12:27:44.509128+0000","last_clean":"2024-09-05T12:27:44.509128+0000","last_became_active":"2024-09-05T12:24:48.514304+0000","last_became_peered":"2024-09-05T12:24:48.514304+0000","last_unstale":"2024-09-05T12:27:44.509128+0000","last_undegraded":"2024-09-05T12:27:44.509128+0000","last_fullsized":"2024-09-05T12:27:44.509128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:38:15.104854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.521146+0000","last_change":"2024-09-05T12:24:48.511541+0000","last_active":"2024-09-05T12:27:44.521146+0000","last_peered":"2024-09-05T12:27:44.521146+0000","last_clean":"2024-09-05T12:27:44.521146+0000","last_became_active":"2024-09-05T12:24:48.511162+0000","last_became_peered":"2024-09-05T12:24:48.511162+0000","last_unstale":"2024-09-05T12:27:44.521146+0000","last_undegraded":"2024-09-05T12:27:44.521146+0000","last_fullsized":"2024-09-05T12:27:44.521146+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T16:47:05.103596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.885260+0000","last_change":"2024-09-05T12:24:48.527155+0000","last_active":"2024-09-05T12:27:44.885260+0000","last_peered":"2024-09-05T12:27:44.885260+0000","last_clean":"2024-09-05T12:27:44.885260+0000","last_became_active":"2024-09-05T12:24:48.525275+0000","last_became_peered":"2024-09-05T12:24:48.525275+0000","last_unstale":"2024-09-05T12:27:44.885260+0000","last_undegraded":"2024-09-05T12:27:44.885260+0000","last_fullsized":"2024-09-05T12:27:44.885260+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T14:50:38.971057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.887150+0000","last_change":"2024-09-05T12:24:48.529829+0000","last_active":"2024-09-05T12:27:44.887150+0000","last_peered":"2024-09-05T12:27:44.887150+0000","last_clean":"2024-09-05T12:27:44.887150+0000","last_became_active":"2024-09-05T12:24:48.529520+0000","last_became_peered":"2024-09-05T12:24:48.529520+0000","last_unstale":"2024-09-05T12:27:44.887150+0000","last_undegraded":"2024-09-05T12:27:44.887150+0000","last_fullsized":"2024-09-05T12:27:44.887150+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:03:32.884222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"94'8","reported_seq":43,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-05T12:27:44.727308+0000","last_change":"2024-09-05T12:24:48.537283+0000","last_active":"2024-09-05T12:27:44.727308+0000","last_peered":"2024-09-05T12:27:44.727308+0000","last_clean":"2024-09-05T12:27:44.727308+0000","last_became_active":"2024-09-05T12:24:48.537002+0000","last_became_peered":"2024-09-05T12:24:48.537002+0000","last_unstale":"2024-09-05T12:27:44.727308+0000","last_undegraded":"2024-09-05T12:27:44.727308+0000","last_fullsized":"2024-09-05T12:27:44.727308+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-05T12:24:47.492289+0000","last_clean_scrub_stamp":"2024-09-05T12:24:47.492289+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-06T20:24:15.185652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798878,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1713,"num_read_kb":1079,"num_write":509,"num_write_kb":2387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2764800,"data_stored":2396634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449138,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1101824,"data_stored":544673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":71,"seq":304942678070,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29108,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716044,"statfs":{"total":95995035648,"available":95965229056,"internally_reserved":0,"allocated":2134016,"data_stored":1531066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":66,"seq":283467841595,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712488,"statfs":{"total":95995035648,"available":95961587712,"internally_reserved":0,"allocated":1515520,"data_stored":936477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":58,"seq":249108103233,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36816,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708336,"statfs":{"total":95995035648,"available":95957336064,"internally_reserved":0,"allocated":1507328,"data_stored":936833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364870,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37020,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708132,"statfs":{"total":95995035648,"available":95957127168,"internally_reserved":0,"allocated":1716224,"data_stored":1141235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528395,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1097728,"data_stored":539158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790034,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29152,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716000,"statfs":{"total":95995035648,"available":95965184000,"internally_reserved":0,"allocated":2056192,"data_stored":1464263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953558,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1105920,"data_stored":543232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28936,"kb_used_data":1792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716216,"statfs":{"total":95995035648,"available":95965405184,"internally_reserved":0,"allocated":1835008,"data_stored":1259921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36512,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708640,"statfs":{"total":95995035648,"available":95957647360,"internally_reserved":0,"allocated":1191936,"data_stored":610237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36492,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708660,"statfs":{"total":95995035648,"available":95957667840,"internally_reserved":0,"allocated":1171456,"data_stored":608386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1122304,"data_stored":541541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":70343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":72194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":721878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":331756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":398559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-05T12:27:54.110 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:27:54.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:54 smithi071 ceph-mon[29845]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-09-05T12:27:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:54 smithi071 ceph-mon[29845]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:54 smithi123 ceph-mon[34321]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-09-05T12:27:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:54 smithi123 ceph-mon[34321]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:54 smithi174 ceph-mon[34313]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 131 KiB/s wr, 78 op/s 2024-09-05T12:27:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:54 smithi174 ceph-mon[34313]: from='client.25319 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-05T12:27:54.513 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.131+0000 7fbbed200640 1 -- 172.21.15.71:0/3303178132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8102790 msgr2=0x7fbbe81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.131+0000 7fbbed200640 1 --2- 172.21.15.71:0/3303178132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8102790 0x7fbbe81067e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009900 tx=0x7fbbd002a410 comp rx=0 tx=0).stop 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 -- 172.21.15.71:0/3303178132 shutdown_connections 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 --2- 172.21.15.71:0/3303178132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8106f10 0x7fbbe810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 --2- 172.21.15.71:0/3303178132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8102790 0x7fbbe81067e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 --2- 172.21.15.71:0/3303178132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbe8101de0 0x7fbbe81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 -- 172.21.15.71:0/3303178132 >> 172.21.15.71:0/3303178132 conn(0x7fbbe80fd8c0 msgr2=0x7fbbe80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 -- 172.21.15.71:0/3303178132 shutdown_connections 2024-09-05T12:27:55.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.132+0000 7fbbed200640 1 -- 172.21.15.71:0/3303178132 wait complete. 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 Processor -- start 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 -- start start 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 0x7fbbe819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 0x7fbbe819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbe8106f10 0x7fbbe819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111f30 con 0x7fbbe8101de0 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8111db0 con 0x7fbbe8102790 2024-09-05T12:27:55.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.133+0000 7fbbed200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81120b0 con 0x7fbbe8106f10 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 0x7fbbe819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbdd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 0x7fbbe819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 0x7fbbe819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41796/0 (socket says 172.21.15.71:41796) 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbe8106f10 0x7fbbe819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbdd400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 0x7fbbe819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53424/0 (socket says 172.21.15.71:53424) 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 -- 172.21.15.71:0/780327188 learned_addr learned my addr 172.21.15.71:0/780327188 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 -- 172.21.15.71:0/780327188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbe8106f10 msgr2=0x7fbbe819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbe8106f10 0x7fbbe819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 -- 172.21.15.71:0/780327188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 msgr2=0x7fbbe819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 0x7fbbe819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 -- 172.21.15.71:0/780327188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe819f6a0 con 0x7fbbe8101de0 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbdd400640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 0x7fbbe819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:55.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.134+0000 7fbbe5e00640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 0x7fbbe819a6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e9f0 tx=0x7fbbd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:55.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.135+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd800cd80 con 0x7fbbe8101de0 2024-09-05T12:27:55.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.135+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbd8010910 con 0x7fbbe8101de0 2024-09-05T12:27:55.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.135+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe819f990 con 0x7fbbe8101de0 2024-09-05T12:27:55.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.135+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8018990 con 0x7fbbe8101de0 2024-09-05T12:27:55.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.135+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a7270 con 0x7fbbe8101de0 2024-09-05T12:27:55.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.136+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe81021c0 con 0x7fbbe8101de0 2024-09-05T12:27:55.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.138+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7fbbd8010ac0 con 0x7fbbe8101de0 2024-09-05T12:27:55.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.139+0000 7fbbdf400640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbbcc0a2d80 0x7fbbcc0a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:55.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.139+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbbd80cfca0 con 0x7fbbe8101de0 2024-09-05T12:27:55.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.139+0000 7fbbdd400640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbbcc0a2d80 0x7fbbcc0a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:55.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.140+0000 7fbbdd400640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbbcc0a2d80 0x7fbbcc0a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009230 tx=0x7fbbd00058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:55.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.143+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbd809bd60 con 0x7fbbe8101de0 2024-09-05T12:27:55.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.351+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbe81a74a0 con 0x7fbbe8101de0 2024-09-05T12:27:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.352+0000 7fbbdf400640 1 -- 172.21.15.71:0/780327188 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11968 (secure 0 0 0) 0x7fbbd80a0060 con 0x7fbbe8101de0 2024-09-05T12:27:55.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:55.371 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-05T12:27:44:191847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:27:44.168192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:27:55.371 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-05T12:27:55.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbbcc0a2d80 msgr2=0x7fbbcc0a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbbcc0a2d80 0x7fbbcc0a5240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009230 tx=0x7fbbd00058d0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 msgr2=0x7fbbe819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 0x7fbbe819a6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e9f0 tx=0x7fbbd800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 shutdown_connections 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbe8106f10 0x7fbbe819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe8102790 0x7fbbe819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbbcc0a2d80 0x7fbbcc0a5240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 --2- 172.21.15.71:0/780327188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbe8101de0 0x7fbbe819a6f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.354+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 >> 172.21.15.71:0/780327188 conn(0x7fbbe80fd8c0 msgr2=0x7fbbe80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.355+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 shutdown_connections 2024-09-05T12:27:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:55.355+0000 7fbbed200640 1 -- 172.21.15.71:0/780327188 wait complete. 2024-09-05T12:27:56.243 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:27:56.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:56 smithi123 ceph-mon[34321]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-09-05T12:27:56.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/780327188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:56 smithi174 ceph-mon[34313]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-09-05T12:27:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/780327188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:56.610 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:56 smithi071 ceph-mon[29845]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 102 KiB/s wr, 61 op/s 2024-09-05T12:27:56.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/780327188' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:57.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 -- 172.21.15.71:0/2280886695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 msgr2=0x7f3bac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/2280886695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac1021c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009900 tx=0x7f3b9c02a410 comp rx=0 tx=0).stop 2024-09-05T12:27:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 -- 172.21.15.71:0/2280886695 shutdown_connections 2024-09-05T12:27:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/2280886695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bac106f10 0x7f3bac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/2280886695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bac102790 0x7f3bac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/2280886695 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac1021c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 -- 172.21.15.71:0/2280886695 >> 172.21.15.71:0/2280886695 conn(0x7f3bac0fd8c0 msgr2=0x7f3bac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:57.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 -- 172.21.15.71:0/2280886695 shutdown_connections 2024-09-05T12:27:57.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.219+0000 7f3bb2e00640 1 -- 172.21.15.71:0/2280886695 wait complete. 2024-09-05T12:27:57.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.220+0000 7f3bb2e00640 1 Processor -- start 2024-09-05T12:27:57.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.220+0000 7f3bb2e00640 1 -- start start 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.220+0000 7f3bb2e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.220+0000 7f3bb2e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bac102790 0x7f3bac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.220+0000 7f3bb2e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bac106f10 0x7f3bac19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3bb2e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac111fc0 con 0x7f3bac101de0 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3bb2e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac111e40 con 0x7f3bac102790 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3bb2e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac112140 con 0x7f3bac106f10 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41816/0 (socket says 172.21.15.71:41816) 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 -- 172.21.15.71:0/1191056374 learned_addr learned my addr 172.21.15.71:0/1191056374 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3ba3400640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bac102790 0x7f3bac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3bb0e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bac106f10 0x7f3bac19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 -- 172.21.15.71:0/1191056374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bac106f10 msgr2=0x7f3bac19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bac106f10 0x7f3bac19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 -- 172.21.15.71:0/1191056374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bac102790 msgr2=0x7f3bac198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bac102790 0x7f3bac198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.221+0000 7f3babe00640 1 -- 172.21.15.71:0/1191056374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac19d3f0 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.222+0000 7f3babe00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac198440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c004770 tx=0x7f3b9c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.222+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c03f070 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.222+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c031ca0 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.222+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac19d680 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.222+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c035030 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.222+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1a4fc0 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.223+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac1032c0 con 0x7f3bac101de0 2024-09-05T12:27:57.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.225+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f3b9c042070 con 0x7f3bac101de0 2024-09-05T12:27:57.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.226+0000 7f3ba9600640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b900a2d80 0x7f3b900a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:57.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.226+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f3b9c0eff70 con 0x7f3bac101de0 2024-09-05T12:27:57.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.226+0000 7f3ba3400640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b900a2d80 0x7f3b900a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:57.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.227+0000 7f3ba3400640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b900a2d80 0x7f3b900a5240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0fdce0 tx=0x7f3b94005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:57.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.230+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3b9c0bc0f0 con 0x7f3bac101de0 2024-09-05T12:27:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.441+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bac10a0f0 con 0x7f3bac101de0 2024-09-05T12:27:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.442+0000 7f3ba9600640 1 -- 172.21.15.71:0/1191056374 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11968 (secure 0 0 0) 0x7f3b9c0c03f0 con 0x7f3bac101de0 2024-09-05T12:27:57.444 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:57.444 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-05T12:27:44:191847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:27:44.168192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:27:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-05T12:27:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b900a2d80 msgr2=0x7f3b900a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b900a2d80 0x7f3b900a5240 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3bac0fdce0 tx=0x7f3b94005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 msgr2=0x7f3bac198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac198440 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c004770 tx=0x7f3b9c003c10 comp rx=0 tx=0).stop 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 shutdown_connections 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bac106f10 0x7f3bac19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bac102790 0x7f3bac198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b900a2d80 0x7f3b900a5240 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.444+0000 7f3bb2e00640 1 --2- 172.21.15.71:0/1191056374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bac101de0 0x7f3bac198440 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.445+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 >> 172.21.15.71:0/1191056374 conn(0x7f3bac0fd8c0 msgr2=0x7f3bac0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.445+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 shutdown_connections 2024-09-05T12:27:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:57.445+0000 7f3bb2e00640 1 -- 172.21.15.71:0/1191056374 wait complete. 2024-09-05T12:27:58.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:27:58.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:58 smithi123 ceph-mon[34321]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-05T12:27:58.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:27:58 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1191056374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:58 smithi174 ceph-mon[34313]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-05T12:27:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:27:58 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1191056374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:58.622 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:27:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:58 smithi071 ceph-mon[29845]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-09-05T12:27:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:27:58 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1191056374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:27:59.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 -- 172.21.15.71:0/3526376544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01019c0 msgr2=0x7f35f0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 --2- 172.21.15.71:0/3526376544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01019c0 0x7f35f0101e20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009930 tx=0x7f35d802a420 comp rx=0 tx=0).stop 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 -- 172.21.15.71:0/3526376544 shutdown_connections 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 --2- 172.21.15.71:0/3526376544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35f0102360 0x7f35f010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 --2- 172.21.15.71:0/3526376544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01019c0 0x7f35f0101e20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 --2- 172.21.15.71:0/3526376544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01079c0 0x7f35f0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 -- 172.21.15.71:0/3526376544 >> 172.21.15.71:0/3526376544 conn(0x7f35f00fd740 msgr2=0x7f35f00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 -- 172.21.15.71:0/3526376544 shutdown_connections 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.237+0000 7f35f4a00640 1 -- 172.21.15.71:0/3526376544 wait complete. 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 Processor -- start 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 -- start start 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01019c0 0x7f35f019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:59.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35f0102360 0x7f35f019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 0x7f35f019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0111e10 con 0x7f35f01079c0 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0111c90 con 0x7f35f01019c0 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.238+0000 7f35f4a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35f0111f90 con 0x7f35f0102360 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 0x7f35f019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 0x7f35f019ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41840/0 (socket says 172.21.15.71:41840) 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 -- 172.21.15.71:0/2209215112 learned_addr learned my addr 172.21.15.71:0/2209215112 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35e7e00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35f0102360 0x7f35f019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35ef400640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01019c0 0x7f35f019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 -- 172.21.15.71:0/2209215112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35f0102360 msgr2=0x7f35f019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35f0102360 0x7f35f019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 -- 172.21.15.71:0/2209215112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01019c0 msgr2=0x7f35f019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:59.270 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01019c0 0x7f35f019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35efe00640 1 -- 172.21.15.71:0/2209215112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f019f460 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.239+0000 7f35ef400640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01019c0 0x7f35f019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.240+0000 7f35efe00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 0x7f35f019ed80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35e800eb10 tx=0x7f35e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.240+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8004b20 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.240+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35e8004cc0 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.240+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8010760 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.240+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f019f750 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.240+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f01a7030 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.241+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f0101e20 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.243+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f35e8005020 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.244+0000 7f35ecc00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35d40a2d80 0x7f35d40a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.244+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f35e80cfff0 con 0x7f35f01079c0 2024-09-05T12:27:59.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.244+0000 7f35ef400640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35d40a2d80 0x7f35d40a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:27:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.245+0000 7f35ef400640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35d40a2d80 0x7f35d40a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009f40 tx=0x7f35e0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:27:59.272 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.248+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35e809c740 con 0x7f35f01079c0 2024-09-05T12:27:59.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.425+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35f01a72d0 con 0x7f35f01079c0 2024-09-05T12:27:59.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.427+0000 7f35ecc00640 1 -- 172.21.15.71:0/2209215112 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+25118 (secure 0 0 0) 0x7f35e809c040 con 0x7f35f01079c0 2024-09-05T12:27:59.428 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:27:59.428 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":95,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:27:44.498880+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/2607784636":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/4208428214":"2024-09-06T12:27:44.498834+0000","172.21.15.71:6801/2050574497":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/2050574497":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/3957197147":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/372010080":"2024-09-06T12:27:44.498834+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35d40a2d80 msgr2=0x7f35d40a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35d40a2d80 0x7f35d40a5240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009f40 tx=0x7f35e0005f90 comp rx=0 tx=0).stop 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 msgr2=0x7f35f019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 0x7f35f019ed80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f35e800eb10 tx=0x7f35e800c2d0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 shutdown_connections 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35d40a2d80 0x7f35d40a5240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f35f01079c0 0x7f35f019ed80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35f0102360 0x7f35f019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 --2- 172.21.15.71:0/2209215112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35f01019c0 0x7f35f019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:27:59.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 >> 172.21.15.71:0/2209215112 conn(0x7f35f00fd740 msgr2=0x7f35f010d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:27:59.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 shutdown_connections 2024-09-05T12:27:59.431 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:27:59.429+0000 7f35f4a00640 1 -- 172.21.15.71:0/2209215112 wait complete. 2024-09-05T12:28:00.392 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-05T12:28:00.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:00 smithi123 ceph-mon[34321]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-05T12:28:00.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:00 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2209215112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:28:00.441 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-mon[29845]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-05T12:28:00.441 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2209215112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:28:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:00 smithi174 ceph-mon[34313]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-05T12:28:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:00 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2209215112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:28:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 -- 172.21.15.71:0/3372002419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744102ae0 msgr2=0x7f6744102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 --2- 172.21.15.71:0/3372002419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744102ae0 0x7f6744102ec0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6734009ca0 tx=0x7f673402a950 comp rx=0 tx=0).stop 2024-09-05T12:28:00.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 -- 172.21.15.71:0/3372002419 shutdown_connections 2024-09-05T12:28:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 --2- 172.21.15.71:0/3372002419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f674410bf00 0x7f674410e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 --2- 172.21.15.71:0/3372002419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744103400 0x7f674410b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 --2- 172.21.15.71:0/3372002419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744102ae0 0x7f6744102ec0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.644+0000 7f6749800640 1 -- 172.21.15.71:0/3372002419 >> 172.21.15.71:0/3372002419 conn(0x7f67440777c0 msgr2=0x7f67440feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.645+0000 7f6749800640 1 -- 172.21.15.71:0/3372002419 shutdown_connections 2024-09-05T12:28:00.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.645+0000 7f6749800640 1 -- 172.21.15.71:0/3372002419 wait complete. 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.645+0000 7f6749800640 1 Processor -- start 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 -- start start 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 0x7f6744198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744103400 0x7f6744198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f674410bf00 0x7f674419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6744111d20 con 0x7f674410bf00 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6744111ba0 con 0x7f6744102ae0 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6749800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6744111ea0 con 0x7f6744103400 2024-09-05T12:28:00.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6743e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 0x7f6744198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6743e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 0x7f6744198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53492/0 (socket says 172.21.15.71:53492) 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6748e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f674410bf00 0x7f674419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f6743e00640 1 -- 172.21.15.71:0/4260844794 learned_addr learned my addr 172.21.15.71:0/4260844794 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.646+0000 7f673b400640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744103400 0x7f6744198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6743e00640 1 -- 172.21.15.71:0/4260844794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744103400 msgr2=0x7f6744198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6743e00640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744103400 0x7f6744198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6743e00640 1 -- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f674410bf00 msgr2=0x7f674419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6743e00640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f674410bf00 0x7f674419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6743e00640 1 -- 172.21.15.71:0/4260844794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674419d220 con 0x7f6744102ae0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6743e00640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 0x7f6744198230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6734005e30 tx=0x7f6734003ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6741600640 1 -- 172.21.15.71:0/4260844794 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734034040 con 0x7f6744102ae0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674419d4b0 con 0x7f6744102ae0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6741600640 1 -- 172.21.15.71:0/4260844794 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f673402fe20 con 0x7f6744102ae0 2024-09-05T12:28:00.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6741600640 1 -- 172.21.15.71:0/4260844794 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734004800 con 0x7f6744102ae0 2024-09-05T12:28:00.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.647+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67441a4da0 con 0x7f6744102ae0 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.650+0000 7f674dcce740 1 Processor -- start 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 -- start start 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f6741600640 1 -- 172.21.15.71:0/4260844794 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 138009+0+0 (secure 0 0 0) 0x7f6734037020 con 0x7f6744102ae0 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb820790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb82d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5602fb786100 con 0x5602fb8203b0 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5602fb72e1b0 con 0x5602fb82b570 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f674dcce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602fb7820c0 con 0x5602fb820cd0 2024-09-05T12:28:00.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f6743e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f6748e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb820790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f6743e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:38352/0 (socket says 172.21.15.71:38352) 2024-09-05T12:28:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f6743e00640 1 -- 172.21.15.71:0/1205250567 learned_addr learned my addr 172.21.15.71:0/1205250567 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:00.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.651+0000 7f673b400640 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb82d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f673b400640 1 -- 172.21.15.71:0/1205250567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 msgr2=0x5602fb82b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f673b400640 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f673b400640 1 -- 172.21.15.71:0/1205250567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 msgr2=0x5602fb820790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f673b400640 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb820790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f673b400640 1 -- 172.21.15.71:0/1205250567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602fb499210 con 0x5602fb82b570 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f6743e00640 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f673b400640 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb82d960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f672c005d00 tx=0x7f672c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcc83a3ec7b2862b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f6739600640 1 -- 172.21.15.71:0/1205250567 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c01c480 con 0x5602fb82b570 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f6739600640 1 -- 172.21.15.71:0/1205250567 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c01cb00 con 0x5602fb82b570 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f6739600640 1 -- 172.21.15.71:0/1205250567 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672c011c50 con 0x5602fb82b570 2024-09-05T12:28:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.652+0000 7f6741600640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67280a2d80 0x7f67280a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f6741600640 1 -- 172.21.15.71:0/4260844794 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f67340f0720 con 0x7f6744102ae0 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f6748e00640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67280a2d80 0x7f67280a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 -- 172.21.15.71:0/1205250567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 msgr2=0x5602fb82d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb82d960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f672c005d00 tx=0x7f672c0107a0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 -- 172.21.15.71:0/1205250567 shutdown_connections 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb82d960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 --2- 172.21.15.71:0/1205250567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb820790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 -- 172.21.15.71:0/1205250567 >> 172.21.15.71:0/1205250567 conn(0x5602fb81c920 msgr2=0x5602fb81cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 -- 172.21.15.71:0/1205250567 shutdown_connections 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f674dcce740 1 -- 172.21.15.71:0/1205250567 wait complete. 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.653+0000 7f6748e00640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67280a2d80 0x7f67280a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f673c009d20 tx=0x7f673c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 -- start start 2024-09-05T12:28:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 Processor -- start 2024-09-05T12:28:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb828500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb828a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb828f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5602fb82e050 con 0x5602fb82b570 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5602fb70f090 con 0x5602fb820cd0 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.654+0000 7f674dcce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602fb82e1d0 con 0x5602fb8203b0 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f674dcce740 10 client.0 _ll_register_callbacks cb 0x5602fb763f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6748e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb828f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f673b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb828a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb828500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f673b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb828a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53516/0 (socket says 172.21.15.71:53516) 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f673b400640 1 -- 172.21.15.71:0/532954247 learned_addr learned my addr 172.21.15.71:0/532954247 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 -- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 msgr2=0x5602fb828a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb828a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 -- 172.21.15.71:0/532954247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 msgr2=0x5602fb828f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb828f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 -- 172.21.15.71:0/532954247 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602fb565620 con 0x5602fb8203b0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f673b400640 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb828a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6743e00640 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb828500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6734000f80 tx=0x7f6734009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=996f0a12ac430b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6732000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.655+0000 7f6732000640 1 -- 172.21.15.71:0/532954247 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673402c040 con 0x5602fb8203b0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672c01cb00 con 0x5602fb8203b0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f6732000640 1 -- 172.21.15.71:0/532954247 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f67340f0320 con 0x5602fb8203b0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f6732000640 1 -- 172.21.15.71:0/532954247 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67340bc680 con 0x5602fb8203b0 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 10 client.25357 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602fb82e0b0 con 0x5602fb8203b0 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f6732000640 1 -- 172.21.15.71:0/532954247 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672c01cb00 con 0x5602fb8203b0 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f6732000640 1 -- 172.21.15.71:0/532954247 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f67340edcc0 con 0x5602fb8203b0 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 10 client.25357 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 10 client.25357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 20 client.25357 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb83b300 0x5602fb83d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 4 client.25357 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.656+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602fb793830 con 0x5602fb83b300 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.657+0000 7f673b400640 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb83b300 0x5602fb83d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.657+0000 7f673b400640 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb83b300 0x5602fb83d770 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbfb0c984e078230 server_cookie=2acb0d1dc6689238 in_seq=0 out_seq=0 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.657+0000 7f6732000640 10 client.25357 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f6732000640 1 -- 172.21.15.71:0/532954247 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5602fb793830 con 0x5602fb83b300 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f6732000640 10 client.25357 handle_command_reply: tid=0 2024-09-05T12:28:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 client.25357 shutdown 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 msgr2=0x5602fb828500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb828500 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6734000f80 tx=0x7f6734009680 comp rx=0 tx=0).stop 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 shutdown_connections 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb83b300 0x5602fb83d770 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb828f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f6732000640 0 client.25357 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb828a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 --2- 172.21.15.71:0/532954247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb828500 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 >> 172.21.15.71:0/532954247 conn(0x5602fb81c920 msgr2=0x5602fb822680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 shutdown_connections 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 1 -- 172.21.15.71:0/532954247 wait complete. 2024-09-05T12:28:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.658+0000 7f674dcce740 20 client.25357 trim_cache size 0 max 16384 2024-09-05T12:28:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 Processor -- start 2024-09-05T12:28:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 -- start start 2024-09-05T12:28:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb81b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb843080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb8435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f67441aec80 con 0x5602fb82b570 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67441b07c0 con 0x5602fb820cd0 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.675+0000 7f674dcce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67441af350 con 0x5602fb8203b0 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb81b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb81b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:38368/0 (socket says 172.21.15.71:38368) 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 -- 172.21.15.71:0/542949053 learned_addr learned my addr 172.21.15.71:0/542949053 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6748e00640 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb8435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f673b400640 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb843080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 -- 172.21.15.71:0/542949053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 msgr2=0x5602fb843080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb843080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 -- 172.21.15.71:0/542949053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 msgr2=0x5602fb8435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb8435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 -- 172.21.15.71:0/542949053 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602fb8201d0 con 0x5602fb8203b0 2024-09-05T12:28:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.676+0000 7f6743e00640 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb81b8b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67340bc840 tx=0x7f673402a950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1768c3e16088713 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f6738c00640 1 -- 172.21.15.71:0/542949053 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673402c040 con 0x5602fb8203b0 2024-09-05T12:28:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f6738c00640 1 -- 172.21.15.71:0/542949053 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67340d9070 con 0x5602fb8203b0 2024-09-05T12:28:00.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f6738c00640 1 -- 172.21.15.71:0/542949053 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67340f0520 con 0x5602fb8203b0 2024-09-05T12:28:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 -- 172.21.15.71:0/542949053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 msgr2=0x5602fb81b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb81b8b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67340bc840 tx=0x7f673402a950 comp rx=0 tx=0).stop 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 -- 172.21.15.71:0/542949053 shutdown_connections 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb82b570 0x5602fb8435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb820cd0 0x5602fb843080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 --2- 172.21.15.71:0/542949053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb8203b0 0x5602fb81b8b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.677+0000 7f674dcce740 1 -- 172.21.15.71:0/542949053 >> 172.21.15.71:0/542949053 conn(0x5602fb81c920 msgr2=0x5602fb81b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 -- 172.21.15.71:0/542949053 shutdown_connections 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 -- 172.21.15.71:0/542949053 wait complete. 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 -- start start 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 Processor -- start 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb82eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb8a4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f67441b00f0 con 0x5602fb8203b0 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f67441afa20 con 0x5602fb82b570 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.678+0000 7f674dcce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67441acc60 con 0x5602fb820cd0 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f674dcce740 10 client.0 _ll_register_callbacks cb 0x7f67340f6220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb82eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb82eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41902/0 (socket says 172.21.15.71:41902) 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 -- 172.21.15.71:0/1323585501 learned_addr learned my addr 172.21.15.71:0/1323585501 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6748e00640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb8a4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f673b400640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 -- 172.21.15.71:0/1323585501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 msgr2=0x5602fb82eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 -- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 msgr2=0x5602fb8a4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb8a4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 -- 172.21.15.71:0/1323585501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602fb8a4580 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6743e00640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb82eab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f67340c4f80 tx=0x7f6734004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d3cc8904e70af2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.679+0000 7f6732000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f6732000640 1 -- 172.21.15.71:0/1323585501 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673402c040 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f6732000640 1 -- 172.21.15.71:0/1323585501 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f67340a7330 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67340d9070 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f6732000640 1 -- 172.21.15.71:0/1323585501 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f67340bc210 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 10 client.15309 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5602fb8a57f0 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f6732000640 1 -- 172.21.15.71:0/1323585501 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67340d9070 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f6732000640 1 -- 172.21.15.71:0/1323585501 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f67340bdb70 con 0x5602fb8203b0 2024-09-05T12:28:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 10 client.15309 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 10 client.15309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 20 client.15309 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb8a5cc0 0x5602fb8a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 4 client.15309 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.680+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602fb78db80 con 0x5602fb8a5cc0 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.681+0000 7f673b400640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb8a5cc0 0x5602fb8a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.681+0000 7f673b400640 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb8a5cc0 0x5602fb8a80b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832ab775463c38e8 server_cookie=e411cec089f60139 in_seq=0 out_seq=0 2024-09-05T12:28:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:00.681+0000 7f6732000640 10 client.15309 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:00.794 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:00.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:28:00.475+0000 7f0a6ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:28:00.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:28:00.475+0000 7f0a6ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:28:00.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:28:00.475+0000 7f0a6ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:28:00.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:28:00.475+0000 7f0a6ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:28:00.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:28:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:28:00.475+0000 7f0a6ae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-05T12:28:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.185+0000 7f6741600640 1 -- 172.21.15.71:0/4260844794 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f67340c5700 con 0x7f6744102ae0 2024-09-05T12:28:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f6732000640 1 -- 172.21.15.71:0/1323585501 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5602fb78db80 con 0x5602fb8a5cc0 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f6732000640 10 client.15309 handle_command_reply: tid=0 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 client.15309 shutdown 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 msgr2=0x5602fb82eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb82eab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f67340c4f80 tx=0x7f6734004270 comp rx=0 tx=0).stop 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 shutdown_connections 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5602fb8a5cc0 0x5602fb8a80b0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602fb82b570 0x5602fb8a4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602fb820cd0 0x5602fb82eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f6732000640 0 client.15309 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 --2- 172.21.15.71:0/1323585501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5602fb8203b0 0x5602fb82eab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 >> 172.21.15.71:0/1323585501 conn(0x5602fb81c920 msgr2=0x5602fb828ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 shutdown_connections 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.941+0000 7f674dcce740 1 -- 172.21.15.71:0/1323585501 wait complete. 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f674dcce740 20 client.15309 trim_cache size 0 max 16384 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:28:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67280a2d80 msgr2=0x7f67280a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67280a2d80 0x7f67280a5240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f673c009d20 tx=0x7f673c005f90 comp rx=0 tx=0).stop 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 msgr2=0x7f6744198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 0x7f6744198230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6734005e30 tx=0x7f6734003ce0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 shutdown_connections 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67280a2d80 0x7f67280a5240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f674410bf00 0x7f674419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6744103400 0x7f6744198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 --2- 172.21.15.71:0/4260844794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6744102ae0 0x7f6744198230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 >> 172.21.15.71:0/4260844794 conn(0x7f67440777c0 msgr2=0x7f67440fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 shutdown_connections 2024-09-05T12:28:01.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:01.942+0000 7f6749800640 1 -- 172.21.15.71:0/4260844794 wait complete. 2024-09-05T12:28:02.253 INFO:teuthology.orchestra.run.smithi071.stdout:{"run_dir":"/var/run/ceph"} 2024-09-05T12:28:02.254 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-05T12:28:02.254 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 8220 2024-09-05T12:28:02.255 DEBUG:tasks.quiescer:Yielding 2024-09-05T12:28:02.255 INFO:teuthology.run_tasks:Running task workunit... 2024-09-05T12:28:02.278 INFO:tasks.workunit:Pulling workunits from ref e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:28:02.278 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-05T12:28:02.278 INFO:tasks.workunit:timeout=3h 2024-09-05T12:28:02.279 INFO:tasks.workunit:cleanup=True 2024-09-05T12:28:02.279 DEBUG:teuthology.orchestra.run.smithi071:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-05T12:28:02.280 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-09-05 12:25:49.379112303 +0000 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-09-05 12:25:49.379112303 +0000 2024-09-05T12:28:02.317 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-09-05 12:27:43.295779309 +0000 2024-09-05T12:28:02.318 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-09-05 12:25:49.379112303 +0000 2024-09-05T12:28:02.318 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-05T12:28:02.318 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-05T12:28:02.427 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e856d6b2462509a377ef39149a05475ed006ebac 2024-09-05T12:28:02.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:02 smithi123 ceph-mon[34321]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-05T12:28:02.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:02 smithi123 ceph-mon[34321]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-09-05T12:28:02.465 INFO:tasks.workunit.client.0.smithi071.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-05T12:28:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:02 smithi174 ceph-mon[34313]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-05T12:28:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:02 smithi174 ceph-mon[34313]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-09-05T12:28:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:02 smithi071 ceph-mon[29845]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 49 op/s 2024-09-05T12:28:02.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:02 smithi071 ceph-mon[29845]: mgrmap e25: x(active, since 16s), standbys: z, y 2024-09-05T12:28:02.667 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:03.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 -- 172.21.15.71:0/2699402066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310109120 msgr2=0x7fb310110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:03.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 --2- 172.21.15.71:0/2699402066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310109120 0x7fb310110ca0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb308009ca0 tx=0x7fb30802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 -- 172.21.15.71:0/2699402066 shutdown_connections 2024-09-05T12:28:03.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 --2- 172.21.15.71:0/2699402066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310109120 0x7fb310110ca0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 --2- 172.21.15.71:0/2699402066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3101049a0 0x7fb3101089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 --2- 172.21.15.71:0/2699402066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb310103ff0 0x7fb3101043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 -- 172.21.15.71:0/2699402066 >> 172.21.15.71:0/2699402066 conn(0x7fb3100ffa90 msgr2=0x7fb310101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 -- 172.21.15.71:0/2699402066 shutdown_connections 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.272+0000 7fb314a00640 1 -- 172.21.15.71:0/2699402066 wait complete. 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.273+0000 7fb314a00640 1 Processor -- start 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 -- start start 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 0x7fb31019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3101049a0 0x7fb31019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb310109120 0x7fb31019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb310114180 con 0x7fb310103ff0 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb310114000 con 0x7fb3101049a0 2024-09-05T12:28:03.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb314a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb310114300 con 0x7fb310109120 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.274+0000 7fb30de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 0x7fb31019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3101049a0 0x7fb31019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 0x7fb31019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33680/0 (socket says 172.21.15.71:33680) 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30de00640 1 -- 172.21.15.71:0/1126636707 learned_addr learned my addr 172.21.15.71:0/1126636707 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30e800640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb310109120 0x7fb31019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 -- 172.21.15.71:0/1126636707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb310109120 msgr2=0x7fb31019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb310109120 0x7fb31019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 -- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 msgr2=0x7fb31019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 0x7fb31019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 -- 172.21.15.71:0/1126636707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31019f570 con 0x7fb3101049a0 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30e800640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb310109120 0x7fb31019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30de00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 0x7fb31019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:03.307 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.275+0000 7fb30d400640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3101049a0 0x7fb31019ab00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00e9f0 tx=0x7fb2fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.276+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc00ccc0 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.276+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2fc010910 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.276+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31019f860 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.276+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc018960 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.276+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3101a7140 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.277+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3101043d0 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.279+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fb2fc010ac0 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.280+0000 7fb306a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb2f40a2e90 0x7fb2f40a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.281+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb2fc0d0030 con 0x7fb3101049a0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.281+0000 7fb30de00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb2f40a2e90 0x7fb2f40a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.281+0000 7fb30de00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb2f40a2e90 0x7fb2f40a5350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb300011550 tx=0x7fb300005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:03.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.283+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2fc09c6c0 con 0x7fb3101049a0 2024-09-05T12:28:03.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.493+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb31019b920 con 0x7fb3101049a0 2024-09-05T12:28:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.494+0000 7fb306a00640 1 -- 172.21.15.71:0/1126636707 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11968 (secure 0 0 0) 0x7fb31019b920 con 0x7fb3101049a0 2024-09-05T12:28:03.495 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:28:03.495 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-05T12:27:44:191847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:27:44.168192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:28:03.495 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-05T12:28:03.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.496+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb2f40a2e90 msgr2=0x7fb2f40a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:03.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.496+0000 7fb314a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb2f40a2e90 0x7fb2f40a5350 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb300011550 tx=0x7fb300005f90 comp rx=0 tx=0).stop 2024-09-05T12:28:03.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.496+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3101049a0 msgr2=0x7fb31019ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.496+0000 7fb314a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3101049a0 0x7fb31019ab00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc00e9f0 tx=0x7fb2fc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 shutdown_connections 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb310109120 0x7fb31019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb3101049a0 0x7fb31019ab00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb2f40a2e90 0x7fb2f40a5350 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 --2- 172.21.15.71:0/1126636707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb310103ff0 0x7fb31019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 >> 172.21.15.71:0/1126636707 conn(0x7fb3100ffa90 msgr2=0x7fb3101004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 shutdown_connections 2024-09-05T12:28:03.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:03.497+0000 7fb314a00640 1 -- 172.21.15.71:0/1126636707 wait complete. 2024-09-05T12:28:04.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-05T12:24:49.631865+0000', 'modified': '2024-09-05T12:27:44.168192+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24514, 'mds_2': 24502, 'mds_3': 24490, 'mds_4': 24485}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14469': {'gid': 14469, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6841/991266062', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6840', 'nonce': 991266062}, {'type': 'v1', 'addr': '172.21.15.71:6841', 'nonce': 991266062}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/679053656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 679053656}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 679053656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/1436525377', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1436525377}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1436525377}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6835/2774200097', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 2774200097}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 2774200097}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24478': {'gid': 24478, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/626911382', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 626911382}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 626911382}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.123:6837/3417870839', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 3417870839}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 3417870839}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24490': {'gid': 24490, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 20, 'addr': '172.21.15.174:6837/870087433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 870087433}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 870087433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.174:6839/584492863', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 584492863}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 584492863}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'g', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.71:6839/2593934735', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 2593934735}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 2593934735}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.123:6839/782979893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 782979893}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 782979893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24514, 24502, 24490, 24524, 24485]} 2024-09-05T12:28:04.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-05T12:28:04.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.h status 2024-09-05T12:28:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:04 smithi174 ceph-mon[34313]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 64 op/s 2024-09-05T12:28:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:04 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1126636707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:04.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:04 smithi071 ceph-mon[29845]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 64 op/s 2024-09-05T12:28:04.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:04 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1126636707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:04 smithi123 ceph-mon[34321]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 99 KiB/s wr, 64 op/s 2024-09-05T12:28:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:04 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1126636707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:04.833 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:05.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 -- 172.21.15.71:0/3627987215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 msgr2=0x7f2c10107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 --2- 172.21.15.71:0/3627987215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c10107da0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009930 tx=0x7f2c0002a420 comp rx=0 tx=0).stop 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 -- 172.21.15.71:0/3627987215 shutdown_connections 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 --2- 172.21.15.71:0/3627987215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 0x7f2c1010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 --2- 172.21.15.71:0/3627987215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 --2- 172.21.15.71:0/3627987215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c10107da0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 -- 172.21.15.71:0/3627987215 >> 172.21.15.71:0/3627987215 conn(0x7f2c100fd740 msgr2=0x7f2c100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 -- 172.21.15.71:0/3627987215 shutdown_connections 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.494+0000 7f2c15600640 1 -- 172.21.15.71:0/3627987215 wait complete. 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.495+0000 7f2c15600640 1 Processor -- start 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.495+0000 7f2c15600640 1 -- start start 2024-09-05T12:28:05.525 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c15600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c15600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 0x7f2c10198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c15600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c1019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c15600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10111e10 con 0x7f2c101079c0 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c15600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10111c90 con 0x7f2c10102360 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c15600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c10111f90 con 0x7f2c101019c0 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c14c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c1019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c14c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c1019cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33686/0 (socket says 172.21.15.71:33686) 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35238/0 (socket says 172.21.15.71:35238) 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c14c00640 1 -- 172.21.15.71:0/2572422874 learned_addr learned my addr 172.21.15.71:0/2572422874 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.496+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 0x7f2c10198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c14c00640 1 -- 172.21.15.71:0/2572422874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 msgr2=0x7f2c10198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c14c00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c14c00640 1 -- 172.21.15.71:0/2572422874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 msgr2=0x7f2c10198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c14c00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 0x7f2c10198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c14c00640 1 -- 172.21.15.71:0/2572422874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c1019d2c0 con 0x7f2c101079c0 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 0x7f2c10198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c14c00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c1019cbe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800eb10 tx=0x7f2c0800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c0d600640 1 -- 172.21.15.71:0/2572422874 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08004b20 con 0x7f2c101079c0 2024-09-05T12:28:05.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c1019d5b0 con 0x7f2c101079c0 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c0d600640 1 -- 172.21.15.71:0/2572422874 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c08004cc0 con 0x7f2c101079c0 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c0d600640 1 -- 172.21.15.71:0/2572422874 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08010760 con 0x7f2c101079c0 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.497+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c101a4e90 con 0x7f2c101079c0 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.500+0000 7f2c0d600640 1 -- 172.21.15.71:0/2572422874 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f2c08005020 con 0x7f2c101079c0 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 Processor -- start 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 -- start start 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd08a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 0x5579dd095180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd097ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579dcff00b0 con 0x5579dd08a4e0 2024-09-05T12:28:05.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579dcf97ab0 con 0x5579dd08ae00 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c195f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579dcfec070 con 0x5579dd0956c0 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd08a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 0x5579dd095180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 0x5579dd095180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36546/0 (socket says 172.21.15.71:36546) 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.501+0000 7f2c0fe00640 1 -- 172.21.15.71:0/1303065929 learned_addr learned my addr 172.21.15.71:0/1303065929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c07e00640 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd097ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0fe00640 1 -- 172.21.15.71:0/1303065929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 msgr2=0x5579dd097ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd097ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0fe00640 1 -- 172.21.15.71:0/1303065929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 msgr2=0x5579dd08a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd08a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0fe00640 1 -- 172.21.15.71:0/1303065929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579dcfa44a0 con 0x5579dd08ae00 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c07e00640 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd08a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:05.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 0x5579dd095180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009930 tx=0x7f2c00000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a06b865d64f96f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0d600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bf40a2e10 0x7f2bf40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c0d600640 1 -- 172.21.15.71:0/2572422874 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f2c080cffa0 con 0x7f2c101079c0 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c05200640 1 -- 172.21.15.71:0/1303065929 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00035040 con 0x5579dd08ae00 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c05200640 1 -- 172.21.15.71:0/1303065929 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c00004760 con 0x5579dd08ae00 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c05200640 1 -- 172.21.15.71:0/1303065929 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00003970 con 0x5579dd08ae00 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.502+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bf40a2e10 0x7f2bf40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 -- 172.21.15.71:0/1303065929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 msgr2=0x5579dd095180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 0x5579dd095180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009930 tx=0x7f2c00000bc0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bf40a2e10 0x7f2bf40a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009fd0 tx=0x7f2bf8005850 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 -- 172.21.15.71:0/1303065929 shutdown_connections 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd097ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08ae00 0x5579dd095180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 --2- 172.21.15.71:0/1303065929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd08a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 -- 172.21.15.71:0/1303065929 >> 172.21.15.71:0/1303065929 conn(0x5579dd086a70 msgr2=0x5579dd086e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 -- 172.21.15.71:0/1303065929 shutdown_connections 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.503+0000 7f2c195f5740 1 -- 172.21.15.71:0/1303065929 wait complete. 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 -- start start 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 Processor -- start 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd099bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 0x5579dd09e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0982d0 con 0x5579dd0956c0 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579dcf78920 con 0x5579dd08a4e0 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579dd098450 con 0x5579dd08ae00 2024-09-05T12:28:05.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.504+0000 7f2c195f5740 10 client.0 _ll_register_callbacks cb 0x5579dcf9c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 0x5579dd09e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd099bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c07e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35258/0 (socket says 172.21.15.71:35258) 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd099bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36550/0 (socket says 172.21.15.71:36550) 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c0fe00640 1 -- 172.21.15.71:0/2841050846 learned_addr learned my addr 172.21.15.71:0/2841050846 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 0x5579dd09e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33710/0 (socket says 172.21.15.71:33710) 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 -- 172.21.15.71:0/2841050846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 msgr2=0x5579dd09a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 -- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 msgr2=0x5579dd099bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd099bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 -- 172.21.15.71:0/2841050846 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579dcd612b0 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2c14c00640 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 0x5579dd09e260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c080cd1d0 tx=0x7f2c0809bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f635d0eab6e23d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.505+0000 7f2bfd600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2bfd600640 1 -- 172.21.15.71:0/2841050846 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080a0280 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2bfd600640 1 -- 172.21.15.71:0/2841050846 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f2c080cd4b0 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c00004760 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2bfd600640 1 -- 172.21.15.71:0/2841050846 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c080a4b40 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2c195f5740 10 client.15327 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579dd0a8360 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2bfd600640 1 -- 172.21.15.71:0/2841050846 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c080b8070 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.506+0000 7f2bfd600640 1 -- 172.21.15.71:0/2841050846 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2c0809b320 con 0x5579dd0956c0 2024-09-05T12:28:05.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c195f5740 10 client.15327 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c195f5740 10 client.15327 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c195f5740 20 client.15327 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd0aa980 0x5579dd0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c195f5740 4 client.15327 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 --> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579dcffd620 con 0x5579dd0aa980 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.507+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd0aa980 0x5579dd0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.508+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd0aa980 0x5579dd0abdf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=baf858952120e8b1 server_cookie=2b5de89b972d7f3b in_seq=0 out_seq=0 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.508+0000 7f2bfd600640 10 client.15327 ms_handle_connect on v2:172.21.15.123:6836/3417870839 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2bfd600640 1 -- 172.21.15.71:0/2841050846 <== mds.4 v2:172.21.15.123:6836/3417870839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579dcffd620 con 0x5579dd0aa980 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2bfd600640 10 client.15327 handle_command_reply: tid=0 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 client.15327 shutdown 2024-09-05T12:28:05.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 msgr2=0x5579dd09e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 0x5579dd09e260 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2c080cd1d0 tx=0x7f2c0809bad0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 shutdown_connections 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd0aa980 0x5579dd0abdf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd0956c0 0x5579dd09e260 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2841050846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd099bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2bfd600640 0 client.15327 ms_handle_reset on v2:172.21.15.123:6836/3417870839 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 >> 172.21.15.71:0/2841050846 conn(0x5579dd086a70 msgr2=0x5579dd08c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 shutdown_connections 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 1 -- 172.21.15.71:0/2841050846 wait complete. 2024-09-05T12:28:05.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.509+0000 7f2c195f5740 20 client.15327 trim_cache size 0 max 16384 2024-09-05T12:28:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.532+0000 7f2c195f5740 1 Processor -- start 2024-09-05T12:28:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.532+0000 7f2c195f5740 1 -- start start 2024-09-05T12:28:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd0abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd0956c0 0x5579dd091c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c195f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0aa130 con 0x5579dd08a4e0 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c195f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0aa800 con 0x5579dd0956c0 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c195f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0aaed0 con 0x5579dd08ae00 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd0abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd0abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33726/0 (socket says 172.21.15.71:33726) 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c0fe00640 1 -- 172.21.15.71:0/3382425211 learned_addr learned my addr 172.21.15.71:0/3382425211 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c14c00640 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd0956c0 0x5579dd091c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.533+0000 7f2c07e00640 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c14c00640 1 -- 172.21.15.71:0/3382425211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 msgr2=0x5579dd09e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c14c00640 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c14c00640 1 -- 172.21.15.71:0/3382425211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 msgr2=0x5579dd0abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c14c00640 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd0abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c14c00640 1 -- 172.21.15.71:0/3382425211 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579dd0868a0 con 0x5579dd0956c0 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd0abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.534+0000 7f2c14c00640 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd0956c0 0x5579dd091c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2c0809bbd0 tx=0x7f2c08004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a3a3db16282b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.535+0000 7f2bffe00640 1 -- 172.21.15.71:0/3382425211 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080a0280 con 0x5579dd0956c0 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.535+0000 7f2bffe00640 1 -- 172.21.15.71:0/3382425211 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c080b8070 con 0x5579dd0956c0 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.535+0000 7f2bffe00640 1 -- 172.21.15.71:0/3382425211 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c080cd4b0 con 0x5579dd0956c0 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 -- 172.21.15.71:0/3382425211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd0956c0 msgr2=0x5579dd091c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd0956c0 0x5579dd091c70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2c0809bbd0 tx=0x7f2c08004ab0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 -- 172.21.15.71:0/3382425211 shutdown_connections 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd0956c0 0x5579dd091c70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd08ae00 0x5579dd09e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 --2- 172.21.15.71:0/3382425211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08a4e0 0x5579dd0abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 -- 172.21.15.71:0/3382425211 >> 172.21.15.71:0/3382425211 conn(0x5579dd086a70 msgr2=0x5579dd099bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 -- 172.21.15.71:0/3382425211 shutdown_connections 2024-09-05T12:28:05.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 -- 172.21.15.71:0/3382425211 wait complete. 2024-09-05T12:28:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 -- start start 2024-09-05T12:28:05.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.536+0000 7f2c195f5740 1 Processor -- start 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd0a0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08ae00 0x5579dd0a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd0a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0abc70 con 0x5579dd08ae00 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0ab5a0 con 0x5579dd08a4e0 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579dd0ac340 con 0x5579dd0956c0 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c195f5740 10 client.0 _ll_register_callbacks cb 0x7f2c08098170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c14c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd0a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c0fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd0a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c14c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd0a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35280/0 (socket says 172.21.15.71:35280) 2024-09-05T12:28:05.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c14c00640 1 -- 172.21.15.71:0/2199406409 learned_addr learned my addr 172.21.15.71:0/2199406409 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.537+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08ae00 0x5579dd0a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c0fe00640 1 -- 172.21.15.71:0/2199406409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 msgr2=0x5579dd0a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd0a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c0fe00640 1 -- 172.21.15.71:0/2199406409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08ae00 msgr2=0x5579dd0a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08ae00 0x5579dd0a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c0fe00640 1 -- 172.21.15.71:0/2199406409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579dd091760 con 0x5579dd08a4e0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c0fe00640 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd0a0680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5579dd09e7f0 tx=0x7f2c000035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70f9f2b28f733b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2bfd600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2bfd600640 1 -- 172.21.15.71:0/2199406409 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00035050 con 0x5579dd08a4e0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.538+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c080b8070 con 0x5579dd08a4e0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2bfd600640 1 -- 172.21.15.71:0/2199406409 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f2c000053e0 con 0x5579dd08a4e0 2024-09-05T12:28:05.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2bfd600640 1 -- 172.21.15.71:0/2199406409 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2c0005e050 con 0x5579dd08a4e0 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2c195f5740 10 client.25385 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5579dd1157b0 con 0x5579dd08a4e0 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2bfd600640 1 -- 172.21.15.71:0/2199406409 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c080b8070 con 0x5579dd08a4e0 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2bfd600640 1 -- 172.21.15.71:0/2199406409 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2c00031640 con 0x5579dd08a4e0 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.539+0000 7f2c195f5740 10 client.25385 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.540+0000 7f2c195f5740 10 client.25385 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.540+0000 7f2c195f5740 20 client.25385 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.540+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd119f80 0x5579dd118c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.540+0000 7f2c195f5740 4 client.25385 mds_command: new command op to 24485 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.540+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 --> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] -- command(tid 0: {"prefix": "status"}) -- 0x5579dcff7b30 con 0x5579dd119f80 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.540+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd119f80 0x5579dd118c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2bfd600640 10 client.25385 ms_handle_connect on v2:172.21.15.123:6836/3417870839 2024-09-05T12:28:05.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c07e00640 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd119f80 0x5579dd118c90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e34c26ebabf17ddc server_cookie=b3877beab995d846 in_seq=0 out_seq=0 2024-09-05T12:28:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2bfd600640 1 -- 172.21.15.71:0/2199406409 <== mds.4 v2:172.21.15.123:6836/3417870839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5579dcff7b30 con 0x5579dd119f80 2024-09-05T12:28:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2bfd600640 10 client.25385 handle_command_reply: tid=0 2024-09-05T12:28:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 client.25385 shutdown 2024-09-05T12:28:05.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 msgr2=0x5579dd0a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd0a0680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x5579dd09e7f0 tx=0x7f2c000035f0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 shutdown_connections 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:6836/3417870839,v1:172.21.15.123:6837/3417870839] conn(0x5579dd119f80 0x5579dd118c90 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579dd0956c0 0x5579dd0a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2bfd600640 0 client.25385 ms_handle_reset on v2:172.21.15.123:6836/3417870839 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579dd08ae00 0x5579dd0a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 --2- 172.21.15.71:0/2199406409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5579dd08a4e0 0x5579dd0a0680 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.541+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 >> 172.21.15.71:0/2199406409 conn(0x5579dd086a70 msgr2=0x5579dd091c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 shutdown_connections 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c195f5740 1 -- 172.21.15.71:0/2199406409 wait complete. 2024-09-05T12:28:05.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c195f5740 20 client.25385 trim_cache size 0 max 16384 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 4, 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24485, 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 147.228301063, 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 30, 2024-09-05T12:28:05.548 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 95, 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 221.43978198600001 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bf40a2e10 msgr2=0x7f2bf40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c15600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bf40a2e10 0x7f2bf40a52d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009fd0 tx=0x7f2bf8005850 comp rx=0 tx=0).stop 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 msgr2=0x7f2c1019cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:05.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.542+0000 7f2c15600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c1019cbe0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800eb10 tx=0x7f2c0800c2d0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 shutdown_connections 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bf40a2e10 0x7f2bf40a52d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2c101079c0 0x7f2c1019cbe0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c10102360 0x7f2c10198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 --2- 172.21.15.71:0/2572422874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c101019c0 0x7f2c10198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 >> 172.21.15.71:0/2572422874 conn(0x7f2c100fd740 msgr2=0x7f2c100fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 shutdown_connections 2024-09-05T12:28:05.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:05.543+0000 7f2c15600640 1 -- 172.21.15.71:0/2572422874 wait complete. 2024-09-05T12:28:06.298 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.c status 2024-09-05T12:28:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:06 smithi174 ceph-mon[34313]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-05T12:28:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:06 smithi071 ceph-mon[29845]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-05T12:28:06.669 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:06 smithi123 ceph-mon[34321]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-09-05T12:28:07.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 -- 172.21.15.71:0/4161087881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102da0 msgr2=0x7fecb8103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 --2- 172.21.15.71:0/4161087881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102da0 0x7fecb8103180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fecac009ca0 tx=0x7fecac02a950 comp rx=0 tx=0).stop 2024-09-05T12:28:07.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 -- 172.21.15.71:0/4161087881 shutdown_connections 2024-09-05T12:28:07.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 --2- 172.21.15.71:0/4161087881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecb810c180 0x7fecb810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 --2- 172.21.15.71:0/4161087881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecb81036c0 0x7fecb810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 --2- 172.21.15.71:0/4161087881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102da0 0x7fecb8103180 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 -- 172.21.15.71:0/4161087881 >> 172.21.15.71:0/4161087881 conn(0x7fecb80fc7a0 msgr2=0x7fecb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:07.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 -- 172.21.15.71:0/4161087881 shutdown_connections 2024-09-05T12:28:07.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.235+0000 7fecc0200640 1 -- 172.21.15.71:0/4161087881 wait complete. 2024-09-05T12:28:07.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.236+0000 7fecc0200640 1 Processor -- start 2024-09-05T12:28:07.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.236+0000 7fecc0200640 1 -- start start 2024-09-05T12:28:07.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecc0200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecb8102da0 0x7fecb819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecc0200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecb81036c0 0x7fecb819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecc0200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 0x7fecb819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecc0200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8111fb0 con 0x7fecb810c180 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecc0200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8111e30 con 0x7fecb8102da0 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecc0200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecb8112130 con 0x7fecb81036c0 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 0x7fecb819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecb81036c0 0x7fecb819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 0x7fecb819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33756/0 (socket says 172.21.15.71:33756) 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 -- 172.21.15.71:0/1060089059 learned_addr learned my addr 172.21.15.71:0/1060089059 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbd800640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecb8102da0 0x7fecb819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 -- 172.21.15.71:0/1060089059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecb81036c0 msgr2=0x7fecb819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecb81036c0 0x7fecb819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 -- 172.21.15.71:0/1060089059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecb8102da0 msgr2=0x7fecb819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecb8102da0 0x7fecb819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.237+0000 7fecbe200640 1 -- 172.21.15.71:0/1060089059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb819f5d0 con 0x7fecb810c180 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.238+0000 7fecbe200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 0x7fecb819eef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e970 tx=0x7fecb000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.238+0000 7fecb6000640 1 -- 172.21.15.71:0/1060089059 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000ccc0 con 0x7fecb810c180 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.238+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb819f8c0 con 0x7fecb810c180 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.238+0000 7fecb6000640 1 -- 172.21.15.71:0/1060089059 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb000ce60 con 0x7fecb810c180 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.238+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb81a71a0 con 0x7fecb810c180 2024-09-05T12:28:07.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.238+0000 7fecb6000640 1 -- 172.21.15.71:0/1060089059 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0018630 con 0x7fecb810c180 2024-09-05T12:28:07.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.241+0000 7fecc2e5e740 1 Processor -- start 2024-09-05T12:28:07.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.241+0000 7fecb6000640 1 -- 172.21.15.71:0/1060089059 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fecb00187d0 con 0x7fecb810c180 2024-09-05T12:28:07.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 -- start start 2024-09-05T12:28:07.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d8c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 0x55f606d96fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d97510 0x55f606d99900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f606cf20b0 con 0x55f606d8cc50 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f606c99ab0 con 0x55f606d97510 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f606cee070 con 0x55f606d8c330 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecbd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 0x55f606d96fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecbd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 0x55f606d96fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33766/0 (socket says 172.21.15.71:33766) 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecbce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d97510 0x55f606d99900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.242+0000 7fecbd800640 1 -- 172.21.15.71:0/3007895471 learned_addr learned my addr 172.21.15.71:0/3007895471 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbce00640 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d8c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbd800640 1 -- 172.21.15.71:0/3007895471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 msgr2=0x55f606d8c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbd800640 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d8c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbd800640 1 -- 172.21.15.71:0/3007895471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d97510 msgr2=0x55f606d99900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbd800640 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d97510 0x55f606d99900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbd800640 1 -- 172.21.15.71:0/3007895471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f606ca64a0 con 0x55f606d8cc50 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecb6000640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feca00a2e10 0x7feca00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbd800640 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 0x55f606d96fd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fecac02ae70 tx=0x7fecac02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb37a015150213f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecb6000640 1 -- 172.21.15.71:0/1060089059 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fecb00cff30 con 0x7fecb810c180 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.243+0000 7fecbce00640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feca00a2e10 0x7feca00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.244+0000 7fecaaa00640 1 -- 172.21.15.71:0/3007895471 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac048070 con 0x55f606d8cc50 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.244+0000 7fecaaa00640 1 -- 172.21.15.71:0/3007895471 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecac003e50 con 0x55f606d8cc50 2024-09-05T12:28:07.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.244+0000 7fecaaa00640 1 -- 172.21.15.71:0/3007895471 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac034040 con 0x55f606d8cc50 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3007895471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 msgr2=0x55f606d96fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 0x55f606d96fd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fecac02ae70 tx=0x7fecac02f450 comp rx=0 tx=0).stop 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecbce00640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feca00a2e10 0x7feca00a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55f606d89c40 tx=0x7feca4005c90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3007895471 shutdown_connections 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d97510 0x55f606d99900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8cc50 0x55f606d96fd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3007895471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d8c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3007895471 >> 172.21.15.71:0/3007895471 conn(0x55f606d888c0 msgr2=0x55f606d88ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3007895471 shutdown_connections 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3007895471 wait complete. 2024-09-05T12:28:07.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 -- start start 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.245+0000 7fecc2e5e740 1 Processor -- start 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 0x55f606d94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d97510 0x55f606d950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f606d9a060 con 0x55f606d8c330 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f606c7a920 con 0x55f606d8cc50 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f606d9a1e0 con 0x55f606d97510 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecc2e5e740 10 client.0 _ll_register_callbacks cb 0x55f606c9e8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecbd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 0x55f606d94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecbd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 0x55f606d94630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33770/0 (socket says 172.21.15.71:33770) 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecbd800640 1 -- 172.21.15.71:0/2587071347 learned_addr learned my addr 172.21.15.71:0/2587071347 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecbe200640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d97510 0x55f606d950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.246+0000 7fecbce00640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbd800640 1 -- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d97510 msgr2=0x55f606d950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbd800640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d97510 0x55f606d950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbd800640 1 -- 172.21.15.71:0/2587071347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 msgr2=0x55f606d94b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbd800640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d94b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbd800640 1 -- 172.21.15.71:0/2587071347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f606a632b0 con 0x55f606d8c330 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbce00640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecbd800640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 0x55f606d94630 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fecac02af40 tx=0x7fecac02fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9561fc87d28efc16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:07.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fec9f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fec9f400640 1 -- 172.21.15.71:0/2587071347 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac048030 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fec9f400640 1 -- 172.21.15.71:0/2587071347 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fecac040050 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.247+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecac003e50 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fec9f400640 1 -- 172.21.15.71:0/2587071347 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecac034040 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fecc2e5e740 10 client.15345 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f606da5e80 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fec9f400640 1 -- 172.21.15.71:0/2587071347 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecac003e50 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fec9f400640 1 -- 172.21.15.71:0/2587071347 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fecac030440 con 0x55f606d8c330 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fecc2e5e740 10 client.15345 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fecc2e5e740 10 client.15345 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T12:28:07.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fecc2e5e740 20 client.15345 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:07.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.248+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606da7ff0 0x55f606da9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.249+0000 7fecc2e5e740 4 client.15345 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:07.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.249+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 --> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f606cff620 con 0x55f606da7ff0 2024-09-05T12:28:07.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.249+0000 7fecbce00640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606da7ff0 0x55f606da9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.250+0000 7fecbce00640 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606da7ff0 0x55f606da9460 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1894375573291a14 server_cookie=f89a504ed3b774fa in_seq=0 out_seq=0 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.250+0000 7fec9f400640 10 client.15345 ms_handle_connect on v2:172.21.15.174:6836/870087433 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.250+0000 7fec9f400640 1 -- 172.21.15.71:0/2587071347 <== mds.3 v2:172.21.15.174:6836/870087433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f606cff620 con 0x55f606da7ff0 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.250+0000 7fec9f400640 10 client.15345 handle_command_reply: tid=0 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.250+0000 7fecc2e5e740 1 client.15345 shutdown 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 msgr2=0x55f606d94630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 0x55f606d94630 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fecac02af40 tx=0x7fecac02fb70 comp rx=0 tx=0).stop 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 shutdown_connections 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606da7ff0 0x55f606da9460 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d97510 0x55f606d950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d94b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/2587071347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d8c330 0x55f606d94630 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fec9f400640 0 client.15345 ms_handle_reset on v2:172.21.15.174:6836/870087433 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 >> 172.21.15.71:0/2587071347 conn(0x55f606d888c0 msgr2=0x55f606d8e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 shutdown_connections 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 1 -- 172.21.15.71:0/2587071347 wait complete. 2024-09-05T12:28:07.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.251+0000 7fecc2e5e740 20 client.15345 trim_cache size 0 max 16384 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 Processor -- start 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 -- start start 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d94f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d91890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606d91dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f606da77a0 con 0x55f606d97510 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f606da7e70 con 0x55f606d8cc50 2024-09-05T12:28:07.271 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f606da8540 con 0x55f606d8c330 2024-09-05T12:28:07.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecbe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606d91dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecbe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606d91dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33772/0 (socket says 172.21.15.71:33772) 2024-09-05T12:28:07.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecbe200640 1 -- 172.21.15.71:0/767497292 learned_addr learned my addr 172.21.15.71:0/767497292 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:07.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecbce00640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d91890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.270+0000 7fecbd800640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d94f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.273 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbe200640 1 -- 172.21.15.71:0/767497292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 msgr2=0x55f606d94f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbe200640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d94f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbe200640 1 -- 172.21.15.71:0/767497292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 msgr2=0x55f606d91890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbe200640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d91890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbe200640 1 -- 172.21.15.71:0/767497292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f606d899c0 con 0x55f606d97510 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbce00640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d91890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbd800640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d94f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecbe200640 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606d91dd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fecb000c6a0 tx=0x7fecb009c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bcd98d09eac741 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecaa000640 1 -- 172.21.15.71:0/767497292 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb00a4ad0 con 0x55f606d97510 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecaa000640 1 -- 172.21.15.71:0/767497292 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb009a160 con 0x55f606d97510 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.271+0000 7fecaa000640 1 -- 172.21.15.71:0/767497292 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb009b2b0 con 0x55f606d97510 2024-09-05T12:28:07.274 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 -- 172.21.15.71:0/767497292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 msgr2=0x55f606d91dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606d91dd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fecb000c6a0 tx=0x7fecb009c620 comp rx=0 tx=0).stop 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 -- 172.21.15.71:0/767497292 shutdown_connections 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606d91dd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606d91890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/767497292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606d94f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 -- 172.21.15.71:0/767497292 >> 172.21.15.71:0/767497292 conn(0x55f606d888c0 msgr2=0x55f606d92f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 -- 172.21.15.71:0/767497292 shutdown_connections 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 -- 172.21.15.71:0/767497292 wait complete. 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.272+0000 7fecc2e5e740 1 -- start start 2024-09-05T12:28:07.275 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 Processor -- start 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606db1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606dab210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606dab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f606da92e0 con 0x55f606d97510 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f606da8c10 con 0x55f606d8cc50 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f606da99b0 con 0x55f606d8c330 2024-09-05T12:28:07.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecc2e5e740 10 client.0 _ll_register_callbacks cb 0x7feca400be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecbe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606dab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecbe200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606dab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33782/0 (socket says 172.21.15.71:33782) 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecbe200640 1 -- 172.21.15.71:0/3924416604 learned_addr learned my addr 172.21.15.71:0/3924416604 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.273+0000 7fecbce00640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606dab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbd800640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606db1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbe200640 1 -- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 msgr2=0x55f606db1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbe200640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606db1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbe200640 1 -- 172.21.15.71:0/3924416604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 msgr2=0x55f606dab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbe200640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606dab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbe200640 1 -- 172.21.15.71:0/3924416604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f606dabff0 con 0x55f606d97510 2024-09-05T12:28:07.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbce00640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606dab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecbe200640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606dab770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d010 tx=0x7fecb00cd0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a33edd225b7e011 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fec9f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fec9f400640 1 -- 172.21.15.71:0/3924416604 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0080070 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fec9f400640 1 -- 172.21.15.71:0/3924416604 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fecb00cd440 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.274+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb009a160 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fec9f400640 1 -- 172.21.15.71:0/3924416604 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fecb00cd640 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 10 client.15351 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f606e12760 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fec9f400640 1 -- 172.21.15.71:0/3924416604 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb009a160 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fec9f400640 1 -- 172.21.15.71:0/3924416604 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fecb0018cc0 con 0x55f606d97510 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 10 client.15351 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 10 client.15351 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-05T12:28:07.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 20 client.15351 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606e16df0 0x55f606e15b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 4 client.15351 mds_command: new command op to 24490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 --> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] -- command(tid 0: {"prefix": "status"}) -- 0x55f606cf9b30 con 0x55f606e16df0 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.275+0000 7fecbd800640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606e16df0 0x55f606e15b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecbd800640 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606e16df0 0x55f606e15b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=944c1c89d09d418d server_cookie=4106d632cb970b65 in_seq=0 out_seq=0 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fec9f400640 10 client.15351 ms_handle_connect on v2:172.21.15.174:6836/870087433 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fec9f400640 1 -- 172.21.15.71:0/3924416604 <== mds.3 v2:172.21.15.174:6836/870087433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55f606cf9b30 con 0x55f606e16df0 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fec9f400640 10 client.15351 handle_command_reply: tid=0 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 client.15351 shutdown 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 msgr2=0x55f606dab770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606dab770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fecb000d010 tx=0x7fecb00cd0a0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 shutdown_connections 2024-09-05T12:28:07.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:6836/870087433,v1:172.21.15.174:6837/870087433] conn(0x55f606e16df0 0x55f606e15b00 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f606d97510 0x55f606dab770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f606d8cc50 0x55f606dab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 --2- 172.21.15.71:0/3924416604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f606d8c330 0x55f606db1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 >> 172.21.15.71:0/3924416604 conn(0x55f606d888c0 msgr2=0x55f606d87200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.276+0000 7fec9f400640 0 client.15351 ms_handle_reset on v2:172.21.15.174:6836/870087433 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 shutdown_connections 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc2e5e740 1 -- 172.21.15.71:0/3924416604 wait complete. 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc2e5e740 20 client.15351 trim_cache size 0 max 16384 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 3, 2024-09-05T12:28:07.283 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24490, 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 149.973331164, 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 30, 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 95, 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 225.922627541 2024-09-05T12:28:07.284 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feca00a2e10 msgr2=0x7feca00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc0200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feca00a2e10 0x7feca00a52d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55f606d89c40 tx=0x7feca4005c90 comp rx=0 tx=0).stop 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 msgr2=0x7fecb819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.277+0000 7fecc0200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 0x7fecb819eef0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e970 tx=0x7fecb000ed70 comp rx=0 tx=0).stop 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 shutdown_connections 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feca00a2e10 0x7feca00a52d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb810c180 0x7fecb819eef0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecb81036c0 0x7fecb819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 --2- 172.21.15.71:0/1060089059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecb8102da0 0x7fecb819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 >> 172.21.15.71:0/1060089059 conn(0x7fecb80fc7a0 msgr2=0x7fecb8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:07.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 shutdown_connections 2024-09-05T12:28:07.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:07.278+0000 7fecc0200640 1 -- 172.21.15.71:0/1060089059 wait complete. 2024-09-05T12:28:08.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.i status 2024-09-05T12:28:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:08 smithi071 ceph-mon[29845]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-09-05T12:28:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:08 smithi174 ceph-mon[34313]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-09-05T12:28:08.551 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:08 smithi123 ceph-mon[34321]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 -- 172.21.15.71:0/2033076272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae0102360 msgr2=0x7f6ae010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 --2- 172.21.15.71:0/2033076272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae0102360 0x7f6ae010e890 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009cd0 tx=0x7f6ad802a8e0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 -- 172.21.15.71:0/2033076272 shutdown_connections 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 --2- 172.21.15.71:0/2033076272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae0102360 0x7f6ae010e890 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 --2- 172.21.15.71:0/2033076272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae01019c0 0x7f6ae0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 --2- 172.21.15.71:0/2033076272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ae01079c0 0x7f6ae0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.183+0000 7f6adfe00640 1 -- 172.21.15.71:0/2033076272 >> 172.21.15.71:0/2033076272 conn(0x7f6ae00fd740 msgr2=0x7f6ae00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.184+0000 7f6adfe00640 1 -- 172.21.15.71:0/2033076272 shutdown_connections 2024-09-05T12:28:09.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.184+0000 7f6adfe00640 1 -- 172.21.15.71:0/2033076272 wait complete. 2024-09-05T12:28:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.185+0000 7f6adfe00640 1 Processor -- start 2024-09-05T12:28:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.185+0000 7f6adfe00640 1 -- start start 2024-09-05T12:28:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae01019c0 0x7f6ae0198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae0102360 0x7f6ae0198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ae01079c0 0x7f6ae019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adfe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0111dc0 con 0x7f6ae01019c0 2024-09-05T12:28:09.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0111c40 con 0x7f6ae0102360 2024-09-05T12:28:09.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adfe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae0111f40 con 0x7f6ae01079c0 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae01019c0 0x7f6ae0198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6ade000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae0102360 0x7f6ae0198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae01019c0 0x7f6ae0198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33800/0 (socket says 172.21.15.71:33800) 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.186+0000 7f6adea00640 1 -- 172.21.15.71:0/38665623 learned_addr learned my addr 172.21.15.71:0/38665623 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6adf400640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ae01079c0 0x7f6ae019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6ade000640 1 -- 172.21.15.71:0/38665623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ae01079c0 msgr2=0x7f6ae019cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6ade000640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ae01079c0 0x7f6ae019cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6ade000640 1 -- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae01019c0 msgr2=0x7f6ae0198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6ade000640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae01019c0 0x7f6ae0198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6ade000640 1 -- 172.21.15.71:0/38665623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae019d1f0 con 0x7f6ae0102360 2024-09-05T12:28:09.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.187+0000 7f6ade000640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae0102360 0x7f6ae0198780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009e60 tx=0x7f6acc00edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.189+0000 7f6ad7400640 1 -- 172.21.15.71:0/38665623 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc020070 con 0x7f6ae0102360 2024-09-05T12:28:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.189+0000 7f6ad7400640 1 -- 172.21.15.71:0/38665623 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6acc00ccc0 con 0x7f6ae0102360 2024-09-05T12:28:09.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.189+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae019d4e0 con 0x7f6ae0102360 2024-09-05T12:28:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.189+0000 7f6ad7400640 1 -- 172.21.15.71:0/38665623 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc01bb40 con 0x7f6ae0102360 2024-09-05T12:28:09.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.189+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae0074420 con 0x7f6ae0102360 2024-09-05T12:28:09.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.192+0000 7f6ad7400640 1 -- 172.21.15.71:0/38665623 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f6acc023070 con 0x7f6ae0102360 2024-09-05T12:28:09.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 Processor -- start 2024-09-05T12:28:09.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 -- start start 2024-09-05T12:28:09.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856d9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d4856340b0 con 0x55d4856cee00 2024-09-05T12:28:09.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4855dbab0 con 0x55d4856d96c0 2024-09-05T12:28:09.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.193+0000 7f6ae806e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d485630070 con 0x55d4856ce4e0 2024-09-05T12:28:09.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856ce8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35364/0 (socket says 172.21.15.71:35364) 2024-09-05T12:28:09.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 -- 172.21.15.71:0/3669042134 learned_addr learned my addr 172.21.15.71:0/3669042134 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6ade000640 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adea00640 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856d9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 -- 172.21.15.71:0/3669042134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 msgr2=0x55d4856dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 -- 172.21.15.71:0/3669042134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 msgr2=0x55d4856d9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856d9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6adf400640 1 -- 172.21.15.71:0/3669042134 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4855e84a0 con 0x55d4856ce4e0 2024-09-05T12:28:09.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.194+0000 7f6ad7400640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6ac40a2e10 0x7f6ac40a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.195+0000 7f6ad7400640 1 -- 172.21.15.71:0/38665623 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f6acc0d3a50 con 0x7f6ae0102360 2024-09-05T12:28:09.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.195+0000 7f6adf400640 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856ce8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005cc0 tx=0x7f6ad8009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b248877226494104 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:09.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.195+0000 7f6adea00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6ac40a2e10 0x7f6ac40a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.195+0000 7f6acb400640 1 -- 172.21.15.71:0/3669042134 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80316b0 con 0x55d4856ce4e0 2024-09-05T12:28:09.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.195+0000 7f6acb400640 1 -- 172.21.15.71:0/3669042134 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad8036040 con 0x55d4856ce4e0 2024-09-05T12:28:09.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.195+0000 7f6acb400640 1 -- 172.21.15.71:0/3669042134 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad8004500 con 0x55d4856ce4e0 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 -- 172.21.15.71:0/3669042134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 msgr2=0x55d4856ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6adea00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6ac40a2e10 0x7f6ac40a52d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009d80 tx=0x7f6ad0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856ce8c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8005cc0 tx=0x7f6ad8009a20 comp rx=0 tx=0).stop 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 -- 172.21.15.71:0/3669042134 shutdown_connections 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856d9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3669042134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856ce8c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 -- 172.21.15.71:0/3669042134 >> 172.21.15.71:0/3669042134 conn(0x55d4856caa70 msgr2=0x55d4856cae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 -- 172.21.15.71:0/3669042134 shutdown_connections 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.196+0000 7f6ae806e740 1 -- 172.21.15.71:0/3669042134 wait complete. 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.197+0000 7f6ae806e740 1 -- start start 2024-09-05T12:28:09.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.197+0000 7f6ae806e740 1 Processor -- start 2024-09-05T12:28:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856ce4e0 0x55d4856d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856cee00 0x55d4856d6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d4856dc2d0 con 0x55d4856d96c0 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4855bc920 con 0x55d4856ce4e0 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4856dc450 con 0x55d4856cee00 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6ae806e740 10 client.0 _ll_register_callbacks cb 0x55d4855e08a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.198+0000 7f6adea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856ce4e0 0x55d4856d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6adea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856ce4e0 0x55d4856d6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36638/0 (socket says 172.21.15.71:36638) 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6adea00640 1 -- 172.21.15.71:0/3044900091 learned_addr learned my addr 172.21.15.71:0/3044900091 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6adf400640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856cee00 0x55d4856d6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 -- 172.21.15.71:0/3044900091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856ce4e0 msgr2=0x55d4856d6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856ce4e0 0x55d4856d6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 -- 172.21.15.71:0/3044900091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 msgr2=0x55d4856d70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 -- 172.21.15.71:0/3044900091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4853a52b0 con 0x55d4856cee00 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6adf400640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6ade000640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856cee00 0x55d4856d6b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0d0d90 tx=0x7f6acc09f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=408eda914324a9bb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.199+0000 7f6abfe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:09.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6abfe00640 1 -- 172.21.15.71:0/3044900091 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0a8470 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6abfe00640 1 -- 172.21.15.71:0/3044900091 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f6acc0d0f60 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ad8036040 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6abfe00640 1 -- 172.21.15.71:0/3044900091 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6acc0a3e40 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6ae806e740 10 client.25414 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4856f0810 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6abfe00640 1 -- 172.21.15.71:0/3044900091 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ad8036040 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.200+0000 7f6abfe00640 1 -- 172.21.15.71:0/3044900091 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6acc01bce0 con 0x55d4856cee00 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6ae806e740 10 client.25414 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6ae806e740 10 client.25414 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6ae806e740 20 client.25414 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d4856f2d80 0x55d4856f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6ae806e740 4 client.25414 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:09.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 --> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d485641620 con 0x55d4856f2d80 2024-09-05T12:28:09.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.201+0000 7f6adea00640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d4856f2d80 0x55d4856f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.202+0000 7f6abfe00640 10 client.25414 ms_handle_connect on v2:172.21.15.174:6838/584492863 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.202+0000 7f6adea00640 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d4856f2d80 0x55d4856f41f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8bdda849db32de67 server_cookie=91c87e61bca30f69 in_seq=0 out_seq=0 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6abfe00640 1 -- 172.21.15.71:0/3044900091 <== mds.2 v2:172.21.15.174:6838/584492863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d485641620 con 0x55d4856f2d80 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6abfe00640 10 client.25414 handle_command_reply: tid=0 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 client.25414 shutdown 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856cee00 msgr2=0x55d4856d6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856cee00 0x55d4856d6b80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0d0d90 tx=0x7f6acc09f680 comp rx=0 tx=0).stop 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 shutdown_connections 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d4856f2d80 0x55d4856f41f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856cee00 0x55d4856d6b80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3044900091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856ce4e0 0x55d4856d6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6abfe00640 0 client.25414 ms_handle_reset on v2:172.21.15.174:6838/584492863 2024-09-05T12:28:09.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 >> 172.21.15.71:0/3044900091 conn(0x55d4856caa70 msgr2=0x55d4856d07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:09.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 shutdown_connections 2024-09-05T12:28:09.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 1 -- 172.21.15.71:0/3044900091 wait complete. 2024-09-05T12:28:09.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.203+0000 7f6ae806e740 20 client.25414 trim_cache size 0 max 16384 2024-09-05T12:28:09.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.228+0000 7f6ae806e740 1 Processor -- start 2024-09-05T12:28:09.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.228+0000 7f6ae806e740 1 -- start start 2024-09-05T12:28:09.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856cee00 0x55d4856d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ae806e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d4856f2530 con 0x55d4856d96c0 2024-09-05T12:28:09.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ae806e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4856f2c00 con 0x55d4856cee00 2024-09-05T12:28:09.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ae806e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4856f32d0 con 0x55d4856ce4e0 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6adf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6adf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33824/0 (socket says 172.21.15.71:33824) 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6adf400640 1 -- 172.21.15.71:0/2243959421 learned_addr learned my addr 172.21.15.71:0/2243959421 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6adea00640 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ade000640 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856cee00 0x55d4856d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ade000640 1 -- 172.21.15.71:0/2243959421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 msgr2=0x55d4856f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ade000640 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ade000640 1 -- 172.21.15.71:0/2243959421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 msgr2=0x55d4856d3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ade000640 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.229+0000 7f6ade000640 1 -- 172.21.15.71:0/2243959421 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4856dbab0 con 0x55d4856cee00 2024-09-05T12:28:09.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.230+0000 7f6adf400640 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:09.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.230+0000 7f6ade000640 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856cee00 0x55d4856d38a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55d4856d7bf0 tx=0x7f6acc09fcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d750a5ac021a5511 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:09.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.230+0000 7f6acaa00640 1 -- 172.21.15.71:0/2243959421 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc09f4a0 con 0x55d4856cee00 2024-09-05T12:28:09.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.230+0000 7f6acaa00640 1 -- 172.21.15.71:0/2243959421 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6acc0a8700 con 0x55d4856cee00 2024-09-05T12:28:09.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.230+0000 7f6acaa00640 1 -- 172.21.15.71:0/2243959421 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0d0ed0 con 0x55d4856cee00 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 -- 172.21.15.71:0/2243959421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856cee00 msgr2=0x55d4856d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856cee00 0x55d4856d38a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55d4856d7bf0 tx=0x7f6acc09fcc0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 -- 172.21.15.71:0/2243959421 shutdown_connections 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856d96c0 0x55d4856d3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856cee00 0x55d4856d38a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 --2- 172.21.15.71:0/2243959421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 -- 172.21.15.71:0/2243959421 >> 172.21.15.71:0/2243959421 conn(0x55d4856caa70 msgr2=0x55d4856d4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 -- 172.21.15.71:0/2243959421 shutdown_connections 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.231+0000 7f6ae806e740 1 -- 172.21.15.71:0/2243959421 wait complete. 2024-09-05T12:28:09.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.232+0000 7f6ae806e740 1 -- start start 2024-09-05T12:28:09.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.232+0000 7f6ae806e740 1 Processor -- start 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.232+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ae806e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856fb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ae806e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d4856f4070 con 0x55d4856cee00 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ae806e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d4856f39a0 con 0x55d4856d96c0 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ae806e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4856f4740 con 0x55d4856ce4e0 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ae806e740 10 client.0 _ll_register_callbacks cb 0x7f6acc00ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:09.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ade000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856fb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36662/0 (socket says 172.21.15.71:36662) 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 -- 172.21.15.71:0/3691117883 learned_addr learned my addr 172.21.15.71:0/3691117883 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 -- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 msgr2=0x55d4856f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 -- 172.21.15.71:0/3691117883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 msgr2=0x55d4856f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6adf400640 1 -- 172.21.15.71:0/3691117883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4856fb690 con 0x55d4856d96c0 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.233+0000 7f6ade000640 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.234+0000 7f6adf400640 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856fb050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009620 tx=0x7f6ad8003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125b7c921f33dee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.234+0000 7f6abfe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:28:09.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.234+0000 7f6abfe00640 1 -- 172.21.15.71:0/3691117883 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80307a0 con 0x55d4856d96c0 2024-09-05T12:28:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.234+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6acc0a8700 con 0x55d4856d96c0 2024-09-05T12:28:09.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.234+0000 7f6abfe00640 1 -- 172.21.15.71:0/3691117883 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f6ad8030940 con 0x55d4856d96c0 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6abfe00640 1 -- 172.21.15.71:0/3691117883 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6ad8059050 con 0x55d4856d96c0 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 10 client.25421 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d48575ddd0 con 0x55d4856d96c0 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6abfe00640 1 -- 172.21.15.71:0/3691117883 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6acc0a8700 con 0x55d4856d96c0 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6abfe00640 1 -- 172.21.15.71:0/3691117883 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f6ad8030070 con 0x55d4856d96c0 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 10 client.25421 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 10 client.25421 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 20 client.25421 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d485762480 0x55d485761190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 4 client.25421 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.235+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 --> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] -- command(tid 0: {"prefix": "status"}) -- 0x55d48563bb30 con 0x55d485762480 2024-09-05T12:28:09.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.236+0000 7f6adea00640 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d485762480 0x55d485761190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.236+0000 7f6abfe00640 10 client.25421 ms_handle_connect on v2:172.21.15.174:6838/584492863 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.236+0000 7f6adea00640 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d485762480 0x55d485761190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9567d43351c3941 server_cookie=cf4360ebe4162d4f in_seq=0 out_seq=0 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6abfe00640 1 -- 172.21.15.71:0/3691117883 <== mds.2 v2:172.21.15.174:6838/584492863 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d48563bb30 con 0x55d485762480 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6abfe00640 10 client.25421 handle_command_reply: tid=0 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 client.25421 shutdown 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 msgr2=0x55d4856fb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856fb050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ad8009620 tx=0x7f6ad8003ba0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 shutdown_connections 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:6838/584492863,v1:172.21.15.174:6839/584492863] conn(0x55d485762480 0x55d485761190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d4856d96c0 0x55d4856fb050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4856cee00 0x55d4856f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 --2- 172.21.15.71:0/3691117883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4856ce4e0 0x55d4856f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 >> 172.21.15.71:0/3691117883 conn(0x55d4856caa70 msgr2=0x55d4856cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:09.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6abfe00640 0 client.25421 ms_handle_reset on v2:172.21.15.174:6838/584492863 2024-09-05T12:28:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 shutdown_connections 2024-09-05T12:28:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 1 -- 172.21.15.71:0/3691117883 wait complete. 2024-09-05T12:28:09.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.237+0000 7f6ae806e740 20 client.25421 trim_cache size 0 max 16384 2024-09-05T12:28:09.241 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:09.241 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:28:09.241 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 2, 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24502, 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 152.94597910499999, 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 30, 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 95, 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 222.426722979 2024-09-05T12:28:09.242 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6ac40a2e10 msgr2=0x7f6ac40a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6ac40a2e10 0x7f6ac40a52d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0009d80 tx=0x7f6ad0005f90 comp rx=0 tx=0).stop 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae0102360 msgr2=0x7f6ae0198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae0102360 0x7f6ae0198780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009e60 tx=0x7f6acc00edc0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 shutdown_connections 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ae01079c0 0x7f6ae019cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae0102360 0x7f6ae0198780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6ac40a2e10 0x7f6ac40a52d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 --2- 172.21.15.71:0/38665623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ae01019c0 0x7f6ae0198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 >> 172.21.15.71:0/38665623 conn(0x7f6ae00fd740 msgr2=0x7f6ae00fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 shutdown_connections 2024-09-05T12:28:09.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:09.238+0000 7f6adfe00640 1 -- 172.21.15.71:0/38665623 wait complete. 2024-09-05T12:28:10.284 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.g status 2024-09-05T12:28:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:10 smithi174 ceph-mon[34313]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-05T12:28:10.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:10 smithi071 ceph-mon[29845]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-05T12:28:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:10 smithi123 ceph-mon[34321]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-05T12:28:10.683 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.314+0000 7f33aa800640 1 -- 172.21.15.71:0/2450415654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a41047e0 msgr2=0x7f33a4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.314+0000 7f33aa800640 1 --2- 172.21.15.71:0/2450415654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a41047e0 0x7f33a4108830 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3390009900 tx=0x7f339002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.314+0000 7f33aa800640 1 -- 172.21.15.71:0/2450415654 shutdown_connections 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.314+0000 7f33aa800640 1 --2- 172.21.15.71:0/2450415654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4108f60 0x7f33a4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.314+0000 7f33aa800640 1 --2- 172.21.15.71:0/2450415654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a41047e0 0x7f33a4108830 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.315+0000 7f33aa800640 1 --2- 172.21.15.71:0/2450415654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a4103e30 0x7f33a4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.315+0000 7f33aa800640 1 -- 172.21.15.71:0/2450415654 >> 172.21.15.71:0/2450415654 conn(0x7f33a40ff910 msgr2=0x7f33a4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.315+0000 7f33aa800640 1 -- 172.21.15.71:0/2450415654 shutdown_connections 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.315+0000 7f33aa800640 1 -- 172.21.15.71:0/2450415654 wait complete. 2024-09-05T12:28:11.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 Processor -- start 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 -- start start 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4103e30 0x7f33a419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 0x7f33a419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a4108f60 0x7f33a419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4113fd0 con 0x7f33a41047e0 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4113e50 con 0x7f33a4108f60 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.316+0000 7f33aa800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4114150 con 0x7f33a4103e30 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 0x7f33a419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 0x7f33a419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33852/0 (socket says 172.21.15.71:33852) 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 -- 172.21.15.71:0/421701031 learned_addr learned my addr 172.21.15.71:0/421701031 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a9e00640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a4108f60 0x7f33a419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.336 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a9400640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4103e30 0x7f33a419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 -- 172.21.15.71:0/421701031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4103e30 msgr2=0x7f33a419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4103e30 0x7f33a419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 -- 172.21.15.71:0/421701031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a4108f60 msgr2=0x7f33a419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a4108f60 0x7f33a419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 -- 172.21.15.71:0/421701031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a419f440 con 0x7f33a41047e0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a9400640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4103e30 0x7f33a419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.317+0000 7f33a8a00640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 0x7f33a419a9d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3390004770 tx=0x7f3390003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.318+0000 7f33a2000640 1 -- 172.21.15.71:0/421701031 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339003f070 con 0x7f33a41047e0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.318+0000 7f33a2000640 1 -- 172.21.15.71:0/421701031 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3390031ca0 con 0x7f33a41047e0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.318+0000 7f33a2000640 1 -- 172.21.15.71:0/421701031 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390035030 con 0x7f33a41047e0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.318+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a419f6d0 con 0x7f33a41047e0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.318+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a41a7010 con 0x7f33a41047e0 2024-09-05T12:28:11.337 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.321+0000 7f33ae827740 1 Processor -- start 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.321+0000 7f33a2000640 1 -- 172.21.15.71:0/421701031 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f3390042070 con 0x7f33a41047e0 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.321+0000 7f33ae827740 1 -- start start 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33ae827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33ae827740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33ae827740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33ae827740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc81c0b0 con 0x55f7cc8b6e00 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33ae827740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc7c3ab0 con 0x55f7cc8c16c0 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33ae827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc818070 con 0x55f7cc8b64e0 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35418/0 (socket says 172.21.15.71:35418) 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33a9e00640 1 -- 172.21.15.71:0/1531704137 learned_addr learned my addr 172.21.15.71:0/1531704137 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33a9400640 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.338 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.322+0000 7f33a8a00640 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9e00640 1 -- 172.21.15.71:0/1531704137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8c16c0 msgr2=0x55f7cc8c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9e00640 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9e00640 1 -- 172.21.15.71:0/1531704137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 msgr2=0x55f7cc8c1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9e00640 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8c1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9e00640 1 -- 172.21.15.71:0/1531704137 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7cc7d04a0 con 0x55f7cc8b64e0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9400640 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a9e00640 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b68c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f339c009f50 tx=0x7f339c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ba01a36e9d9532 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a2000640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f338c0a2e90 0x7f338c0a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f3396600640 1 -- 172.21.15.71:0/1531704137 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c004d20 con 0x55f7cc8b64e0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f3396600640 1 -- 172.21.15.71:0/1531704137 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c004050 con 0x55f7cc8b64e0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f3396600640 1 -- 172.21.15.71:0/1531704137 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c005280 con 0x55f7cc8b64e0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.323+0000 7f33a2000640 1 -- 172.21.15.71:0/421701031 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f33900efef0 con 0x7f33a41047e0 2024-09-05T12:28:11.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33a9400640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f338c0a2e90 0x7f338c0a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 -- 172.21.15.71:0/1531704137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 msgr2=0x55f7cc8b68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b68c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f339c009f50 tx=0x7f339c0118d0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 -- 172.21.15.71:0/1531704137 shutdown_connections 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8c1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 --2- 172.21.15.71:0/1531704137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b68c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 -- 172.21.15.71:0/1531704137 >> 172.21.15.71:0/1531704137 conn(0x55f7cc8b2a70 msgr2=0x55f7cc8b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33a9400640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f338c0a2e90 0x7f338c0a5350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3398009fd0 tx=0x7f3398005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.324+0000 7f33ae827740 1 -- 172.21.15.71:0/1531704137 shutdown_connections 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.325+0000 7f33ae827740 1 -- 172.21.15.71:0/1531704137 wait complete. 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.325+0000 7f33ae827740 1 -- start start 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.325+0000 7f33ae827740 1 Processor -- start 2024-09-05T12:28:11.340 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8be5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8beb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8bf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8c4330 con 0x55f7cc8b6e00 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc7a4920 con 0x55f7cc8b64e0 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8c44b0 con 0x55f7cc8c16c0 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33ae827740 10 client.0 _ll_register_callbacks cb 0x55f7cc7c88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8beb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8be5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8bf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8be5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36688/0 (socket says 172.21.15.71:36688) 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8bf060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35428/0 (socket says 172.21.15.71:35428) 2024-09-05T12:28:11.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a9400640 1 -- 172.21.15.71:0/3079791497 learned_addr learned my addr 172.21.15.71:0/3079791497 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a8a00640 1 -- 172.21.15.71:0/3079791497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8c16c0 msgr2=0x55f7cc8bf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a8a00640 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8bf060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a8a00640 1 -- 172.21.15.71:0/3079791497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 msgr2=0x55f7cc8be5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a8a00640 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8be5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.326+0000 7f33a8a00640 1 -- 172.21.15.71:0/3079791497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7cc58d2b0 con 0x55f7cc8b6e00 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f33a8a00640 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8beb20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33900ed120 tx=0x7f339003b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2693952aac331aa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f338aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f338aa00640 1 -- 172.21.15.71:0/3079791497 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390049070 con 0x55f7cc8b6e00 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f338aa00640 1 -- 172.21.15.71:0/3079791497 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f339003b4f0 con 0x55f7cc8b6e00 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339c004050 con 0x55f7cc8b6e00 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f338aa00640 1 -- 172.21.15.71:0/3079791497 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f33900bbe30 con 0x55f7cc8b6e00 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f33ae827740 10 client.15381 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:11.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.327+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f7cc8d0010 con 0x55f7cc8b6e00 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f338aa00640 1 -- 172.21.15.71:0/3079791497 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339c004050 con 0x55f7cc8b6e00 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f338aa00640 1 -- 172.21.15.71:0/3079791497 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f33900bc2c0 con 0x55f7cc8b6e00 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33ae827740 10 client.15381 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33ae827740 10 client.15381 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33ae827740 20 client.15381 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33ae827740 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8d2670 0x55f7cc8d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33ae827740 4 client.15381 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 --> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7cc829620 con 0x55f7cc8d2670 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.328+0000 7f33a9400640 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8d2670 0x55f7cc8d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.329+0000 7f33a9400640 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8d2670 0x55f7cc8d3ae0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9b085d64983818c4 server_cookie=6a8e9b03c68d4f0e in_seq=0 out_seq=0 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.329+0000 7f338aa00640 10 client.15381 ms_handle_connect on v2:172.21.15.71:6838/2593934735 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f338aa00640 1 -- 172.21.15.71:0/3079791497 <== mds.1 v2:172.21.15.71:6838/2593934735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7cc829620 con 0x55f7cc8d2670 2024-09-05T12:28:11.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f338aa00640 10 client.15381 handle_command_reply: tid=0 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 client.15381 shutdown 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 msgr2=0x55f7cc8beb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8beb20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f33900ed120 tx=0x7f339003b460 comp rx=0 tx=0).stop 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 shutdown_connections 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8d2670 0x55f7cc8d3ae0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8c16c0 0x55f7cc8bf060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8beb20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 --2- 172.21.15.71:0/3079791497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8be5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f338aa00640 0 client.15381 ms_handle_reset on v2:172.21.15.71:6838/2593934735 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 >> 172.21.15.71:0/3079791497 conn(0x55f7cc8b2a70 msgr2=0x55f7cc8b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 shutdown_connections 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.330+0000 7f33ae827740 1 -- 172.21.15.71:0/3079791497 wait complete. 2024-09-05T12:28:11.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.331+0000 7f33ae827740 20 client.15381 trim_cache size 0 max 16384 2024-09-05T12:28:11.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 Processor -- start 2024-09-05T12:28:11.350 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 -- start start 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8d1e20 con 0x55f7cc8b2a70 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8d24f0 con 0x55f7cc8b64e0 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33ae827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8d2bc0 con 0x55f7cc8b6e00 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33a9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33a8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33a8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36694/0 (socket says 172.21.15.71:36694) 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33a9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33874/0 (socket says 172.21.15.71:33874) 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.350+0000 7f33a8a00640 1 -- 172.21.15.71:0/203949648 learned_addr learned my addr 172.21.15.71:0/203949648 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:11.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a9e00640 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a8a00640 1 -- 172.21.15.71:0/203949648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 msgr2=0x55f7cc8bb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a8a00640 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8bb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a8a00640 1 -- 172.21.15.71:0/203949648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 msgr2=0x55f7cc8d3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a8a00640 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8d3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a8a00640 1 -- 172.21.15.71:0/203949648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7cc8bcb30 con 0x55f7cc8b64e0 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a9400640 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:11.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f33a8a00640 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b8f40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f33900c46f0 tx=0x7f33900ba570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b687ab80affd5eb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f3395c00640 1 -- 172.21.15.71:0/203949648 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390049070 con 0x55f7cc8b64e0 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f3395c00640 1 -- 172.21.15.71:0/203949648 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33900bc060 con 0x55f7cc8b64e0 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.351+0000 7f3395c00640 1 -- 172.21.15.71:0/203949648 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3390005d50 con 0x55f7cc8b64e0 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.352+0000 7f33ae827740 1 -- 172.21.15.71:0/203949648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 msgr2=0x55f7cc8b8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.352+0000 7f33ae827740 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b8f40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f33900c46f0 tx=0x7f33900ba570 comp rx=0 tx=0).stop 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 -- 172.21.15.71:0/203949648 shutdown_connections 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8bb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8b8f40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 --2- 172.21.15.71:0/203949648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8d3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 -- 172.21.15.71:0/203949648 >> 172.21.15.71:0/203949648 conn(0x55f7cc8d1a40 msgr2=0x55f7cc8bceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:11.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 -- 172.21.15.71:0/203949648 shutdown_connections 2024-09-05T12:28:11.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 -- 172.21.15.71:0/203949648 wait complete. 2024-09-05T12:28:11.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 -- start start 2024-09-05T12:28:11.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.353+0000 7f33ae827740 1 Processor -- start 2024-09-05T12:28:11.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8dbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8d6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8d3960 con 0x55f7cc8b2a70 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8d3290 con 0x55f7cc8b64e0 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f7cc8d4030 con 0x55f7cc8b6e00 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33ae827740 10 client.0 _ll_register_callbacks cb 0x7f3390009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33a9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8dbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33a9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8dbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33884/0 (socket says 172.21.15.71:33884) 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33a9400640 1 -- 172.21.15.71:0/1525952503 learned_addr learned my addr 172.21.15.71:0/1525952503 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.354+0000 7f33a8a00640 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33a9400640 1 -- 172.21.15.71:0/1525952503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 msgr2=0x55f7cc8d6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33a9400640 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8d6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33a9400640 1 -- 172.21.15.71:0/1525952503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 msgr2=0x55f7cc8dc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33a9400640 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8dc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33a9400640 1 -- 172.21.15.71:0/1525952503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7cc8d71e0 con 0x55f7cc8b2a70 2024-09-05T12:28:11.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33a9400640 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8dbcf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3398009530 tx=0x7f33980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad84bb5d1472385 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f338aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f338aa00640 1 -- 172.21.15.71:0/1525952503 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3398004410 con 0x55f7cc8b2a70 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f338aa00640 1 -- 172.21.15.71:0/1525952503 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f339800f9c0 con 0x55f7cc8b2a70 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.355+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33900bc060 con 0x55f7cc8b2a70 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f338aa00640 1 -- 172.21.15.71:0/1525952503 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f339803c050 con 0x55f7cc8b2a70 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 10 client.15387 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:11.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f7cc93d4e0 con 0x55f7cc8b2a70 2024-09-05T12:28:11.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f338aa00640 1 -- 172.21.15.71:0/1525952503 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33900bc060 con 0x55f7cc8b2a70 2024-09-05T12:28:11.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f338aa00640 1 -- 172.21.15.71:0/1525952503 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f339802b380 con 0x55f7cc8b2a70 2024-09-05T12:28:11.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 10 client.15387 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 10 client.15387 resolve_mds: resolved name 'g' to daemon mds.g 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 20 client.15387 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8c16c0 0x55f7cc940e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 4 client.15387 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.356+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 --> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] -- command(tid 0: {"prefix": "status"}) -- 0x55f7cc823b30 con 0x55f7cc8c16c0 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.357+0000 7f33a8a00640 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8c16c0 0x55f7cc940e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.357+0000 7f33a8a00640 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8c16c0 0x55f7cc940e60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37869f2b705e0154 server_cookie=84d74c1c9230e98b in_seq=0 out_seq=0 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.357+0000 7f338aa00640 10 client.15387 ms_handle_connect on v2:172.21.15.71:6838/2593934735 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f338aa00640 1 -- 172.21.15.71:0/1525952503 <== mds.1 v2:172.21.15.71:6838/2593934735 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55f7cc823b30 con 0x55f7cc8c16c0 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f338aa00640 10 client.15387 handle_command_reply: tid=0 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 client.15387 shutdown 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 msgr2=0x55f7cc8dbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8dbcf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3398009530 tx=0x7f33980046c0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 shutdown_connections 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:6838/2593934735,v1:172.21.15.71:6839/2593934735] conn(0x55f7cc8c16c0 0x55f7cc940e60 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f7cc8b6e00 0x55f7cc8d6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f7cc8b64e0 0x55f7cc8dc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 --2- 172.21.15.71:0/1525952503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7cc8b2a70 0x55f7cc8dbcf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 >> 172.21.15.71:0/1525952503 conn(0x55f7cc8d1a40 msgr2=0x55f7cc8bfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f338aa00640 0 client.15387 ms_handle_reset on v2:172.21.15.71:6838/2593934735 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 shutdown_connections 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 1 -- 172.21.15.71:0/1525952503 wait complete. 2024-09-05T12:28:11.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.358+0000 7f33ae827740 20 client.15387 trim_cache size 0 max 16384 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 1, 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24514, 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 156.07467692899999, 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 30, 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 95, 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 220.60542594 2024-09-05T12:28:11.362 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f338c0a2e90 msgr2=0x7f338c0a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f338c0a2e90 0x7f338c0a5350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3398009fd0 tx=0x7f3398005f90 comp rx=0 tx=0).stop 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 msgr2=0x7f33a419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 0x7f33a419a9d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3390004770 tx=0x7f3390003c10 comp rx=0 tx=0).stop 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 shutdown_connections 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33a4108f60 0x7f33a419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f338c0a2e90 0x7f338c0a5350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f33a41047e0 0x7f33a419a9d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 --2- 172.21.15.71:0/421701031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33a4103e30 0x7f33a419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.359+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 >> 172.21.15.71:0/421701031 conn(0x7f33a40ff910 msgr2=0x7f33a4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.360+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 shutdown_connections 2024-09-05T12:28:11.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:11.360+0000 7f33aa800640 1 -- 172.21.15.71:0/421701031 wait complete. 2024-09-05T12:28:12.188 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph tell mds.b status 2024-09-05T12:28:12.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:12 smithi071 ceph-mon[29845]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-05T12:28:12.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:12 smithi174 ceph-mon[34313]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-05T12:28:12.573 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:12 smithi123 ceph-mon[34321]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 -- 172.21.15.71:0/479851688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 msgr2=0x7f05101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 --2- 172.21.15.71:0/479851688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 0x7f05101067e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009900 tx=0x7f04fc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 -- 172.21.15.71:0/479851688 shutdown_connections 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 --2- 172.21.15.71:0/479851688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510106f10 0x7f051010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 --2- 172.21.15.71:0/479851688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 0x7f05101067e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 --2- 172.21.15.71:0/479851688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0510101de0 0x7f05101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 -- 172.21.15.71:0/479851688 >> 172.21.15.71:0/479851688 conn(0x7f05100fd8c0 msgr2=0x7f05100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 -- 172.21.15.71:0/479851688 shutdown_connections 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.217+0000 7f0517400640 1 -- 172.21.15.71:0/479851688 wait complete. 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.218+0000 7f0517400640 1 Processor -- start 2024-09-05T12:28:13.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.218+0000 7f0517400640 1 -- start start 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0517400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 0x7f051019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0517400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 0x7f051019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0517400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0510106f10 0x7f051019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0517400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0510111fc0 con 0x7f0510102790 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0517400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0510111e40 con 0x7f0510101de0 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0517400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0510112140 con 0x7f0510106f10 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0514a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 0x7f051019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0514a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 0x7f051019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57242/0 (socket says 172.21.15.71:57242) 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0514a00640 1 -- 172.21.15.71:0/1205737430 learned_addr learned my addr 172.21.15.71:0/1205737430 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f0515400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0510106f10 0x7f051019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.219+0000 7f050fe00640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 0x7f051019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0514a00640 1 -- 172.21.15.71:0/1205737430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0510106f10 msgr2=0x7f051019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0514a00640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0510106f10 0x7f051019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0514a00640 1 -- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 msgr2=0x7f051019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0514a00640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 0x7f051019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0514a00640 1 -- 172.21.15.71:0/1205737430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051019f690 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0514a00640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 0x7f051019a6e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f050400e9f0 tx=0x7f050400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f050d600640 1 -- 172.21.15.71:0/1205737430 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050400ccc0 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f050d600640 1 -- 172.21.15.71:0/1205737430 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0504010910 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051019f980 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f050d600640 1 -- 172.21.15.71:0/1205737430 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040188a0 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.220+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05101a7260 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.223+0000 7f050d600640 1 -- 172.21.15.71:0/1205737430 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f0504010ac0 con 0x7f0510101de0 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 Processor -- start 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 -- start start 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 0x55a84100a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100ac50 0x55a841014fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a841017900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a840f700b0 con 0x55a841015510 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a840f17ab0 con 0x55a84100a330 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.224+0000 7f0519ed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a840f6c070 con 0x55a84100ac50 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f0515400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 0x55a84100a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f0514a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100ac50 0x55a841014fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a841017900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f0515400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 0x55a84100a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57252/0 (socket says 172.21.15.71:57252) 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f0515400640 1 -- 172.21.15.71:0/1510074499 learned_addr learned my addr 172.21.15.71:0/1510074499 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f050d600640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04f80a2e90 0x7f04f80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f050d600640 1 -- 172.21.15.71:0/1205737430 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f05040cff10 con 0x7f0510101de0 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.225+0000 7f0514a00640 1 -- 172.21.15.71:0/1510074499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 msgr2=0x55a84100a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0514a00640 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 0x55a84100a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0514a00640 1 -- 172.21.15.71:0/1510074499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 msgr2=0x55a841017900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0514a00640 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a841017900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0514a00640 1 -- 172.21.15.71:0/1510074499 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a840f244a0 con 0x55a84100ac50 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0515400640 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 0x55a84100a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f050fe00640 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a841017900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0515400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04f80a2e90 0x7f04f80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0514a00640 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100ac50 0x55a841014fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f05040cd140 tx=0x7f0504009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=929d2924cd7e35a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.226+0000 7f0501c00640 1 -- 172.21.15.71:0/1510074499 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050409be40 con 0x55a84100ac50 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.227+0000 7f0501c00640 1 -- 172.21.15.71:0/1510074499 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05040a01f0 con 0x55a84100ac50 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.227+0000 7f0501c00640 1 -- 172.21.15.71:0/1510074499 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05040cd420 con 0x55a84100ac50 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 -- 172.21.15.71:0/1510074499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100ac50 msgr2=0x55a841014fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0515400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04f80a2e90 0x7f04f80a5350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0508010790 tx=0x7f0508004a40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100ac50 0x55a841014fd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f05040cd140 tx=0x7f0504009700 comp rx=0 tx=0).stop 2024-09-05T12:28:13.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 -- 172.21.15.71:0/1510074499 shutdown_connections 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a841017900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100ac50 0x55a841014fd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 --2- 172.21.15.71:0/1510074499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84100a330 0x55a84100a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 -- 172.21.15.71:0/1510074499 >> 172.21.15.71:0/1510074499 conn(0x55a8410068c0 msgr2=0x55a841006ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 -- 172.21.15.71:0/1510074499 shutdown_connections 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 -- 172.21.15.71:0/1510074499 wait complete. 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.228+0000 7f0519ed3740 1 -- start start 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 Processor -- start 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841013150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a841018060 con 0x55a84100ac50 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a840ef8920 con 0x55a841015510 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a8410181e0 con 0x55a84100a330 2024-09-05T12:28:13.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f0519ed3740 10 client.0 _ll_register_callbacks cb 0x55a840f1c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.229+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34994/0 (socket says 172.21.15.71:34994) 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34638/0 (socket says 172.21.15.71:34638) 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 -- 172.21.15.71:0/3960363010 learned_addr learned my addr 172.21.15.71:0/3960363010 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0515400640 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841013150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 -- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 msgr2=0x55a841013150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841013150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 -- 172.21.15.71:0/3960363010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 msgr2=0x55a841012bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 -- 172.21.15.71:0/3960363010 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a840ce12b0 con 0x55a84100a330 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f0514a00640 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05040982a0 tx=0x7f050400cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce23b2eddae4e6bc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f04f6000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:13.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f04f6000640 1 -- 172.21.15.71:0/3960363010 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050409e070 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.230+0000 7f04f6000640 1 -- 172.21.15.71:0/3960363010 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f05040cd630 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.231+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05040a01f0 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.231+0000 7f04f6000640 1 -- 172.21.15.71:0/3960363010 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f050409be40 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.231+0000 7f0519ed3740 10 client.25447 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.231+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a841018060 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.231+0000 7f04f6000640 1 -- 172.21.15.71:0/3960363010 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05040a01f0 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.231+0000 7f04f6000640 1 -- 172.21.15.71:0/3960363010 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f050409ad00 con 0x55a84100a330 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f0519ed3740 10 client.25447 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f0519ed3740 10 client.25447 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f0519ed3740 20 client.25447 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f0519ed3740 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a841025e70 0x55a8410272e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f0519ed3740 4 client.25447 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:13.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a840f7d620 con 0x55a841025e70 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.232+0000 7f050fe00640 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a841025e70 0x55a8410272e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.233+0000 7f04f6000640 10 client.25447 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.233+0000 7f050fe00640 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a841025e70 0x55a8410272e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b329190bd36919 server_cookie=45e3ee59d6dc00a8 in_seq=0 out_seq=0 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f04f6000640 1 -- 172.21.15.71:0/3960363010 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a840f7d620 con 0x55a841025e70 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f04f6000640 10 client.25447 handle_command_reply: tid=0 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 client.25447 shutdown 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 msgr2=0x55a841012690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012690 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f05040982a0 tx=0x7f050400cef0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 shutdown_connections 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a841025e70 0x55a8410272e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f04f6000640 0 client.25447 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841013150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 --2- 172.21.15.71:0/3960363010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012690 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 >> 172.21.15.71:0/3960363010 conn(0x55a8410068c0 msgr2=0x55a84100c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 shutdown_connections 2024-09-05T12:28:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 1 -- 172.21.15.71:0/3960363010 wait complete. 2024-09-05T12:28:13.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.234+0000 7f0519ed3740 20 client.25447 trim_cache size 0 max 16384 2024-09-05T12:28:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.253+0000 7f0519ed3740 1 Processor -- start 2024-09-05T12:28:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.253+0000 7f0519ed3740 1 -- start start 2024-09-05T12:28:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841010f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0519ed3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a841025620 con 0x55a84100ac50 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0519ed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a841025cf0 con 0x55a841015510 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0519ed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a8410263c0 con 0x55a84100a330 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0514a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0514a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35000/0 (socket says 172.21.15.71:35000) 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34648/0 (socket says 172.21.15.71:34648) 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0514a00640 1 -- 172.21.15.71:0/339131289 learned_addr learned my addr 172.21.15.71:0/339131289 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.254+0000 7f0515400640 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841010f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f050fe00640 1 -- 172.21.15.71:0/339131289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 msgr2=0x55a841012650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f050fe00640 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f050fe00640 1 -- 172.21.15.71:0/339131289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 msgr2=0x55a841010f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f050fe00640 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841010f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f050fe00640 1 -- 172.21.15.71:0/339131289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a84100a0b0 con 0x55a84100ac50 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f0514a00640 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f050fe00640 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f04fc02a9e0 tx=0x7f04fc005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfe13cebb95159d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f0501200640 1 -- 172.21.15.71:0/339131289 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc035040 con 0x55a84100ac50 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f0501200640 1 -- 172.21.15.71:0/339131289 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04fc005520 con 0x55a84100ac50 2024-09-05T12:28:13.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.255+0000 7f0501200640 1 -- 172.21.15.71:0/339131289 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc0315b0 con 0x55a84100ac50 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 -- 172.21.15.71:0/339131289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 msgr2=0x55a841012b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012b90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f04fc02a9e0 tx=0x7f04fc005b50 comp rx=0 tx=0).stop 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 -- 172.21.15.71:0/339131289 shutdown_connections 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a841015510 0x55a841010f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a84100ac50 0x55a841012b90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 --2- 172.21.15.71:0/339131289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a841012650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 -- 172.21.15.71:0/339131289 >> 172.21.15.71:0/339131289 conn(0x55a8410068c0 msgr2=0x55a84100d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 -- 172.21.15.71:0/339131289 shutdown_connections 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.256+0000 7f0519ed3740 1 -- 172.21.15.71:0/339131289 wait complete. 2024-09-05T12:28:13.261 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.257+0000 7f0519ed3740 1 -- start start 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.257+0000 7f0519ed3740 1 Processor -- start 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.257+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a84102fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.257+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a84102a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0519ed3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84102a9b0 0x55a84102ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0519ed3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a841027160 con 0x55a841015510 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0519ed3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a841026a90 con 0x55a84102a9b0 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0519ed3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a841027830 con 0x55a84100a330 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0519ed3740 10 client.0 _ll_register_callbacks cb 0x7f050409bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a84102a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f050fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a84102a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34654/0 (socket says 172.21.15.71:34654) 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f050fe00640 1 -- 172.21.15.71:0/768004568 learned_addr learned my addr 172.21.15.71:0/768004568 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0514a00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a84102fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0515400640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84102a9b0 0x55a84102ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0514a00640 1 -- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84102a9b0 msgr2=0x55a84102ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0514a00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84102a9b0 0x55a84102ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0514a00640 1 -- 172.21.15.71:0/768004568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 msgr2=0x55a84102a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0514a00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a84102a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f0514a00640 1 -- 172.21.15.71:0/768004568 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a841090580 con 0x55a84100a330 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.258+0000 7f050fe00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a84102a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f0514a00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a84102fae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f050400c6a0 tx=0x7f0504009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be312b7a2379d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f04f6000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f04f6000640 1 -- 172.21.15.71:0/768004568 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050409e040 con 0x55a84100a330 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04fc005520 con 0x55a84100a330 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f04f6000640 1 -- 172.21.15.71:0/768004568 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f0504097b90 con 0x55a84100a330 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f04f6000640 1 -- 172.21.15.71:0/768004568 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0504097dd0 con 0x55a84100a330 2024-09-05T12:28:13.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f0519ed3740 10 client.25453 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.259+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a841099f10 con 0x55a84100a330 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f04f6000640 1 -- 172.21.15.71:0/768004568 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04fc005520 con 0x55a84100a330 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f04f6000640 1 -- 172.21.15.71:0/768004568 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f050409b290 con 0x55a84100a330 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f0519ed3740 10 client.25453 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f0519ed3740 10 client.25453 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f0519ed3740 20 client.25453 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f0519ed3740 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a84109e860 0x55a84109d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f0519ed3740 4 client.25453 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-05T12:28:13.264 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.260+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "status"}) -- 0x55a840f77b30 con 0x55a84109e860 2024-09-05T12:28:13.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.261+0000 7f050fe00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a84109e860 0x55a84109d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:13.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.261+0000 7f050fe00640 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a84109e860 0x55a84109d570 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6044e0ac12e2a0b6 server_cookie=ca92428d665905ae in_seq=0 out_seq=0 2024-09-05T12:28:13.265 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.261+0000 7f04f6000640 10 client.25453 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.261+0000 7f04f6000640 1 -- 172.21.15.71:0/768004568 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a840f77b30 con 0x55a84109e860 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.261+0000 7f04f6000640 10 client.25453 handle_command_reply: tid=0 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.261+0000 7f0519ed3740 1 client.25453 shutdown 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 msgr2=0x55a84102fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a84102fae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f050400c6a0 tx=0x7f0504009700 comp rx=0 tx=0).stop 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 shutdown_connections 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a84109e860 0x55a84109d570 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a84102a9b0 0x55a84102ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a841015510 0x55a84102a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 --2- 172.21.15.71:0/768004568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a84100a330 0x55a84102fae0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 >> 172.21.15.71:0/768004568 conn(0x55a8410068c0 msgr2=0x55a841006ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f04f6000640 0 client.25453 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 shutdown_connections 2024-09-05T12:28:13.266 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 1 -- 172.21.15.71:0/768004568 wait complete. 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.262+0000 7f0519ed3740 20 client.25453 trim_cache size 0 max 16384 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "cluster_fsid": "8c9ae41c-6b80-11ef-bcd6-c7b262605968", 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "whoami": 0, 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "id": 24524, 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "want_state": "up:active", 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "state": "up:active", 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "fs_name": "cephfs", 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "rank_uptime": 202.75382821400001, 2024-09-05T12:28:13.267 INFO:teuthology.orchestra.run.smithi071.stdout: "mdsmap_epoch": 30, 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch": 95, 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_epoch_barrier": 94, 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stdout: "uptime": 216.33004727299999 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04f80a2e90 msgr2=0x7f04f80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04f80a2e90 0x7f04f80a5350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0508010790 tx=0x7f0508004a40 comp rx=0 tx=0).stop 2024-09-05T12:28:13.268 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 msgr2=0x7f051019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 0x7f051019a6e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f050400e9f0 tx=0x7f050400edf0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 shutdown_connections 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0510106f10 0x7f051019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04f80a2e90 0x7f04f80a5350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0510102790 0x7f051019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 --2- 172.21.15.71:0/1205737430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0510101de0 0x7f051019a6e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 >> 172.21.15.71:0/1205737430 conn(0x7f05100fd8c0 msgr2=0x7f05100feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 shutdown_connections 2024-09-05T12:28:13.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:13.263+0000 7f0517400640 1 -- 172.21.15.71:0/1205737430 wait complete. 2024-09-05T12:28:14.096 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-05T12:28:14.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:14 smithi174 ceph-mon[34313]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-09-05T12:28:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:14 smithi071 ceph-mon[29845]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-09-05T12:28:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:14 smithi123 ceph-mon[34321]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 32 op/s 2024-09-05T12:28:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 -- 172.21.15.71:0/3642940731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 msgr2=0x7f83f0104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 --2- 172.21.15.71:0/3642940731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f0104330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009900 tx=0x7f83e002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 -- 172.21.15.71:0/3642940731 shutdown_connections 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 --2- 172.21.15.71:0/3642940731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f01090a0 0x7f83f0110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 --2- 172.21.15.71:0/3642940731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f0104900 0x7f83f0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 --2- 172.21.15.71:0/3642940731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f0104330 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 -- 172.21.15.71:0/3642940731 >> 172.21.15.71:0/3642940731 conn(0x7f83f00ff9e0 msgr2=0x7f83f0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 -- 172.21.15.71:0/3642940731 shutdown_connections 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.058+0000 7f83f5600640 1 -- 172.21.15.71:0/3642940731 wait complete. 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.059+0000 7f83f5600640 1 Processor -- start 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.059+0000 7f83f5600640 1 -- start start 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83f5600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83f5600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 0x7f83f019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83f5600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f019ee50 0x7f83f019f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83f5600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01140e0 con 0x7f83f0103f50 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83f5600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0113f60 con 0x7f83f01090a0 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83f5600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f0114260 con 0x7f83f019ee50 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 0x7f83f019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83ef400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f019ee50 0x7f83f019f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 0x7f83f019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57292/0 (socket says 172.21.15.71:57292) 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83e7e00640 1 -- 172.21.15.71:0/967624003 learned_addr learned my addr 172.21.15.71:0/967624003 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.060+0000 7f83eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34668/0 (socket says 172.21.15.71:34668) 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7e00640 1 -- 172.21.15.71:0/967624003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f019ee50 msgr2=0x7f83f019f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7e00640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f019ee50 0x7f83f019f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7e00640 1 -- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 msgr2=0x7f83f019a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7e00640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f019a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7e00640 1 -- 172.21.15.71:0/967624003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f019f940 con 0x7f83f01090a0 2024-09-05T12:28:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7e00640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 0x7f83f019aad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9c0 tx=0x7f83d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7400640 1 -- 172.21.15.71:0/967624003 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800cd80 con 0x7f83f01090a0 2024-09-05T12:28:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7400640 1 -- 172.21.15.71:0/967624003 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8005500 con 0x7f83f01090a0 2024-09-05T12:28:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f019fc30 con 0x7f83f01090a0 2024-09-05T12:28:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83e7400640 1 -- 172.21.15.71:0/967624003 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8010690 con 0x7f83f01090a0 2024-09-05T12:28:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.061+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f01a7420 con 0x7f83f01090a0 2024-09-05T12:28:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 Processor -- start 2024-09-05T12:28:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 -- start start 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c015c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83e7400640 1 -- 172.21.15.71:0/967624003 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f83d80046c0 con 0x7f83f01090a0 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c0be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c0e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558a00b67100 con 0x558a00c011e0 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558a00b0f1b0 con 0x558a00c0c3a0 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83f85cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a00b630c0 con 0x558a00c01b00 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34684/0 (socket says 172.21.15.71:34684) 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.065+0000 7f83ef400640 1 -- 172.21.15.71:0/3585053004 learned_addr learned my addr 172.21.15.71:0/3585053004 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83e7e00640 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c0e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83eea00640 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c0be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83ef400640 1 -- 172.21.15.71:0/3585053004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 msgr2=0x558a00c0be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83ef400640 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c0be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83ef400640 1 -- 172.21.15.71:0/3585053004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 msgr2=0x558a00c0e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83ef400640 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c0e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83ef400640 1 -- 172.21.15.71:0/3585053004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a0087a210 con 0x558a00c011e0 2024-09-05T12:28:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83e7e00640 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.066+0000 7f83ef400640 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c015c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009f50 tx=0x7f83e80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8270c605e9f05f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.067+0000 7f83df400640 1 -- 172.21.15.71:0/3585053004 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8004d20 con 0x558a00c011e0 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.067+0000 7f83df400640 1 -- 172.21.15.71:0/3585053004 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004050 con 0x558a00c011e0 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.067+0000 7f83df400640 1 -- 172.21.15.71:0/3585053004 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8005260 con 0x558a00c011e0 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.067+0000 7f83e7400640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f83d40a2ec0 0x7f83d40a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.067+0000 7f83e7400640 1 -- 172.21.15.71:0/967624003 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f83d80cfe50 con 0x7f83f01090a0 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.067+0000 7f83eea00640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f83d40a2ec0 0x7f83d40a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 -- 172.21.15.71:0/3585053004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 msgr2=0x558a00c015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c015c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009f50 tx=0x7f83e80118d0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 -- 172.21.15.71:0/3585053004 shutdown_connections 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c0e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c0be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3585053004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c015c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 -- 172.21.15.71:0/3585053004 >> 172.21.15.71:0/3585053004 conn(0x558a00bfd750 msgr2=0x558a00bfdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 -- 172.21.15.71:0/3585053004 shutdown_connections 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 -- 172.21.15.71:0/3585053004 wait complete. 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83eea00640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f83d40a2ec0 0x7f83d40a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009230 tx=0x7f83e00058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 -- start start 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.068+0000 7f83f85cb740 1 Processor -- start 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c09820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c09d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558a00c0efd0 con 0x558a00c011e0 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558a00af0090 con 0x558a00c0c3a0 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a00c0f150 con 0x558a00c01b00 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83f85cb740 10 client.0 _ll_register_callbacks cb 0x558a00b44f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c09d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c09d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57316/0 (socket says 172.21.15.71:57316) 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c09820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.069+0000 7f83ef400640 1 -- 172.21.15.71:0/324850840 learned_addr learned my addr 172.21.15.71:0/324850840 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83e7e00640 1 -- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 msgr2=0x558a00c09d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83e7e00640 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c09d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83e7e00640 1 -- 172.21.15.71:0/324850840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 msgr2=0x558a00c092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83e7e00640 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c092e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83e7e00640 1 -- 172.21.15.71:0/324850840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a00946620 con 0x558a00c01b00 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83e7e00640 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c09820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83d80cd050 tx=0x7f83d80d3610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a289770e53ab47f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83d3e00640 1 -- 172.21.15.71:0/324850840 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d809b8c0 con 0x558a00c01b00 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83d3e00640 1 -- 172.21.15.71:0/324850840 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f83d80cd360 con 0x558a00c01b00 2024-09-05T12:28:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e8004050 con 0x558a00c01b00 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.070+0000 7f83d3e00640 1 -- 172.21.15.71:0/324850840 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83d809b6e0 con 0x558a00c01b00 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 10 client.25465 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a00c1aba0 con 0x558a00c01b00 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83d3e00640 1 -- 172.21.15.71:0/324850840 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004050 con 0x558a00c01b00 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83d3e00640 1 -- 172.21.15.71:0/324850840 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f83d809b1d0 con 0x558a00c01b00 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 10 client.25465 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:15.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 20 client.25465 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c1c060 0x558a00c1e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 4 client.25465 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a00b74830 con 0x558a00c1c060 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.071+0000 7f83eea00640 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c1c060 0x558a00c1e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.072+0000 7f83eea00640 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c1c060 0x558a00c1e450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860e4d9991da4fec server_cookie=51b9957f17268e23 in_seq=0 out_seq=0 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.072+0000 7f83d3e00640 10 client.25465 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83d3e00640 1 -- 172.21.15.71:0/324850840 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a00b74830 con 0x558a00c1c060 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83d3e00640 10 client.25465 handle_command_reply: tid=0 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 client.25465 shutdown 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 msgr2=0x558a00c09820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c09820 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83d80cd050 tx=0x7f83d80d3610 comp rx=0 tx=0).stop 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 shutdown_connections 2024-09-05T12:28:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c1c060 0x558a00c1e450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c0c3a0 0x558a00c09d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83d3e00640 0 client.25465 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c01b00 0x558a00c09820 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 --2- 172.21.15.71:0/324850840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c011e0 0x558a00c092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 >> 172.21.15.71:0/324850840 conn(0x558a00bfd750 msgr2=0x558a00c03460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 shutdown_connections 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 1 -- 172.21.15.71:0/324850840 wait complete. 2024-09-05T12:28:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.073+0000 7f83f85cb740 20 client.25465 trim_cache size 0 max 16384 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.088+0000 7f83f85cb740 1 Processor -- start 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.088+0000 7f83f85cb740 1 -- start start 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c011e0 0x558a00bf88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c01b00 0x558a00c03350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c0c3a0 0x558a00c23d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83f85cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b1010 con 0x558a00c01b00 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83f85cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b2b50 con 0x558a00c011e0 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83f85cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b16e0 con 0x558a00c0c3a0 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c01b00 0x558a00c03350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83ef400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c0c3a0 0x558a00c23d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83eea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c011e0 0x558a00bf88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c01b00 0x558a00c03350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34702/0 (socket says 172.21.15.71:34702) 2024-09-05T12:28:15.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83e7e00640 1 -- 172.21.15.71:0/1767549322 learned_addr learned my addr 172.21.15.71:0/1767549322 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83ef400640 1 -- 172.21.15.71:0/1767549322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c011e0 msgr2=0x558a00bf88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83ef400640 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c011e0 0x558a00bf88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83ef400640 1 -- 172.21.15.71:0/1767549322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c01b00 msgr2=0x558a00c03350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83ef400640 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c01b00 0x558a00c03350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83ef400640 1 -- 172.21.15.71:0/1767549322 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a00bfe800 con 0x558a00c0c3a0 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.089+0000 7f83eea00640 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c011e0 0x558a00bf88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.090+0000 7f83ef400640 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c0c3a0 0x558a00c23d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009f20 tx=0x7f83e8009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10612a3c74f18c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.090+0000 7f83dea00640 1 -- 172.21.15.71:0/1767549322 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8027030 con 0x558a00c0c3a0 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.090+0000 7f83dea00640 1 -- 172.21.15.71:0/1767549322 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004050 con 0x558a00c0c3a0 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.090+0000 7f83dea00640 1 -- 172.21.15.71:0/1767549322 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8004d00 con 0x558a00c0c3a0 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 -- 172.21.15.71:0/1767549322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c0c3a0 msgr2=0x558a00c23d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c0c3a0 0x558a00c23d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009f20 tx=0x7f83e8009760 comp rx=0 tx=0).stop 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 -- 172.21.15.71:0/1767549322 shutdown_connections 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c0c3a0 0x558a00c23d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c01b00 0x558a00c03350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 --2- 172.21.15.71:0/1767549322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c011e0 0x558a00bf88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 -- 172.21.15.71:0/1767549322 >> 172.21.15.71:0/1767549322 conn(0x558a00bfd750 msgr2=0x558a00c02c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 -- 172.21.15.71:0/1767549322 shutdown_connections 2024-09-05T12:28:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 -- 172.21.15.71:0/1767549322 wait complete. 2024-09-05T12:28:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.091+0000 7f83f85cb740 1 -- start start 2024-09-05T12:28:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 Processor -- start 2024-09-05T12:28:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c011e0 0x558a00c14800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 0x558a00c05ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 0x558a00c063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b2480 con 0x558a00c0c3a0 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01b1db0 con 0x558a00c01b00 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f01aeff0 con 0x558a00c011e0 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83f85cb740 10 client.0 _ll_register_callbacks cb 0x7f83d8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 0x558a00c063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83eea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c011e0 0x558a00c14800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 0x558a00c05ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83ef400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 0x558a00c063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34712/0 (socket says 172.21.15.71:34712) 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83eea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c011e0 0x558a00c14800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35062/0 (socket says 172.21.15.71:35062) 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83eea00640 1 -- 172.21.15.71:0/3508481460 learned_addr learned my addr 172.21.15.71:0/3508481460 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.092+0000 7f83e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 0x558a00c05ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57346/0 (socket says 172.21.15.71:57346) 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83e7e00640 1 -- 172.21.15.71:0/3508481460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c011e0 msgr2=0x558a00c14800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83e7e00640 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c011e0 0x558a00c14800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83e7e00640 1 -- 172.21.15.71:0/3508481460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 msgr2=0x558a00c063e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83e7e00640 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 0x558a00c063e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83e7e00640 1 -- 172.21.15.71:0/3508481460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a00c06a50 con 0x558a00c01b00 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83ef400640 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 0x558a00c063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83e7e00640 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 0x558a00c05ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x558a00c0c0a0 tx=0x7f83d809bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c32b3a97897f9a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83d3e00640 1 -- 172.21.15.71:0/3508481460 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d809b8c0 con 0x558a00c01b00 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83d3e00640 1 -- 172.21.15.71:0/3508481460 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f83d8097ad0 con 0x558a00c01b00 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e8004050 con 0x558a00c01b00 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83d3e00640 1 -- 172.21.15.71:0/3508481460 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f83d80ccfc0 con 0x558a00c01b00 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83f85cb740 10 client.25475 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.093+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558a00c86b90 con 0x558a00c01b00 2024-09-05T12:28:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83d3e00640 1 -- 172.21.15.71:0/3508481460 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e8004050 con 0x558a00c01b00 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83d3e00640 1 -- 172.21.15.71:0/3508481460 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f83d8097cd0 con 0x558a00c01b00 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83f85cb740 10 client.25475 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83f85cb740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83f85cb740 20 client.25475 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c871b0 0x558a00c895d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83f85cb740 4 client.25475 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a00b6eb80 con 0x558a00c871b0 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.094+0000 7f83eea00640 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c871b0 0x558a00c895d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.095+0000 7f83eea00640 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c871b0 0x558a00c895d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=384bff32298f5925 server_cookie=73b9dc268f631594 in_seq=0 out_seq=0 2024-09-05T12:28:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.095+0000 7f83d3e00640 10 client.25475 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83d3e00640 1 -- 172.21.15.71:0/3508481460 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a00b6eb80 con 0x558a00c871b0 2024-09-05T12:28:15.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83d3e00640 10 client.25475 handle_command_reply: tid=0 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 client.25475 shutdown 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 msgr2=0x558a00c05ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 0x558a00c05ea0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x558a00c0c0a0 tx=0x7f83d809bc80 comp rx=0 tx=0).stop 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 shutdown_connections 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a00c871b0 0x558a00c895d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a00c0c3a0 0x558a00c063e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a00c01b00 0x558a00c05ea0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 --2- 172.21.15.71:0/3508481460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a00c011e0 0x558a00c14800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83d3e00640 0 client.25475 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:15.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 >> 172.21.15.71:0/3508481460 conn(0x558a00bfd750 msgr2=0x558a00c09ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.103+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 shutdown_connections 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f85cb740 1 -- 172.21.15.71:0/3508481460 wait complete. 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f85cb740 20 client.25475 trim_cache size 0 max 16384 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:28:15.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f83d40a2ec0 msgr2=0x7f83d40a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f83d40a2ec0 0x7f83d40a5380 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009230 tx=0x7f83e00058d0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 msgr2=0x7f83f019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 0x7f83f019aad0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9c0 tx=0x7f83d800ee80 comp rx=0 tx=0).stop 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 shutdown_connections 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f019ee50 0x7f83f019f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f01090a0 0x7f83f019aad0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f83d40a2ec0 0x7f83d40a5380 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 --2- 172.21.15.71:0/967624003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83f0103f50 0x7f83f019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.104+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 >> 172.21.15.71:0/967624003 conn(0x7f83f00ff9e0 msgr2=0x7f83f0100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.105+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 shutdown_connections 2024-09-05T12:28:15.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:15.105+0000 7f83f5600640 1 -- 172.21.15.71:0/967624003 wait complete. 2024-09-05T12:28:16.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:16 smithi071 ceph-mon[29845]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.8 KiB/s wr, 18 op/s 2024-09-05T12:28:16.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:16 smithi123 ceph-mon[34321]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.8 KiB/s wr, 18 op/s 2024-09-05T12:28:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:16 smithi174 ceph-mon[34313]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 2.8 KiB/s wr, 18 op/s 2024-09-05T12:28:18.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:18 smithi071 ceph-mon[29845]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 24 op/s 2024-09-05T12:28:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:18 smithi174 ceph-mon[34313]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 24 op/s 2024-09-05T12:28:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:18 smithi123 ceph-mon[34321]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.0 KiB/s wr, 24 op/s 2024-09-05T12:28:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:20 smithi174 ceph-mon[34313]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:20.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:20 smithi071 ceph-mon[29845]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:20 smithi123 ceph-mon[34321]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:22 smithi174 ceph-mon[34313]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:22.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:22 smithi071 ceph-mon[29845]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:22 smithi123 ceph-mon[34321]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:24 smithi174 ceph-mon[34313]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:24 smithi071 ceph-mon[29845]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:24 smithi123 ceph-mon[34321]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:26 smithi174 ceph-mon[34313]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:26.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:26 smithi071 ceph-mon[29845]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:26 smithi123 ceph-mon[34321]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-05T12:28:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 -- 172.21.15.71:0/1931863628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41018b0 msgr2=0x7f37c4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1931863628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41018b0 0x7f37c4101d10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009900 tx=0x7f37b002a400 comp rx=0 tx=0).stop 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 -- 172.21.15.71:0/1931863628 shutdown_connections 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1931863628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c4102250 0x7f37c410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1931863628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41018b0 0x7f37c4101d10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1931863628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c41078c0 0x7f37c4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.286+0000 7f37c3e00640 1 -- 172.21.15.71:0/1931863628 >> 172.21.15.71:0/1931863628 conn(0x7f37c40fd5d0 msgr2=0x7f37c40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.287+0000 7f37c3e00640 1 -- 172.21.15.71:0/1931863628 shutdown_connections 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.287+0000 7f37c3e00640 1 -- 172.21.15.71:0/1931863628 wait complete. 2024-09-05T12:28:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.288+0000 7f37c3e00640 1 Processor -- start 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.288+0000 7f37c3e00640 1 -- start start 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c41018b0 0x7f37c4198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c4102250 0x7f37c41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 0x7f37c419cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111d50 con 0x7f37c4102250 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111bd0 con 0x7f37c41078c0 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4111ed0 con 0x7f37c41018b0 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c41018b0 0x7f37c4198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 0x7f37c419cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 0x7f37c419cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35664/0 (socket says 172.21.15.71:35664) 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c3400640 1 -- 172.21.15.71:0/1059420917 learned_addr learned my addr 172.21.15.71:0/1059420917 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.289+0000 7f37c2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c4102250 0x7f37c41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3400640 1 -- 172.21.15.71:0/1059420917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c41018b0 msgr2=0x7f37c4198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3400640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c41018b0 0x7f37c4198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3400640 1 -- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c4102250 msgr2=0x7f37c41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3400640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c4102250 0x7f37c41987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3400640 1 -- 172.21.15.71:0/1059420917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c419d220 con 0x7f37c41078c0 2024-09-05T12:28:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3400640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 0x7f37c419cb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009cf0 tx=0x7f37bc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37bb400640 1 -- 172.21.15.71:0/1059420917 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc003b70 con 0x7f37c41078c0 2024-09-05T12:28:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37bb400640 1 -- 172.21.15.71:0/1059420917 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37bc003d10 con 0x7f37c41078c0 2024-09-05T12:28:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.290+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c419d510 con 0x7f37c41078c0 2024-09-05T12:28:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.291+0000 7f37bb400640 1 -- 172.21.15.71:0/1059420917 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0109f0 con 0x7f37c41078c0 2024-09-05T12:28:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.291+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c406bc10 con 0x7f37c41078c0 2024-09-05T12:28:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.295+0000 7f37cc3d9740 1 Processor -- start 2024-09-05T12:28:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.295+0000 7f37bb400640 1 -- 172.21.15.71:0/1059420917 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f37bc010b90 con 0x7f37c41078c0 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 -- start start 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 0x557d6241afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d62425840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d62428170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557d62378100 con 0x557d6241b4e0 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d623201b0 con 0x557d6241abc0 2024-09-05T12:28:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.296+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d623740c0 con 0x557d62425d80 2024-09-05T12:28:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d62428170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 0x557d6241afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 0x557d6241afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35672/0 (socket says 172.21.15.71:35672) 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 -- 172.21.15.71:0/1903522904 learned_addr learned my addr 172.21.15.71:0/1903522904 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2a00640 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d62425840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 -- 172.21.15.71:0/1903522904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 msgr2=0x557d62428170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d62428170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 -- 172.21.15.71:0/1903522904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 msgr2=0x557d62425840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d62425840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.297+0000 7f37c2000640 1 -- 172.21.15.71:0/1903522904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6208b210 con 0x557d6241abc0 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37c2000640 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 0x557d6241afa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37b0005ec0 tx=0x7f37b0005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b540c46a893e0d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37bb400640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37a80a2e90 0x7f37a80a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37bb400640 1 -- 172.21.15.71:0/1059420917 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f37bc0cfbf0 con 0x7f37c41078c0 2024-09-05T12:28:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37af400640 1 -- 172.21.15.71:0/1903522904 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0035040 con 0x557d6241abc0 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37af400640 1 -- 172.21.15.71:0/1903522904 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b0031c20 con 0x557d6241abc0 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37af400640 1 -- 172.21.15.71:0/1903522904 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b003dca0 con 0x557d6241abc0 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.298+0000 7f37c2a00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37a80a2e90 0x7f37a80a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1903522904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 msgr2=0x557d6241afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 0x557d6241afa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37b0005ec0 tx=0x7f37b0005a20 comp rx=0 tx=0).stop 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37c2a00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37a80a2e90 0x7f37a80a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37b4010d60 tx=0x7f37b4012040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1903522904 shutdown_connections 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d62428170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d62425840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.299+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1903522904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d6241abc0 0x557d6241afa0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.300+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1903522904 >> 172.21.15.71:0/1903522904 conn(0x557d62417130 msgr2=0x557d62417510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.300+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1903522904 shutdown_connections 2024-09-05T12:28:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.300+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1903522904 wait complete. 2024-09-05T12:28:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.300+0000 7f37cc3d9740 1 -- start start 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 Processor -- start 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241abc0 0x557d624141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d624146e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d62424830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557d6240e590 con 0x557d6241abc0 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d62301090 con 0x557d62425d80 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d6240e710 con 0x557d6241b4e0 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.301+0000 7f37cc3d9740 10 client.0 _ll_register_callbacks cb 0x557d62355f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d624146e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241abc0 0x557d624141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d624146e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:41316/0 (socket says 172.21.15.71:41316) 2024-09-05T12:28:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2000640 1 -- 172.21.15.71:0/1297232153 learned_addr learned my addr 172.21.15.71:0/1297232153 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:28.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c3400640 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d62424830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2a00640 1 -- 172.21.15.71:0/1297232153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 msgr2=0x557d624146e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2a00640 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d624146e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2a00640 1 -- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 msgr2=0x557d62424830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2a00640 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d62424830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.302+0000 7f37c2a00640 1 -- 172.21.15.71:0/1297232153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d62157620 con 0x557d6241abc0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.303+0000 7f37c2a00640 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241abc0 0x557d624141a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f37b40106b0 tx=0x7f37b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8e7e9921fcf90b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.303+0000 7f37a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.303+0000 7f37a3e00640 1 -- 172.21.15.71:0/1297232153 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4017050 con 0x557d6241abc0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.303+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b0031c20 con 0x557d6241abc0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.303+0000 7f37a3e00640 1 -- 172.21.15.71:0/1297232153 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f37b401c630 con 0x557d6241abc0 2024-09-05T12:28:28.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.304+0000 7f37a3e00640 1 -- 172.21.15.71:0/1297232153 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37b401c830 con 0x557d6241abc0 2024-09-05T12:28:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.304+0000 7f37cc3d9740 10 client.15432 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.304+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d6243cef0 con 0x557d6241abc0 2024-09-05T12:28:28.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.304+0000 7f37a3e00640 1 -- 172.21.15.71:0/1297232153 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b0031c20 con 0x557d6241abc0 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.304+0000 7f37a3e00640 1 -- 172.21.15.71:0/1297232153 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f37b4026bb0 con 0x557d6241abc0 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37cc3d9740 10 client.15432 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37cc3d9740 10 client.15432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37cc3d9740 20 client.15432 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d6243e3b0 0x557d624407e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37cc3d9740 4 client.15432 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d62385830 con 0x557d6243e3b0 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.305+0000 7f37c2000640 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d6243e3b0 0x557d624407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.306+0000 7f37c2000640 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d6243e3b0 0x557d624407e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da266f1c39fb23ff server_cookie=a643cbd2c0e85a15 in_seq=0 out_seq=0 2024-09-05T12:28:28.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.306+0000 7f37a3e00640 10 client.15432 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:28.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37a3e00640 1 -- 172.21.15.71:0/1297232153 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d62385830 con 0x557d6243e3b0 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37a3e00640 10 client.15432 handle_command_reply: tid=0 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 client.15432 shutdown 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241abc0 msgr2=0x557d624141a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241abc0 0x557d624141a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f37b40106b0 tx=0x7f37b4004730 comp rx=0 tx=0).stop 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 shutdown_connections 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d6243e3b0 0x557d624407e0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d62424830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d624146e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/1297232153 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241abc0 0x557d624141a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 >> 172.21.15.71:0/1297232153 conn(0x557d62417130 msgr2=0x557d6241ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.307+0000 7f37a3e00640 0 client.15432 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:28.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.308+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 shutdown_connections 2024-09-05T12:28:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.308+0000 7f37cc3d9740 1 -- 172.21.15.71:0/1297232153 wait complete. 2024-09-05T12:28:28.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.308+0000 7f37cc3d9740 20 client.15432 trim_cache size 0 max 16384 2024-09-05T12:28:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 Processor -- start 2024-09-05T12:28:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 -- start start 2024-09-05T12:28:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d6240d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d624460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62424990 0x557d62446630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4075af0 con 0x557d6241b4e0 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41b6de0 con 0x557d62424990 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37c40761c0 con 0x557d62425d80 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.338+0000 7f37c2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d624460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d624460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:41320/0 (socket says 172.21.15.71:41320) 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 -- 172.21.15.71:0/2472387252 learned_addr learned my addr 172.21.15.71:0/2472387252 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2a00640 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d6240d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 -- 172.21.15.71:0/2472387252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62424990 msgr2=0x557d62446630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62424990 0x557d62446630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 -- 172.21.15.71:0/2472387252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 msgr2=0x557d6240d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d6240d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2000640 1 -- 172.21.15.71:0/2472387252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d62416880 con 0x557d62425d80 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.339+0000 7f37c2a00640 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d6240d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.340+0000 7f37c2000640 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d624460f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37b0005ec0 tx=0x7f37b003d880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3670224d207e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.340+0000 7f37aea00640 1 -- 172.21.15.71:0/2472387252 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0035040 con 0x557d62425d80 2024-09-05T12:28:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.340+0000 7f37aea00640 1 -- 172.21.15.71:0/2472387252 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b003dc80 con 0x557d62425d80 2024-09-05T12:28:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.340+0000 7f37aea00640 1 -- 172.21.15.71:0/2472387252 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0051610 con 0x557d62425d80 2024-09-05T12:28:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 -- 172.21.15.71:0/2472387252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 msgr2=0x557d624460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d624460f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37b0005ec0 tx=0x7f37b003d880 comp rx=0 tx=0).stop 2024-09-05T12:28:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 -- 172.21.15.71:0/2472387252 shutdown_connections 2024-09-05T12:28:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62424990 0x557d62446630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d62425d80 0x557d624460f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/2472387252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d6241b4e0 0x557d6240d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 -- 172.21.15.71:0/2472387252 >> 172.21.15.71:0/2472387252 conn(0x557d62417130 msgr2=0x557d6240d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.341+0000 7f37cc3d9740 1 -- 172.21.15.71:0/2472387252 shutdown_connections 2024-09-05T12:28:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.342+0000 7f37cc3d9740 1 -- 172.21.15.71:0/2472387252 wait complete. 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.342+0000 7f37cc3d9740 1 -- start start 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.342+0000 7f37cc3d9740 1 Processor -- start 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d62421d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 0x557d6241f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d6241f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41b6710 con 0x557d62424990 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41b6040 con 0x557d62425d80 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4073ad0 con 0x557d6241b4e0 2024-09-05T12:28:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37cc3d9740 10 client.0 _ll_register_callbacks cb 0x7f37b40044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.343+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d6241f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 0x557d6241f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 0x557d6241f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33972/0 (socket says 172.21.15.71:33972) 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 -- 172.21.15.71:0/3446111854 learned_addr learned my addr 172.21.15.71:0/3446111854 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d6241f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35698/0 (socket says 172.21.15.71:35698) 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2a00640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d62421d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 -- 172.21.15.71:0/3446111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 msgr2=0x557d62421d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d62421d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 -- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 msgr2=0x557d6241f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d6241f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 -- 172.21.15.71:0/3446111854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6241fde0 con 0x557d62424990 2024-09-05T12:28:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c3400640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d6241f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2a00640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d62421d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.344+0000 7f37c2000640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 0x557d6241f0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009850 tx=0x7f37b002a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99584d77832cf3e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.345+0000 7f37a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.345+0000 7f37a3e00640 1 -- 172.21.15.71:0/3446111854 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0035040 con 0x557d62424990 2024-09-05T12:28:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.345+0000 7f37a3e00640 1 -- 172.21.15.71:0/3446111854 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f37b0031540 con 0x557d62424990 2024-09-05T12:28:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.345+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b003dc80 con 0x557d62424990 2024-09-05T12:28:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.345+0000 7f37a3e00640 1 -- 172.21.15.71:0/3446111854 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f37b0064050 con 0x557d62424990 2024-09-05T12:28:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 10 client.15438 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d624a9050 con 0x557d62424990 2024-09-05T12:28:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37a3e00640 1 -- 172.21.15.71:0/3446111854 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37b003dc80 con 0x557d62424990 2024-09-05T12:28:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37a3e00640 1 -- 172.21.15.71:0/3446111854 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f37b005b9b0 con 0x557d62424990 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 10 client.15438 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 20 client.15438 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d624a9280 0x557d624ab670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 4 client.15438 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.346+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d6237fb80 con 0x557d624a9280 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.347+0000 7f37c2a00640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d624a9280 0x557d624ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.347+0000 7f37c2a00640 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d624a9280 0x557d624ab670 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c558ff111642c8d5 server_cookie=2062d525efa471db in_seq=0 out_seq=0 2024-09-05T12:28:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.347+0000 7f37a3e00640 10 client.15438 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37a3e00640 1 -- 172.21.15.71:0/3446111854 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d6237fb80 con 0x557d624a9280 2024-09-05T12:28:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37a3e00640 10 client.15438 handle_command_reply: tid=0 2024-09-05T12:28:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 client.15438 shutdown 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 msgr2=0x557d6241f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 0x557d6241f0a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009850 tx=0x7f37b002a400 comp rx=0 tx=0).stop 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 shutdown_connections 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d624a9280 0x557d624ab670 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d62425d80 0x557d6241f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.351+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d62424990 0x557d6241f0a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.352+0000 7f37a3e00640 0 client.15438 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.352+0000 7f37cc3d9740 1 --2- 172.21.15.71:0/3446111854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d6241b4e0 0x557d62421d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.352+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 >> 172.21.15.71:0/3446111854 conn(0x557d62417130 msgr2=0x557d6240d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.352+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 shutdown_connections 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.352+0000 7f37cc3d9740 1 -- 172.21.15.71:0/3446111854 wait complete. 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.352+0000 7f37cc3d9740 20 client.15438 trim_cache size 0 max 16384 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:28:28.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.353+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37a80a2e90 msgr2=0x7f37a80a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.353+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37a80a2e90 0x7f37a80a5350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f37b4010d60 tx=0x7f37b4012040 comp rx=0 tx=0).stop 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.353+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 msgr2=0x7f37c419cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.353+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 0x7f37c419cb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009cf0 tx=0x7f37bc00c6a0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 shutdown_connections 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c41078c0 0x7f37c419cb20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37a80a2e90 0x7f37a80a5350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37c4102250 0x7f37c41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 --2- 172.21.15.71:0/1059420917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c41018b0 0x7f37c4198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 >> 172.21.15.71:0/1059420917 conn(0x7f37c40fd5d0 msgr2=0x7f37c40fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 shutdown_connections 2024-09-05T12:28:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:28.354+0000 7f37c3e00640 1 -- 172.21.15.71:0/1059420917 wait complete. 2024-09-05T12:28:28.506 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:28 smithi071 ceph-mon[29845]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:28 smithi174 ceph-mon[34313]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:28 smithi123 ceph-mon[34321]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:29.098 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.0 2024-09-05T12:28:29.098 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:28:29.691 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:30.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.359+0000 7f067f200640 1 -- 172.21.15.71:0/4172992464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0680102b90 msgr2=0x7f0680102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.359+0000 7f067f200640 1 --2- 172.21.15.71:0/4172992464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0680102b90 0x7f0680102f70 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0670009900 tx=0x7f067002a490 comp rx=0 tx=0).stop 2024-09-05T12:28:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.359+0000 7f067f200640 1 -- 172.21.15.71:0/4172992464 shutdown_connections 2024-09-05T12:28:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.360+0000 7f067f200640 1 --2- 172.21.15.71:0/4172992464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f068010bfc0 0x7f068010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.360+0000 7f067f200640 1 --2- 172.21.15.71:0/4172992464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06801034b0 0x7f068010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.360+0000 7f067f200640 1 --2- 172.21.15.71:0/4172992464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0680102b90 0x7f0680102f70 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.360+0000 7f067f200640 1 -- 172.21.15.71:0/4172992464 >> 172.21.15.71:0/4172992464 conn(0x7f06800fc610 msgr2=0x7f06800fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.360+0000 7f067f200640 1 -- 172.21.15.71:0/4172992464 shutdown_connections 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.360+0000 7f067f200640 1 -- 172.21.15.71:0/4172992464 wait complete. 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.361+0000 7f067f200640 1 Processor -- start 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.361+0000 7f067f200640 1 -- start start 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.362+0000 7f067f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06801034b0 0x7f068019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.362+0000 7f067f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068010bfc0 0x7f068010acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:30.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.362+0000 7f067f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068010b1f0 0x7f068010b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:30.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.362+0000 7f067f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0680111e00 con 0x7f068010bfc0 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.363+0000 7f067de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06801034b0 0x7f068019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.363+0000 7f067de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06801034b0 0x7f068019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:41344/0 (socket says 172.21.15.71:41344) 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.363+0000 7f067de00640 1 -- 172.21.15.71:0/2075558089 learned_addr learned my addr 172.21.15.71:0/2075558089 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.363+0000 7f067e800640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068010b1f0 0x7f068010b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.363+0000 7f067d400640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068010bfc0 0x7f068010acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.362+0000 7f067f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0680111c80 con 0x7f068010b1f0 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.363+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0680111f80 con 0x7f06801034b0 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f067e800640 1 -- 172.21.15.71:0/2075558089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06801034b0 msgr2=0x7f068019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f067e800640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06801034b0 0x7f068019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f067e800640 1 -- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068010bfc0 msgr2=0x7f068010acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f067e800640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068010bfc0 0x7f068010acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f067e800640 1 -- 172.21.15.71:0/2075558089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068019ab30 con 0x7f068010b1f0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f067e800640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068010b1f0 0x7f068010b6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f067800be20 tx=0x7f067800bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.364+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0678019070 con 0x7f068010b1f0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.365+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f068019ae20 con 0x7f068010b1f0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.365+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0678014920 con 0x7f068010b1f0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.365+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06780128d0 con 0x7f068010b1f0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.365+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f068019b360 con 0x7f068010b1f0 2024-09-05T12:28:30.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.367+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068006a850 con 0x7f068010b1f0 2024-09-05T12:28:30.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.369+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f067801c070 con 0x7f068010b1f0 2024-09-05T12:28:30.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.372+0000 7f0676a00640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06640a2e90 0x7f06640a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:30.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.372+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f06780cf1a0 con 0x7f068010b1f0 2024-09-05T12:28:30.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.372+0000 7f067de00640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06640a2e90 0x7f06640a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:30.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.373+0000 7f067de00640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06640a2e90 0x7f06640a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f06700096c0 tx=0x7f06700058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:30.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.375+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f067809b290 con 0x7f068010b1f0 2024-09-05T12:28:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:30 smithi174 ceph-mon[34313]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:28:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:30 smithi071 ceph-mon[29845]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:28:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:30 smithi123 ceph-mon[34321]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:28:30.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.816+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f068019b620 con 0x7f068010b1f0 2024-09-05T12:28:30.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.817+0000 7f0676a00640 1 -- 172.21.15.71:0/2075558089 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11968 (secure 0 0 0) 0x7f06780b9e20 con 0x7f068010b1f0 2024-09-05T12:28:30.819 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:28:30.819 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-05T12:27:44:191847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:27:44.168192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:28:30.820 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-05T12:28:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06640a2e90 msgr2=0x7f06640a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06640a2e90 0x7f06640a5350 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f06700096c0 tx=0x7f06700058d0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068010b1f0 msgr2=0x7f068010b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068010b1f0 0x7f068010b6a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f067800be20 tx=0x7f067800bf10 comp rx=0 tx=0).stop 2024-09-05T12:28:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 shutdown_connections 2024-09-05T12:28:30.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f068010b1f0 0x7f068010b6a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06640a2e90 0x7f06640a5350 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f068010bfc0 0x7f068010acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 --2- 172.21.15.71:0/2075558089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06801034b0 0x7f068019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.820+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 >> 172.21.15.71:0/2075558089 conn(0x7f06800fc610 msgr2=0x7f0680100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.821+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 shutdown_connections 2024-09-05T12:28:30.825 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:30.821+0000 7f067f200640 1 -- 172.21.15.71:0/2075558089 wait complete. 2024-09-05T12:28:31.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:31 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2075558089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:31.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:31 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2075558089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:31.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:31 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2075558089' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:31.738 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 138.0 --await-for 45.0' 2024-09-05T12:28:31.738 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 138.0 --await-for 45.0 2024-09-05T12:28:32.187 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:32 smithi174 ceph-mon[34313]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:28:32.522 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:32 smithi071 ceph-mon[29845]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:28:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:32 smithi123 ceph-mon[34321]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-05T12:28:32.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.892+0000 7fdb5ce00640 1 -- 172.21.15.71:0/1541717814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 msgr2=0x7fdb58107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.892+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/1541717814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb58107da0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009930 tx=0x7fdb4802a420 comp rx=0 tx=0).stop 2024-09-05T12:28:32.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 -- 172.21.15.71:0/1541717814 shutdown_connections 2024-09-05T12:28:32.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/1541717814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 0x7fdb5810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/1541717814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb581019c0 0x7fdb58101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/1541717814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb58107da0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 -- 172.21.15.71:0/1541717814 >> 172.21.15.71:0/1541717814 conn(0x7fdb580fd740 msgr2=0x7fdb580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:32.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 -- 172.21.15.71:0/1541717814 shutdown_connections 2024-09-05T12:28:32.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.893+0000 7fdb5ce00640 1 -- 172.21.15.71:0/1541717814 wait complete. 2024-09-05T12:28:32.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.894+0000 7fdb5ce00640 1 Processor -- start 2024-09-05T12:28:32.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.894+0000 7fdb5ce00640 1 -- start start 2024-09-05T12:28:32.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.895+0000 7fdb5ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb581019c0 0x7fdb5819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.895+0000 7fdb5ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 0x7fdb5819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.895+0000 7fdb5ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb5819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.897 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.895+0000 7fdb5ce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58111dc0 con 0x7fdb581019c0 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.895+0000 7fdb5ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58111c40 con 0x7fdb58102360 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.895+0000 7fdb5ce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb58111f40 con 0x7fdb581079c0 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb56a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 0x7fdb5819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb57e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb5819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb57e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb5819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50844/0 (socket says 172.21.15.71:50844) 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb57e00640 1 -- 172.21.15.71:0/939105614 learned_addr learned my addr 172.21.15.71:0/939105614 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:32.898 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb57400640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb581019c0 0x7fdb5819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb56a00640 1 -- 172.21.15.71:0/939105614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 msgr2=0x7fdb5819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb56a00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb5819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb56a00640 1 -- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb581019c0 msgr2=0x7fdb5819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb56a00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb581019c0 0x7fdb5819a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.896+0000 7fdb56a00640 1 -- 172.21.15.71:0/939105614 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5819f4b0 con 0x7fdb58102360 2024-09-05T12:28:32.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.897+0000 7fdb56a00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 0x7fdb5819aa40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400e9f0 tx=0x7fdb4400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:32.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.897+0000 7fdb4fe00640 1 -- 172.21.15.71:0/939105614 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400cd80 con 0x7fdb58102360 2024-09-05T12:28:32.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.898+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5819f7a0 con 0x7fdb58102360 2024-09-05T12:28:32.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.898+0000 7fdb4fe00640 1 -- 172.21.15.71:0/939105614 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb440035f0 con 0x7fdb58102360 2024-09-05T12:28:32.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.898+0000 7fdb4fe00640 1 -- 172.21.15.71:0/939105614 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44010670 con 0x7fdb58102360 2024-09-05T12:28:32.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.898+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb581a7080 con 0x7fdb58102360 2024-09-05T12:28:32.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.902+0000 7fdb60e88740 1 Processor -- start 2024-09-05T12:28:32.904 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.902+0000 7fdb60e88740 1 -- start start 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe078560 0x55e2fe078940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078e80 0x55e2fe083200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe083740 0x55e2fe085b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb60e88740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e2fdfde530 con 0x55e2fe078560 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb60e88740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2fdf85860 con 0x55e2fe083740 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb60e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2fdfda4f0 con 0x55e2fe078e80 2024-09-05T12:28:32.905 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb57400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078e80 0x55e2fe083200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb57e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe078560 0x55e2fe078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb57e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe078560 0x55e2fe078940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58740/0 (socket says 172.21.15.71:58740) 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.903+0000 7fdb57e00640 1 -- 172.21.15.71:0/300825882 learned_addr learned my addr 172.21.15.71:0/300825882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.904+0000 7fdb57400640 1 -- 172.21.15.71:0/300825882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe083740 msgr2=0x55e2fe085b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.904+0000 7fdb57400640 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe083740 0x55e2fe085b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.904+0000 7fdb57400640 1 -- 172.21.15.71:0/300825882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe078560 msgr2=0x55e2fe078940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.904+0000 7fdb57400640 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe078560 0x55e2fe078940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.904+0000 7fdb57400640 1 -- 172.21.15.71:0/300825882 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2fdd4fb90 con 0x55e2fe078e80 2024-09-05T12:28:32.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.904+0000 7fdb57400640 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078e80 0x55e2fe083200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009900 tx=0x7fdb48005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2409d50ef2deeb9a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:32.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.905+0000 7fdb4cc00640 1 -- 172.21.15.71:0/300825882 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4803f070 con 0x55e2fe078e80 2024-09-05T12:28:32.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.905+0000 7fdb4cc00640 1 -- 172.21.15.71:0/300825882 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb48004030 con 0x55e2fe078e80 2024-09-05T12:28:32.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.905+0000 7fdb4cc00640 1 -- 172.21.15.71:0/300825882 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48035070 con 0x55e2fe078e80 2024-09-05T12:28:32.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.905+0000 7fdb4fe00640 1 -- 172.21.15.71:0/939105614 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fdb440037a0 con 0x7fdb58102360 2024-09-05T12:28:32.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 -- 172.21.15.71:0/300825882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078e80 msgr2=0x55e2fe083200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078e80 0x55e2fe083200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb48009900 tx=0x7fdb48005dd0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 -- 172.21.15.71:0/300825882 shutdown_connections 2024-09-05T12:28:32.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe083740 0x55e2fe085b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078e80 0x55e2fe083200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 --2- 172.21.15.71:0/300825882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe078560 0x55e2fe078940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.906+0000 7fdb60e88740 1 -- 172.21.15.71:0/300825882 >> 172.21.15.71:0/300825882 conn(0x55e2fe074af0 msgr2=0x55e2fe074ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:32.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.907+0000 7fdb4fe00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdb3c0a2f10 0x7fdb3c0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.907+0000 7fdb4fe00640 1 -- 172.21.15.71:0/939105614 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fdb440d0030 con 0x7fdb58102360 2024-09-05T12:28:32.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.908+0000 7fdb60e88740 1 -- 172.21.15.71:0/300825882 shutdown_connections 2024-09-05T12:28:32.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.908+0000 7fdb60e88740 1 -- 172.21.15.71:0/300825882 wait complete. 2024-09-05T12:28:32.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.908+0000 7fdb57400640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdb3c0a2f10 0x7fdb3c0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.909+0000 7fdb60e88740 1 -- start start 2024-09-05T12:28:32.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.909+0000 7fdb60e88740 1 Processor -- start 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb57400640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdb3c0a2f10 0x7fdb3c0a53d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb480096c0 tx=0x7fdb48000bc0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe080870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe080db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe097f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e2fe08eac0 con 0x55e2fe083740 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2fdf98e60 con 0x55e2fe0812f0 2024-09-05T12:28:32.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2fe08ec40 con 0x55e2fe078560 2024-09-05T12:28:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb60e88740 10 client.0 _ll_register_callbacks cb 0x55e2fdf9dcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb57400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe080870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb57400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe080870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50868/0 (socket says 172.21.15.71:50868) 2024-09-05T12:28:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.910+0000 7fdb57400640 1 -- 172.21.15.71:0/1584004827 learned_addr learned my addr 172.21.15.71:0/1584004827 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:32.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb56a00640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe080db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57e00640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe097f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57400640 1 -- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 msgr2=0x55e2fe097f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57400640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe097f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57400640 1 -- 172.21.15.71:0/1584004827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 msgr2=0x55e2fe080db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57400640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe080db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57400640 1 -- 172.21.15.71:0/1584004827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2fdfa4990 con 0x55e2fe078560 2024-09-05T12:28:32.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb56a00640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe080db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb57400640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe080870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb480054c0 tx=0x7fdb48005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd6229241970b391 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.911+0000 7fdb41600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.912+0000 7fdb41600640 1 -- 172.21.15.71:0/1584004827 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4803f050 con 0x55e2fe078560 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.912+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb48004030 con 0x55e2fe078560 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.912+0000 7fdb41600640 1 -- 172.21.15.71:0/1584004827 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fdb480037f0 con 0x55e2fe078560 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.912+0000 7fdb41600640 1 -- 172.21.15.71:0/1584004827 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb48035040 con 0x55e2fe078560 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.912+0000 7fdb60e88740 10 client.25504 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.912+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2fe08eac0 con 0x55e2fe078560 2024-09-05T12:28:32.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb41600640 1 -- 172.21.15.71:0/1584004827 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb48004030 con 0x55e2fe078560 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb41600640 1 -- 172.21.15.71:0/1584004827 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdb48004c90 con 0x55e2fe078560 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb60e88740 10 client.25504 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb60e88740 10 client.25504 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb60e88740 20 client.25504 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb60e88740 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe09e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb60e88740 4 client.25504 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.913+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2fdf81310 con 0x55e2fe09bf00 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.914+0000 7fdb56a00640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe09e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.914+0000 7fdb56a00640 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe09e2f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c9cce8fc3092db1 server_cookie=f02e588624b5f13c in_seq=0 out_seq=0 2024-09-05T12:28:32.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.914+0000 7fdb41600640 10 client.25504 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.915+0000 7fdb41600640 1 -- 172.21.15.71:0/1584004827 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2fdf81310 con 0x55e2fe09bf00 2024-09-05T12:28:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.915+0000 7fdb41600640 10 client.25504 handle_command_reply: tid=0 2024-09-05T12:28:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.915+0000 7fdb60e88740 1 client.25504 shutdown 2024-09-05T12:28:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 msgr2=0x55e2fe080870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe080870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdb480054c0 tx=0x7fdb48005bd0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 shutdown_connections 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe09e2f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb41600640 0 client.25504 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe097f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe080db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 --2- 172.21.15.71:0/1584004827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe080870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 >> 172.21.15.71:0/1584004827 conn(0x55e2fe074af0 msgr2=0x55e2fe07a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 shutdown_connections 2024-09-05T12:28:32.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.916+0000 7fdb60e88740 1 -- 172.21.15.71:0/1584004827 wait complete. 2024-09-05T12:28:32.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.917+0000 7fdb60e88740 20 client.25504 trim_cache size 0 max 16384 2024-09-05T12:28:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.951+0000 7fdb60e88740 1 Processor -- start 2024-09-05T12:28:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.951+0000 7fdb60e88740 1 -- start start 2024-09-05T12:28:32.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 0x55e2fe073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe0812f0 0x55e2fe07f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe083740 0x55e2fe07afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb60e88740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581acc00 con 0x55e2fe0812f0 2024-09-05T12:28:32.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb60e88740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581ae740 con 0x55e2fe078560 2024-09-05T12:28:32.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb60e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581ad2d0 con 0x55e2fe083740 2024-09-05T12:28:32.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 0x55e2fe073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb56a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe0812f0 0x55e2fe07f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb57400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 0x55e2fe073ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53912/0 (socket says 172.21.15.71:53912) 2024-09-05T12:28:32.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb57400640 1 -- 172.21.15.71:0/279582912 learned_addr learned my addr 172.21.15.71:0/279582912 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:32.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.952+0000 7fdb57e00640 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe083740 0x55e2fe07afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57e00640 1 -- 172.21.15.71:0/279582912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 msgr2=0x55e2fe073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57e00640 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 0x55e2fe073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57e00640 1 -- 172.21.15.71:0/279582912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe0812f0 msgr2=0x55e2fe07f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57e00640 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe0812f0 0x55e2fe07f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57e00640 1 -- 172.21.15.71:0/279582912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2fe07c4d0 con 0x55e2fe083740 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57400640 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 0x55e2fe073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb56a00640 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe0812f0 0x55e2fe07f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.953+0000 7fdb57e00640 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe083740 0x55e2fe07afc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb500104e0 tx=0x7fdb500109a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86693260af3246cf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.954+0000 7fdb43e00640 1 -- 172.21.15.71:0/279582912 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb50021070 con 0x55e2fe083740 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.954+0000 7fdb43e00640 1 -- 172.21.15.71:0/279582912 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5000fbc0 con 0x55e2fe083740 2024-09-05T12:28:32.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.954+0000 7fdb43e00640 1 -- 172.21.15.71:0/279582912 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5001cd60 con 0x55e2fe083740 2024-09-05T12:28:32.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 -- 172.21.15.71:0/279582912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe083740 msgr2=0x55e2fe07afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe083740 0x55e2fe07afc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdb500104e0 tx=0x7fdb500109a0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 -- 172.21.15.71:0/279582912 shutdown_connections 2024-09-05T12:28:32.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe083740 0x55e2fe07afc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe0812f0 0x55e2fe07f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 --2- 172.21.15.71:0/279582912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe078560 0x55e2fe073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 -- 172.21.15.71:0/279582912 >> 172.21.15.71:0/279582912 conn(0x55e2fe074af0 msgr2=0x55e2fe080950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:32.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.955+0000 7fdb60e88740 1 -- 172.21.15.71:0/279582912 shutdown_connections 2024-09-05T12:28:32.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.956+0000 7fdb60e88740 1 -- 172.21.15.71:0/279582912 wait complete. 2024-09-05T12:28:32.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.956+0000 7fdb60e88740 1 -- start start 2024-09-05T12:28:32.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.956+0000 7fdb60e88740 1 Processor -- start 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe07da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe07df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe07e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581ae070 con 0x55e2fe083740 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581ad9a0 con 0x55e2fe0812f0 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb581aac00 con 0x55e2fe078560 2024-09-05T12:28:32.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb60e88740 10 client.0 _ll_register_callbacks cb 0x7fdb500093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:32.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb57400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe07da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.957+0000 7fdb57400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe07da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50882/0 (socket says 172.21.15.71:50882) 2024-09-05T12:28:32.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57400640 1 -- 172.21.15.71:0/3749361281 learned_addr learned my addr 172.21.15.71:0/3749361281 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:32.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57e00640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe07e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.960 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb56a00640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe07df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57400640 1 -- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 msgr2=0x55e2fe07df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57400640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe07df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57400640 1 -- 172.21.15.71:0/3749361281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 msgr2=0x55e2fe07e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57400640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe07e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57400640 1 -- 172.21.15.71:0/3749361281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2fe105c40 con 0x55e2fe078560 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb56a00640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe07df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:28:32.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.958+0000 7fdb57e00640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe07e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.959+0000 7fdb57400640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe07da30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb48049040 tx=0x7fdb4803cfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9ae748e52151649 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.959+0000 7fdb41600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.959+0000 7fdb41600640 1 -- 172.21.15.71:0/3749361281 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4803f040 con 0x55e2fe078560 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.959+0000 7fdb41600640 1 -- 172.21.15.71:0/3749361281 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fdb4803c5c0 con 0x55e2fe078560 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.959+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5000fbc0 con 0x55e2fe078560 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb41600640 1 -- 172.21.15.71:0/3749361281 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdb48073050 con 0x55e2fe078560 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 10 client.25510 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:32.962 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e2fe106ee0 con 0x55e2fe078560 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb41600640 1 -- 172.21.15.71:0/3749361281 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb5000fbc0 con 0x55e2fe078560 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb41600640 1 -- 172.21.15.71:0/3749361281 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fdb4806e030 con 0x55e2fe078560 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 10 client.25510 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 10 client.25510 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 20 client.25510 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe109660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 4 client.25510 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.0, "await_for": 45.0}] 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.960+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.0, "await_for": 45.0}) -- 0x55e2fdfebc60 con 0x55e2fe09bf00 2024-09-05T12:28:32.963 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.961+0000 7fdb56a00640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:32.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.961+0000 7fdb56a00640 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe109660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7cf0e421655d0f server_cookie=a73e30adaae59364 in_seq=0 out_seq=0 2024-09-05T12:28:32.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.961+0000 7fdb41600640 10 client.25510 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:32.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb41600640 1 -- 172.21.15.71:0/3749361281 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e2fdfebc60 con 0x55e2fe09bf00 2024-09-05T12:28:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb41600640 10 client.25510 handle_command_reply: tid=0 2024-09-05T12:28:32.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 client.25510 shutdown 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 msgr2=0x55e2fe07da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe07da30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdb48049040 tx=0x7fdb4803cfc0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 shutdown_connections 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e2fe09bf00 0x55e2fe109660 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb41600640 0 client.25510 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e2fe083740 0x55e2fe07e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2fe0812f0 0x55e2fe07df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 --2- 172.21.15.71:0/3749361281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2fe078560 0x55e2fe07da30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.974+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 >> 172.21.15.71:0/3749361281 conn(0x55e2fe074af0 msgr2=0x55e2fe0820d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.975+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 shutdown_connections 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.975+0000 7fdb60e88740 1 -- 172.21.15.71:0/3749361281 wait complete. 2024-09-05T12:28:32.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.975+0000 7fdb60e88740 20 client.25510 trim_cache size 0 max 16384 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 30, 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 2, 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "5a638e43": { 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 2, 2024-09-05T12:28:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 138.0, 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdb3c0a2f10 msgr2=0x7fdb3c0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdb3c0a2f10 0x7fdb3c0a53d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb480096c0 tx=0x7fdb48000bc0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 msgr2=0x7fdb5819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:32.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 0x7fdb5819aa40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400e9f0 tx=0x7fdb4400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 shutdown_connections 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb581079c0 0x7fdb5819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb58102360 0x7fdb5819aa40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdb3c0a2f10 0x7fdb3c0a53d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 --2- 172.21.15.71:0/939105614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb581019c0 0x7fdb5819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.976+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 >> 172.21.15.71:0/939105614 conn(0x7fdb580fd740 msgr2=0x7fdb5810d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.977+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 shutdown_connections 2024-09-05T12:28:32.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:32.977+0000 7fdb5ce00640 1 -- 172.21.15.71:0/939105614 wait complete. 2024-09-05T12:28:33.825 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5a638e43', quiesce took 4.7 seconds. Will release after: 13.3 2024-09-05T12:28:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:34 smithi174 ceph-mon[34313]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:28:34.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:34 smithi071 ceph-mon[29845]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:28:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:34 smithi123 ceph-mon[34321]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:28:36.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:36 smithi174 ceph-mon[34313]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:28:36.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:36 smithi071 ceph-mon[29845]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:28:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:36 smithi123 ceph-mon[34321]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:28:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:38 smithi174 ceph-mon[34313]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-05T12:28:38.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:38 smithi071 ceph-mon[29845]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-05T12:28:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:38 smithi123 ceph-mon[34321]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 938 B/s wr, 13 op/s 2024-09-05T12:28:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:40 smithi174 ceph-mon[34313]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-05T12:28:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:40 smithi071 ceph-mon[29845]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-05T12:28:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:40 smithi123 ceph-mon[34321]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-09-05T12:28:41.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.479+0000 7fcd15c00640 1 -- 172.21.15.71:0/165537326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd101090a0 msgr2=0x7fcd10110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.479+0000 7fcd15c00640 1 --2- 172.21.15.71:0/165537326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd101090a0 0x7fcd10110bf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009ca0 tx=0x7fcd0802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 -- 172.21.15.71:0/165537326 shutdown_connections 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 --2- 172.21.15.71:0/165537326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd101090a0 0x7fcd10110bf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 --2- 172.21.15.71:0/165537326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd10104900 0x7fcd10108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 --2- 172.21.15.71:0/165537326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd10103f50 0x7fcd10104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 -- 172.21.15.71:0/165537326 >> 172.21.15.71:0/165537326 conn(0x7fcd100ff9e0 msgr2=0x7fcd10101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 -- 172.21.15.71:0/165537326 shutdown_connections 2024-09-05T12:28:41.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 -- 172.21.15.71:0/165537326 wait complete. 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.480+0000 7fcd15c00640 1 Processor -- start 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 -- start start 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 0x7fcd1019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd101090a0 0x7fcd1019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd1019edd0 0x7fcd1019f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10114140 con 0x7fcd10103f50 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd10113fc0 con 0x7fcd101090a0 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd15c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101142c0 con 0x7fcd1019edd0 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 0x7fcd1019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 0x7fcd1019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58800/0 (socket says 172.21.15.71:58800) 2024-09-05T12:28:41.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2129189311 learned_addr learned my addr 172.21.15.71:0/2129189311 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.481+0000 7fcd0e000640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd101090a0 0x7fcd1019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0f400640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd1019edd0 0x7fcd1019f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2129189311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd1019edd0 msgr2=0x7fcd1019f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd1019edd0 0x7fcd1019f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2129189311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd101090a0 msgr2=0x7fcd1019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd101090a0 0x7fcd1019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2129189311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1019f8c0 con 0x7fcd10103f50 2024-09-05T12:28:41.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.482+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 0x7fcd1019a510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000ea60 tx=0x7fcd0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.483+0000 7fcd07400640 1 -- 172.21.15.71:0/2129189311 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0000cde0 con 0x7fcd10103f50 2024-09-05T12:28:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.483+0000 7fcd07400640 1 -- 172.21.15.71:0/2129189311 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd000035f0 con 0x7fcd10103f50 2024-09-05T12:28:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.483+0000 7fcd07400640 1 -- 172.21.15.71:0/2129189311 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd00010670 con 0x7fcd10103f50 2024-09-05T12:28:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.483+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1019fbb0 con 0x7fcd10103f50 2024-09-05T12:28:41.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.483+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd101a7320 con 0x7fcd10103f50 2024-09-05T12:28:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.487+0000 7fcd07400640 1 -- 172.21.15.71:0/2129189311 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7fcd000037a0 con 0x7fcd10103f50 2024-09-05T12:28:41.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.487+0000 7fcd18c43740 1 Processor -- start 2024-09-05T12:28:41.492 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.487+0000 7fcd18c43740 1 -- start start 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47f1e0 0x558adf47f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf489e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf48a3a0 0x558adf48c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd18c43740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558adf3e5100 con 0x558adf47f1e0 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd18c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558adf38d1b0 con 0x558adf48a3a0 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd18c43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558adf3e10c0 con 0x558adf47fb00 2024-09-05T12:28:41.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47f1e0 0x558adf47f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd0ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf489e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47f1e0 0x558adf47f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58804/0 (socket says 172.21.15.71:58804) 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd0ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf489e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50894/0 (socket says 172.21.15.71:50894) 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd0f400640 1 -- 172.21.15.71:0/3042370852 learned_addr learned my addr 172.21.15.71:0/3042370852 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.488+0000 7fcd0e000640 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf48a3a0 0x558adf48c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd07400640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fccf40a2f40 0x7fccf40a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0ea00640 1 -- 172.21.15.71:0/3042370852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf48a3a0 msgr2=0x558adf48c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf48a3a0 0x558adf48c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0ea00640 1 -- 172.21.15.71:0/3042370852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47f1e0 msgr2=0x558adf47f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47f1e0 0x558adf47f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0ea00640 1 -- 172.21.15.71:0/3042370852 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558adf0f8210 con 0x558adf47fb00 2024-09-05T12:28:41.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0f400640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fccf40a2f40 0x7fccf40a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf489e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd000cd1f0 tx=0x7fcd000d84b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf3a778c5216d92b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fcd07400640 1 -- 172.21.15.71:0/2129189311 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fcd000cff80 con 0x7fcd10103f50 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.489+0000 7fccfbe00640 1 -- 172.21.15.71:0/3042370852 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0009bcc0 con 0x558adf47fb00 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.490+0000 7fccfbe00640 1 -- 172.21.15.71:0/3042370852 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0009c5f0 con 0x558adf47fb00 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.490+0000 7fccfbe00640 1 -- 172.21.15.71:0/3042370852 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000a0dd0 con 0x558adf47fb00 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.490+0000 7fcd18c43740 1 -- 172.21.15.71:0/3042370852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 msgr2=0x558adf489e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.490+0000 7fcd0f400640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fccf40a2f40 0x7fccf40a5400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd0802aeb0 tx=0x7fcd080095c0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.490+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf489e60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd000cd1f0 tx=0x7fcd000d84b0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 -- 172.21.15.71:0/3042370852 shutdown_connections 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf48a3a0 0x558adf48c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf489e60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3042370852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47f1e0 0x558adf47f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 -- 172.21.15.71:0/3042370852 >> 172.21.15.71:0/3042370852 conn(0x558adf47b750 msgr2=0x558adf47bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 -- 172.21.15.71:0/3042370852 shutdown_connections 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 -- 172.21.15.71:0/3042370852 wait complete. 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 -- start start 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.491+0000 7fcd18c43740 1 Processor -- start 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf487390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4878d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf487e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558adf48cfd0 con 0x558adf47fb00 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558adf36e090 con 0x558adf47f1e0 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558adf48d150 con 0x558adf48a3a0 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd18c43740 10 client.0 _ll_register_callbacks cb 0x558adf3c2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd0e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4878d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd0e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4878d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58814/0 (socket says 172.21.15.71:58814) 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd0e000640 1 -- 172.21.15.71:0/3017312960 learned_addr learned my addr 172.21.15.71:0/3017312960 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf487390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.492+0000 7fcd0f400640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf487e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fcd0e000640 1 -- 172.21.15.71:0/3017312960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 msgr2=0x558adf487e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fcd0e000640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf487e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fcd0e000640 1 -- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 msgr2=0x558adf487390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fcd0e000640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf487390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fcd0e000640 1 -- 172.21.15.71:0/3017312960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558adf1c4620 con 0x558adf47fb00 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fcd0e000640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4878d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fccfc013710 tx=0x7fccfc013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f72ae7f667369eb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fccf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fccf8c00640 1 -- 172.21.15.71:0/3017312960 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccfc011940 con 0x558adf47fb00 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.493+0000 7fccf8c00640 1 -- 172.21.15.71:0/3017312960 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fccfc011ae0 con 0x558adf47fb00 2024-09-05T12:28:41.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0009c5f0 con 0x558adf47fb00 2024-09-05T12:28:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fccf8c00640 1 -- 172.21.15.71:0/3017312960 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fccfc011ce0 con 0x558adf47fb00 2024-09-05T12:28:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fcd18c43740 10 client.15471 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:41.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558adf498c50 con 0x558adf47fb00 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fccf8c00640 1 -- 172.21.15.71:0/3017312960 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0009c5f0 con 0x558adf47fb00 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fccf8c00640 1 -- 172.21.15.71:0/3017312960 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fccfc01a070 con 0x558adf47fb00 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.494+0000 7fcd18c43740 10 client.15471 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.495+0000 7fcd18c43740 10 client.15471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.495+0000 7fcd18c43740 20 client.15471 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.495+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf49a0c0 0x558adf49c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.495+0000 7fcd18c43740 4 client.15471 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.495+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558adf3f2830 con 0x558adf49a0c0 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.495+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf49a0c0 0x558adf49c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.496+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf49a0c0 0x558adf49c530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be6ac75d2ee54a0a server_cookie=e9afa9b4f0a26f29 in_seq=0 out_seq=0 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.496+0000 7fccf8c00640 10 client.15471 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:41.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.496+0000 7fccf8c00640 1 -- 172.21.15.71:0/3017312960 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558adf3f2830 con 0x558adf49a0c0 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.496+0000 7fccf8c00640 10 client.15471 handle_command_reply: tid=0 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.496+0000 7fcd18c43740 1 client.15471 shutdown 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 msgr2=0x558adf4878d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4878d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fccfc013710 tx=0x7fccfc013bd0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 shutdown_connections 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf49a0c0 0x558adf49c530 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf487e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4878d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 --2- 172.21.15.71:0/3017312960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf487390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 >> 172.21.15.71:0/3017312960 conn(0x558adf47b750 msgr2=0x558adf4814b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fccf8c00640 0 client.15471 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 shutdown_connections 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 1 -- 172.21.15.71:0/3017312960 wait complete. 2024-09-05T12:28:41.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.497+0000 7fcd18c43740 20 client.15471 trim_cache size 0 max 16384 2024-09-05T12:28:41.513 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 Processor -- start 2024-09-05T12:28:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 -- start start 2024-09-05T12:28:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf4766a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf4a2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101b1060 con 0x558adf47fb00 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101b2ba0 con 0x558adf47f1e0 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.512+0000 7fcd18c43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101b1730 con 0x558adf48a3a0 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf4766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf4766a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53982/0 (socket says 172.21.15.71:53982) 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2179292504 learned_addr learned my addr 172.21.15.71:0/2179292504 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0f400640 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf4a2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2179292504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 msgr2=0x558adf4a2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf4a2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2179292504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 msgr2=0x558adf4a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2179292504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558adf496f10 con 0x558adf47f1e0 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.513+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf4766a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c760 tx=0x7fcd000d3a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7038153b254fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fccfb400640 1 -- 172.21.15.71:0/2179292504 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000a1430 con 0x558adf47f1e0 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fccfb400640 1 -- 172.21.15.71:0/2179292504 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0009c5f0 con 0x558adf47f1e0 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fccfb400640 1 -- 172.21.15.71:0/2179292504 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd000a2a90 con 0x558adf47f1e0 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fcd18c43740 1 -- 172.21.15.71:0/2179292504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 msgr2=0x558adf4766a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf4766a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000c760 tx=0x7fcd000d3a50 comp rx=0 tx=0).stop 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fcd18c43740 1 -- 172.21.15.71:0/2179292504 shutdown_connections 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf48a3a0 0x558adf4a2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf47fb00 0x558adf4a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.514+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2179292504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf4766a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.515+0000 7fcd18c43740 1 -- 172.21.15.71:0/2179292504 >> 172.21.15.71:0/2179292504 conn(0x558adf47b750 msgr2=0x558adf485cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.515+0000 7fcd18c43740 1 -- 172.21.15.71:0/2179292504 shutdown_connections 2024-09-05T12:28:41.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.515+0000 7fcd18c43740 1 -- 172.21.15.71:0/2179292504 wait complete. 2024-09-05T12:28:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.515+0000 7fcd18c43740 1 -- start start 2024-09-05T12:28:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.515+0000 7fcd18c43740 1 Processor -- start 2024-09-05T12:28:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf492670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf492bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf48a3a0 0x558adf483f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101b24d0 con 0x558adf48a3a0 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101b1e00 con 0x558adf47f1e0 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd101af040 con 0x558adf47fb00 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd18c43740 10 client.0 _ll_register_callbacks cb 0x7fccfc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd0e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf492bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd0f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf48a3a0 0x558adf483f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd0e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf492bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50924/0 (socket says 172.21.15.71:50924) 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd0e000640 1 -- 172.21.15.71:0/2332355182 learned_addr learned my addr 172.21.15.71:0/2332355182 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.516+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf492670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2332355182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 msgr2=0x558adf492bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf492bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2332355182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf48a3a0 msgr2=0x558adf483f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf48a3a0 0x558adf483f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0ea00640 1 -- 172.21.15.71:0/2332355182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558adf484790 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0e000640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf492bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd0ea00640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf492670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d010 tx=0x7fcd000095b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8bb3fe3d7ea898 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fccf8c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fccf8c00640 1 -- 172.21.15.71:0/2332355182 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0009b850 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd0009c5f0 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fccf8c00640 1 -- 172.21.15.71:0/2332355182 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fcd00003b30 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fccf8c00640 1 -- 172.21.15.71:0/2332355182 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd0009ba30 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd18c43740 10 client.25523 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.517+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558adf504a80 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fccf8c00640 1 -- 172.21.15.71:0/2332355182 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd0009c5f0 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fccf8c00640 1 -- 172.21.15.71:0/2332355182 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fcd00020070 con 0x558adf47f1e0 2024-09-05T12:28:41.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd18c43740 10 client.25523 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd18c43740 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd18c43740 20 client.25523 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf504cb0 0x558adf5070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd18c43740 4 client.25523 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558adf476d80 con 0x558adf504cb0 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.518+0000 7fcd0e000640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf504cb0 0x558adf5070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.519+0000 7fcd0e000640 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf504cb0 0x558adf5070a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9e608a0f31eb58 server_cookie=8ae8291a7426cb30 in_seq=0 out_seq=0 2024-09-05T12:28:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.519+0000 7fccf8c00640 10 client.25523 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fccf8c00640 1 -- 172.21.15.71:0/2332355182 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558adf476d80 con 0x558adf504cb0 2024-09-05T12:28:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fccf8c00640 10 client.25523 handle_command_reply: tid=0 2024-09-05T12:28:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 client.25523 shutdown 2024-09-05T12:28:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 msgr2=0x558adf492670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf492670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d010 tx=0x7fcd000095b0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 shutdown_connections 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558adf504cb0 0x558adf5070a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558adf48a3a0 0x558adf483f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558adf47fb00 0x558adf492bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 --2- 172.21.15.71:0/2332355182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558adf47f1e0 0x558adf492670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 >> 172.21.15.71:0/2332355182 conn(0x558adf47b750 msgr2=0x558adf487330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fccf8c00640 0 client.25523 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 shutdown_connections 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.527+0000 7fcd18c43740 1 -- 172.21.15.71:0/2332355182 wait complete. 2024-09-05T12:28:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.528+0000 7fcd18c43740 20 client.25523 trim_cache size 0 max 16384 2024-09-05T12:28:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:28:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:28:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:28:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:28:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.528+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fccf40a2f40 msgr2=0x7fccf40a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.528+0000 7fcd15c00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fccf40a2f40 0x7fccf40a5400 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd0802aeb0 tx=0x7fcd080095c0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.528+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 msgr2=0x7fcd1019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.528+0000 7fcd15c00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 0x7fcd1019a510 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000ea60 tx=0x7fcd0000ef20 comp rx=0 tx=0).stop 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 shutdown_connections 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd1019edd0 0x7fcd1019f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd101090a0 0x7fcd1019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fccf40a2f40 0x7fccf40a5400 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 --2- 172.21.15.71:0/2129189311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd10103f50 0x7fcd1019a510 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 >> 172.21.15.71:0/2129189311 conn(0x7fcd100ff9e0 msgr2=0x7fcd101004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 shutdown_connections 2024-09-05T12:28:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:41.529+0000 7fcd15c00640 1 -- 172.21.15.71:0/2129189311 wait complete. 2024-09-05T12:28:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:42 smithi174 ceph-mon[34313]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-05T12:28:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:42 smithi071 ceph-mon[29845]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-05T12:28:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:42 smithi123 ceph-mon[34321]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-05T12:28:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:44 smithi071 ceph-mon[29845]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-05T12:28:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:44 smithi123 ceph-mon[34321]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-05T12:28:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:44 smithi174 ceph-mon[34313]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1023 B/s wr, 13 op/s 2024-09-05T12:28:46.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:46 smithi174 ceph-mon[34313]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-05T12:28:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:46 smithi071 ceph-mon[29845]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-05T12:28:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:46 smithi123 ceph-mon[34321]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 8 op/s 2024-09-05T12:28:47.127 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:28:47.581 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:48.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 -- 172.21.15.71:0/518432942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41079c0 msgr2=0x7f16c4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:48.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 --2- 172.21.15.71:0/518432942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41079c0 0x7f16c4107da0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009930 tx=0x7f16b402a440 comp rx=0 tx=0).stop 2024-09-05T12:28:48.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 -- 172.21.15.71:0/518432942 shutdown_connections 2024-09-05T12:28:48.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 --2- 172.21.15.71:0/518432942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 0x7f16c410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 --2- 172.21.15.71:0/518432942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41019c0 0x7f16c4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 --2- 172.21.15.71:0/518432942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41079c0 0x7f16c4107da0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 -- 172.21.15.71:0/518432942 >> 172.21.15.71:0/518432942 conn(0x7f16c40fd740 msgr2=0x7f16c40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:48.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 -- 172.21.15.71:0/518432942 shutdown_connections 2024-09-05T12:28:48.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.297+0000 7f16c8e00640 1 -- 172.21.15.71:0/518432942 wait complete. 2024-09-05T12:28:48.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.299+0000 7f16c8e00640 1 Processor -- start 2024-09-05T12:28:48.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.299+0000 7f16c8e00640 1 -- start start 2024-09-05T12:28:48.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41019c0 0x7f16c41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:48.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 0x7f16c41987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41079c0 0x7f16c419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c8e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111dd0 con 0x7f16c4102360 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111c50 con 0x7f16c41079c0 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c8e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4111f50 con 0x7f16c41019c0 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 0x7f16c41987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41019c0 0x7f16c41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 0x7f16c41987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55504/0 (socket says 172.21.15.71:55504) 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c2a00640 1 -- 172.21.15.71:0/1284097442 learned_addr learned my addr 172.21.15.71:0/1284097442 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.300+0000 7f16c3400640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41019c0 0x7f16c41982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59978/0 (socket says 172.21.15.71:59978) 2024-09-05T12:28:48.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.301+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41079c0 0x7f16c419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.301+0000 7f16c3e00640 1 -- 172.21.15.71:0/1284097442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41019c0 msgr2=0x7f16c41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.301+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41019c0 0x7f16c41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.301+0000 7f16c3e00640 1 -- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 msgr2=0x7f16c41987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.301+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 0x7f16c41987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.301+0000 7f16c3e00640 1 -- 172.21.15.71:0/1284097442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c419d280 con 0x7f16c41079c0 2024-09-05T12:28:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.302+0000 7f16c3e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41079c0 0x7f16c419cb80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00e940 tx=0x7f16bc00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.303+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc004d20 con 0x7f16c41079c0 2024-09-05T12:28:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.303+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16bc005020 con 0x7f16c41079c0 2024-09-05T12:28:48.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.303+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc0116b0 con 0x7f16c41079c0 2024-09-05T12:28:48.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.303+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c419d570 con 0x7f16c41079c0 2024-09-05T12:28:48.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.303+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c4074420 con 0x7f16c41079c0 2024-09-05T12:28:48.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.304+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c4103100 con 0x7f16c41079c0 2024-09-05T12:28:48.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.308+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f16bc0054d0 con 0x7f16c41079c0 2024-09-05T12:28:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.310+0000 7f16bbe00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f16a80a2fe0 0x7f16a80a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.311+0000 7f16c3400640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f16a80a2fe0 0x7f16a80a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.311+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f16bc0cfb00 con 0x7f16c41079c0 2024-09-05T12:28:48.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.312+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16bc016030 con 0x7f16c41079c0 2024-09-05T12:28:48.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.313+0000 7f16c3400640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f16a80a2fe0 0x7f16a80a54a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006010 tx=0x7f16b4005a20 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:48.543 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:48 smithi071 ceph-mon[29845]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-05T12:28:48.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.540+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16c4101e20 con 0x7f16c41079c0 2024-09-05T12:28:48.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.541+0000 7f16bbe00640 1 -- 172.21.15.71:0/1284097442 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11968 (secure 0 0 0) 0x7f16c4101e20 con 0x7f16c41079c0 2024-09-05T12:28:48.544 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:28:48.544 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-05T12:27:44:191847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:27:44.168192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:28:48.544 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-05T12:28:48.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.544+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f16a80a2fe0 msgr2=0x7f16a80a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:48.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.544+0000 7f16c8e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f16a80a2fe0 0x7f16a80a54a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16b4006010 tx=0x7f16b4005a20 comp rx=0 tx=0).stop 2024-09-05T12:28:48.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.544+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41079c0 msgr2=0x7f16c419cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:48.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.544+0000 7f16c8e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41079c0 0x7f16c419cb80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00e940 tx=0x7f16bc00ee00 comp rx=0 tx=0).stop 2024-09-05T12:28:48.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 shutdown_connections 2024-09-05T12:28:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c41079c0 0x7f16c419cb80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f16a80a2fe0 0x7f16a80a54a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f16c4102360 0x7f16c41987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 --2- 172.21.15.71:0/1284097442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16c41019c0 0x7f16c41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 >> 172.21.15.71:0/1284097442 conn(0x7f16c40fd740 msgr2=0x7f16c40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:48.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 shutdown_connections 2024-09-05T12:28:48.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:48.546+0000 7f16c8e00640 1 -- 172.21.15.71:0/1284097442 wait complete. 2024-09-05T12:28:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:48 smithi123 ceph-mon[34321]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-05T12:28:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:48 smithi174 ceph-mon[34313]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-09-05T12:28:49.372 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 5a638e43 --release --await' 2024-09-05T12:28:49.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 5a638e43 --release --await 2024-09-05T12:28:49.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:49 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1284097442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:49.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:49 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1284097442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:49.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:49 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1284097442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:28:49.860 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:28:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 -- 172.21.15.71:0/1540576289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4108f60 msgr2=0x7f12f4110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 --2- 172.21.15.71:0/1540576289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4108f60 0x7f12f4110ae0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009be0 tx=0x7f12ec02a6c0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 -- 172.21.15.71:0/1540576289 shutdown_connections 2024-09-05T12:28:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 --2- 172.21.15.71:0/1540576289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f4108f60 0x7f12f4110ae0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 --2- 172.21.15.71:0/1540576289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f41047e0 0x7f12f4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 --2- 172.21.15.71:0/1540576289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12f4103e30 0x7f12f4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 -- 172.21.15.71:0/1540576289 >> 172.21.15.71:0/1540576289 conn(0x7f12f40ff910 msgr2=0x7f12f4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 -- 172.21.15.71:0/1540576289 shutdown_connections 2024-09-05T12:28:50.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.479+0000 7f12fac00640 1 -- 172.21.15.71:0/1540576289 wait complete. 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.480+0000 7f12fac00640 1 Processor -- start 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.480+0000 7f12fac00640 1 -- start start 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103e30 0x7f12f4073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 0x7f12f4073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fac00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12f4108f60 0x7f12f406cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4113fc0 con 0x7f12f4103e30 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4113e40 con 0x7f12f41047e0 2024-09-05T12:28:50.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fac00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12f4114140 con 0x7f12f4108f60 2024-09-05T12:28:50.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12f8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 0x7f12f4073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12f8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 0x7f12f4073ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33348/0 (socket says 172.21.15.71:33348) 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12f8e00640 1 -- 172.21.15.71:0/4185569448 learned_addr learned my addr 172.21.15.71:0/4185569448 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.481+0000 7f12fa200640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12f4108f60 0x7f12f406cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f9800640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103e30 0x7f12f4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f8e00640 1 -- 172.21.15.71:0/4185569448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12f4108f60 msgr2=0x7f12f406cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f8e00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12f4108f60 0x7f12f406cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f8e00640 1 -- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103e30 msgr2=0x7f12f4073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f8e00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103e30 0x7f12f4073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f8e00640 1 -- 172.21.15.71:0/4185569448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f406d490 con 0x7f12f41047e0 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f9800640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103e30 0x7f12f4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:28:50.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.482+0000 7f12f8e00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 0x7f12f4073ec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f12e000e9c0 tx=0x7f12e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.483+0000 7f12f2000640 1 -- 172.21.15.71:0/4185569448 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e000cde0 con 0x7f12f41047e0 2024-09-05T12:28:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.483+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f406d780 con 0x7f12f41047e0 2024-09-05T12:28:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.483+0000 7f12f2000640 1 -- 172.21.15.71:0/4185569448 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e00035f0 con 0x7f12f41047e0 2024-09-05T12:28:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.483+0000 7f12f2000640 1 -- 172.21.15.71:0/4185569448 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0010670 con 0x7f12f41047e0 2024-09-05T12:28:50.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.483+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f40717c0 con 0x7f12f41047e0 2024-09-05T12:28:50.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 Processor -- start 2024-09-05T12:28:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 -- start start 2024-09-05T12:28:50.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909712a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 0x5590971351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559097135700 0x559097137af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559097090650 con 0x55909712ae40 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559097037f10 con 0x55909712a520 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.487+0000 7f12fec8d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55909708c610 con 0x559097135700 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.488+0000 7f12f9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 0x5590971351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.488+0000 7f12f9800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 0x5590971351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55540/0 (socket says 172.21.15.71:55540) 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.488+0000 7f12f9800640 1 -- 172.21.15.71:0/779741722 learned_addr learned my addr 172.21.15.71:0/779741722 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.488+0000 7f12f2000640 1 -- 172.21.15.71:0/4185569448 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f12e00037a0 con 0x7f12f41047e0 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.488+0000 7f12fa200640 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909712a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.489+0000 7f12f9800640 1 -- 172.21.15.71:0/779741722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559097135700 msgr2=0x559097137af0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.489+0000 7f12f9800640 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559097135700 0x559097137af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.489+0000 7f12f9800640 1 -- 172.21.15.71:0/779741722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 msgr2=0x55909712a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.489+0000 7f12f9800640 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909712a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.489+0000 7f12f9800640 1 -- 172.21.15.71:0/779741722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559096d9f580 con 0x55909712ae40 2024-09-05T12:28:50.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.489+0000 7f12f9800640 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 0x5590971351c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009d40 tx=0x7f12e80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed842dfacbc08153 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:50.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.490+0000 7f12e6a00640 1 -- 172.21.15.71:0/779741722 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8012650 con 0x55909712ae40 2024-09-05T12:28:50.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.490+0000 7f12e6a00640 1 -- 172.21.15.71:0/779741722 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8009e30 con 0x55909712ae40 2024-09-05T12:28:50.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.490+0000 7f12e6a00640 1 -- 172.21.15.71:0/779741722 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800fd00 con 0x55909712ae40 2024-09-05T12:28:50.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.490+0000 7f12f2000640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f12dc0a2f10 0x7f12dc0a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.491+0000 7f12fec8d740 1 -- 172.21.15.71:0/779741722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 msgr2=0x5590971351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.491+0000 7f12fec8d740 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 0x5590971351c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009d40 tx=0x7f12e80117b0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.491+0000 7f12f2000640 1 -- 172.21.15.71:0/4185569448 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f12e00cfd60 con 0x7f12f41047e0 2024-09-05T12:28:50.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.491+0000 7f12fa200640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f12dc0a2f10 0x7f12dc0a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fa200640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f12dc0a2f10 0x7f12dc0a53d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec004af0 tx=0x7f12ec005fd0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:50.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fec8d740 1 -- 172.21.15.71:0/779741722 shutdown_connections 2024-09-05T12:28:50.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fec8d740 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559097135700 0x559097137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fec8d740 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55909712ae40 0x5590971351c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fec8d740 1 --2- 172.21.15.71:0/779741722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909712a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fec8d740 1 -- 172.21.15.71:0/779741722 >> 172.21.15.71:0/779741722 conn(0x559097126ab0 msgr2=0x559097126e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:50.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.492+0000 7f12fec8d740 1 -- 172.21.15.71:0/779741722 shutdown_connections 2024-09-05T12:28:50.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.493+0000 7f12fec8d740 1 -- 172.21.15.71:0/779741722 wait complete. 2024-09-05T12:28:50.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.493+0000 7f12fec8d740 1 -- start start 2024-09-05T12:28:50.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.493+0000 7f12fec8d740 1 Processor -- start 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712a520 0x5590971326b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712ae40 0x559097132bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x559097133130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5590971382d0 con 0x559097135700 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55909704b350 con 0x55909712ae40 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559097138450 con 0x55909712a520 2024-09-05T12:28:50.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12fec8d740 10 client.0 _ll_register_callbacks cb 0x55909706e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:50.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12f9800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712a520 0x5590971326b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12f9800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712a520 0x5590971326b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:60014/0 (socket says 172.21.15.71:60014) 2024-09-05T12:28:50.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.494+0000 7f12f9800640 1 -- 172.21.15.71:0/1078564046 learned_addr learned my addr 172.21.15.71:0/1078564046 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:50.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.495+0000 7f12fa200640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x559097133130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.495+0000 7f12fa200640 1 -- 172.21.15.71:0/1078564046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712a520 msgr2=0x5590971326b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:50.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.495+0000 7f12fa200640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712a520 0x5590971326b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.495+0000 7f12fa200640 1 -- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712ae40 msgr2=0x559097132bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:50.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.495+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712ae40 0x559097132bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.496+0000 7f12fa200640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712ae40 0x559097132bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.496+0000 7f12fa200640 1 -- 172.21.15.71:0/1078564046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559096e69960 con 0x559097135700 2024-09-05T12:28:50.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.496+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712ae40 0x559097132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.496+0000 7f12fa200640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x559097133130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009b30 tx=0x7f12ec009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612908ca155d944c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.496+0000 7f12db400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.496+0000 7f12db400640 1 -- 172.21.15.71:0/1078564046 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12ec04f070 con 0x559097135700 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.497+0000 7f12db400640 1 -- 172.21.15.71:0/1078564046 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f12ec003d80 con 0x559097135700 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.497+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e8009e30 con 0x559097135700 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.497+0000 7f12db400640 1 -- 172.21.15.71:0/1078564046 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12ec05b050 con 0x559097135700 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.497+0000 7f12fec8d740 10 client.15495 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:50.499 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.497+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559097143fb0 con 0x559097135700 2024-09-05T12:28:50.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.497+0000 7f12db400640 1 -- 172.21.15.71:0/1078564046 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8009e30 con 0x559097135700 2024-09-05T12:28:50.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12db400640 1 -- 172.21.15.71:0/1078564046 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f12ec04b6b0 con 0x559097135700 2024-09-05T12:28:50.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12fec8d740 10 client.15495 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:50.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12fec8d740 10 client.15495 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:28:50.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12fec8d740 20 client.15495 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:50.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971453c0 0x559097147810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12fec8d740 4 client.15495 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:50.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909709dbc0 con 0x5590971453c0 2024-09-05T12:28:50.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.498+0000 7f12f9800640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971453c0 0x559097147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.499+0000 7f12f9800640 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971453c0 0x559097147810 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3474ff9d88a03522 server_cookie=c7d274bcf495d317 in_seq=0 out_seq=0 2024-09-05T12:28:50.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.499+0000 7f12db400640 10 client.15495 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:50.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.500+0000 7f12db400640 1 -- 172.21.15.71:0/1078564046 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55909709dbc0 con 0x5590971453c0 2024-09-05T12:28:50.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.500+0000 7f12db400640 10 client.15495 handle_command_reply: tid=0 2024-09-05T12:28:50.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.500+0000 7f12fec8d740 1 client.15495 shutdown 2024-09-05T12:28:50.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.500+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 msgr2=0x559097133130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.500+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x559097133130 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009b30 tx=0x7f12ec009500 comp rx=0 tx=0).stop 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 shutdown_connections 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971453c0 0x559097147810 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x559097133130 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712ae40 0x559097132bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1078564046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712a520 0x5590971326b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 >> 172.21.15.71:0/1078564046 conn(0x559097126ab0 msgr2=0x559097125cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12db400640 0 client.15495 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:50.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 shutdown_connections 2024-09-05T12:28:50.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 1 -- 172.21.15.71:0/1078564046 wait complete. 2024-09-05T12:28:50.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.501+0000 7f12fec8d740 20 client.15495 trim_cache size 0 max 16384 2024-09-05T12:28:50.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.531+0000 7f12fec8d740 1 Processor -- start 2024-09-05T12:28:50.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.531+0000 7f12fec8d740 1 -- start start 2024-09-05T12:28:50.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.531+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909714d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.531+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909714d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:50 smithi071 ceph-mon[29845]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-05T12:28:50.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.531+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x55909712f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.532+0000 7f12fec8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b9650 con 0x559097135700 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.532+0000 7f12fec8d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41bb190 con 0x55909712a520 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.532+0000 7f12fec8d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b9d20 con 0x55909712ae40 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.532+0000 7f12f8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909714d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.532+0000 7f12f8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909714d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:60030/0 (socket says 172.21.15.71:60030) 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.532+0000 7f12f8e00640 1 -- 172.21.15.71:0/1996634566 learned_addr learned my addr 172.21.15.71:0/1996634566 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12f8e00640 1 -- 172.21.15.71:0/1996634566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 msgr2=0x55909714d130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909714d130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12f8e00640 1 -- 172.21.15.71:0/1996634566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 msgr2=0x55909712f910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x55909712f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12f8e00640 1 -- 172.21.15.71:0/1996634566 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590971268e0 con 0x55909712ae40 2024-09-05T12:28:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909714d670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c6a0 tx=0x7f12e009b6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b068cb8c1e763c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12e6000640 1 -- 172.21.15.71:0/1996634566 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00a0040 con 0x55909712ae40 2024-09-05T12:28:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12e6000640 1 -- 172.21.15.71:0/1996634566 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e00a4900 con 0x55909712ae40 2024-09-05T12:28:50.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.533+0000 7f12e6000640 1 -- 172.21.15.71:0/1996634566 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e00cd4d0 con 0x55909712ae40 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.534+0000 7f12fec8d740 1 -- 172.21.15.71:0/1996634566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 msgr2=0x55909714d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.534+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909714d670 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c6a0 tx=0x7f12e009b6e0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 -- 172.21.15.71:0/1996634566 shutdown_connections 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x55909712f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909714d670 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1996634566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x55909714d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 -- 172.21.15.71:0/1996634566 >> 172.21.15.71:0/1996634566 conn(0x559097126ab0 msgr2=0x559097125cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 -- 172.21.15.71:0/1996634566 shutdown_connections 2024-09-05T12:28:50.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.535+0000 7f12fec8d740 1 -- 172.21.15.71:0/1996634566 wait complete. 2024-09-05T12:28:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.536+0000 7f12fec8d740 1 -- start start 2024-09-05T12:28:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.536+0000 7f12fec8d740 1 Processor -- start 2024-09-05T12:28:50.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x559097139c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909713a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x5590971a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41baac0 con 0x559097135700 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41ba3f0 con 0x55909712a520 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12f41b7630 con 0x55909712ae40 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12fec8d740 10 client.0 _ll_register_callbacks cb 0x7f12e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12f8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909713a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12f8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909713a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:60034/0 (socket says 172.21.15.71:60034) 2024-09-05T12:28:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.537+0000 7f12f8e00640 1 -- 172.21.15.71:0/1557572357 learned_addr learned my addr 172.21.15.71:0/1557572357 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.538+0000 7f12f9800640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x559097139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.538+0000 7f12fa200640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x5590971a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.538+0000 7f12f9800640 1 -- 172.21.15.71:0/1557572357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 msgr2=0x55909713a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.538+0000 7f12f9800640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909713a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.540 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.538+0000 7f12f9800640 1 -- 172.21.15.71:0/1557572357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 msgr2=0x5590971a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.539+0000 7f12f9800640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x5590971a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.539+0000 7f12f9800640 1 -- 172.21.15.71:0/1557572357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590971a9710 con 0x55909712a520 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.539+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909713a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.539+0000 7f12fa200640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x5590971a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.539+0000 7f12f9800640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x559097139c90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559097148020 tx=0x7f12e8009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc4813dc4f036827 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.539+0000 7f12db400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.540+0000 7f12db400640 1 -- 172.21.15.71:0/1557572357 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e800f9d0 con 0x55909712a520 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.540+0000 7f12db400640 1 -- 172.21.15.71:0/1557572357 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f12e800fb70 con 0x55909712a520 2024-09-05T12:28:50.542 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.540+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e00a4900 con 0x55909712a520 2024-09-05T12:28:50.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12db400640 1 -- 172.21.15.71:0/1557572357 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f12e800fd90 con 0x55909712a520 2024-09-05T12:28:50.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 10 client.25541 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:50.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5590971b92f0 con 0x55909712a520 2024-09-05T12:28:50.543 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12db400640 1 -- 172.21.15.71:0/1557572357 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e00a4900 con 0x55909712a520 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12db400640 1 -- 172.21.15.71:0/1557572357 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f12e8004aa0 con 0x55909712a520 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 10 client.25541 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 10 client.25541 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 20 client.25541 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971b9870 0x5590971bbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 4 client.25541 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5a638e43", "release": true, "await": true}] 2024-09-05T12:28:50.544 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.541+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5a638e43", "release": true, "await": true}) -- 0x5590970980d0 con 0x5590971b9870 2024-09-05T12:28:50.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.542+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971b9870 0x5590971bbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:50.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.543+0000 7f12f8e00640 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971b9870 0x5590971bbc60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad509c48596268a server_cookie=9581e4b82f98ebbc in_seq=0 out_seq=0 2024-09-05T12:28:50.545 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.543+0000 7f12db400640 10 client.25541 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:50.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.544+0000 7f12db400640 1 -- 172.21.15.71:0/1557572357 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5590970980d0 con 0x5590971b9870 2024-09-05T12:28:50.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.544+0000 7f12db400640 10 client.25541 handle_command_reply: tid=0 2024-09-05T12:28:50.546 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.544+0000 7f12fec8d740 1 client.25541 shutdown 2024-09-05T12:28:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 msgr2=0x559097139c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x559097139c90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559097148020 tx=0x7f12e8009cd0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 shutdown_connections 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590971b9870 0x5590971bbc60 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559097135700 0x5590971a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909712ae40 0x55909713a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 --2- 172.21.15.71:0/1557572357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909712a520 0x559097139c90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 >> 172.21.15.71:0/1557572357 conn(0x559097126ab0 msgr2=0x559097132670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.545+0000 7f12db400640 0 client.25541 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.546+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 shutdown_connections 2024-09-05T12:28:50.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.546+0000 7f12fec8d740 1 -- 172.21.15.71:0/1557572357 wait complete. 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.547+0000 7f12fec8d740 20 client.25541 trim_cache size 0 max 16384 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 30, 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 4, 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout: "5a638e43": { 2024-09-05T12:28:50.549 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 4, 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 138.0, 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:28:50.550 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:28:50.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:50.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:50.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:50.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:50.551 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:28:50.551 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f12dc0a2f10 msgr2=0x7f12dc0a53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f12dc0a2f10 0x7f12dc0a53d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f12ec004af0 tx=0x7f12ec005fd0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 msgr2=0x7f12f4073ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 0x7f12f4073ec0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f12e000e9c0 tx=0x7f12e000ee80 comp rx=0 tx=0).stop 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 shutdown_connections 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12f4108f60 0x7f12f406cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12f41047e0 0x7f12f4073ec0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f12dc0a2f10 0x7f12dc0a53d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 --2- 172.21.15.71:0/4185569448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f12f4103e30 0x7f12f4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 >> 172.21.15.71:0/4185569448 conn(0x7f12f40ff910 msgr2=0x7f12f4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:50.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.549+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 shutdown_connections 2024-09-05T12:28:50.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:28:50.550+0000 7f12fac00640 1 -- 172.21.15.71:0/4185569448 wait complete. 2024-09-05T12:28:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:50 smithi123 ceph-mon[34321]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-05T12:28:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:50 smithi174 ceph-mon[34313]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-05T12:28:51.493 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5a638e43', total seconds elapsed: 22.4 2024-09-05T12:28:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:52 smithi071 ceph-mon[29845]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-05T12:28:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:28:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:52 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:52 smithi123 ceph-mon[34321]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-05T12:28:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:28:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:52 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:52 smithi174 ceph-mon[34313]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-05T12:28:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:28:52.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:52.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:52 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:53.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:28:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:28:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:53.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:28:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:28:53.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:28:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:28:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:28:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 -- 172.21.15.71:0/1294754893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 msgr2=0x7f374010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 --2- 172.21.15.71:0/1294754893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374010b8e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f372c009900 tx=0x7f372c02a480 comp rx=0 tx=0).stop 2024-09-05T12:28:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 -- 172.21.15.71:0/1294754893 shutdown_connections 2024-09-05T12:28:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 --2- 172.21.15.71:0/1294754893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374010bf00 0x7f374010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 --2- 172.21.15.71:0/1294754893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374010b8e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 --2- 172.21.15.71:0/1294754893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 0x7f3740102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 -- 172.21.15.71:0/1294754893 >> 172.21.15.71:0/1294754893 conn(0x7f37400777c0 msgr2=0x7f37400feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 -- 172.21.15.71:0/1294754893 shutdown_connections 2024-09-05T12:28:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.651+0000 7f373fe00640 1 -- 172.21.15.71:0/1294754893 wait complete. 2024-09-05T12:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 Processor -- start 2024-09-05T12:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 -- start start 2024-09-05T12:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 0x7f374019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374010bf00 0x7f374019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3740111cd0 con 0x7f3740103400 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3740111b50 con 0x7f3740102ae0 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.652+0000 7f373fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3740111e50 con 0x7f374010bf00 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374019a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38908/0 (socket says 172.21.15.71:38908) 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 -- 172.21.15.71:0/1144195501 learned_addr learned my addr 172.21.15.71:0/1144195501 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373ea00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 0x7f374019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373f400640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374010bf00 0x7f374019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 -- 172.21.15.71:0/1144195501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374010bf00 msgr2=0x7f374019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374010bf00 0x7f374019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 -- 172.21.15.71:0/1144195501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 msgr2=0x7f374019a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 0x7f374019a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373e000640 1 -- 172.21.15.71:0/1144195501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374019f410 con 0x7f3740103400 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.653+0000 7f373ea00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 0x7f374019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.654+0000 7f373e000640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374019a960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f372c006010 tx=0x7f372c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.654+0000 7f3737400640 1 -- 172.21.15.71:0/1144195501 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c03f030 con 0x7f3740103400 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.654+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374019f6a0 con 0x7f3740103400 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.654+0000 7f3737400640 1 -- 172.21.15.71:0/1144195501 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c003ce0 con 0x7f3740103400 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.654+0000 7f3737400640 1 -- 172.21.15.71:0/1144195501 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c035070 con 0x7f3740103400 2024-09-05T12:28:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.654+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401a6f90 con 0x7f3740103400 2024-09-05T12:28:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.657+0000 7f3748a2e740 1 Processor -- start 2024-09-05T12:28:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3737400640 1 -- 172.21.15.71:0/1144195501 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f372c042030 con 0x7f3740103400 2024-09-05T12:28:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 -- start start 2024-09-05T12:28:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6838790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6843030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6845960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560bd679e100 con 0x560bd68383b0 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560bd67461b0 con 0x560bd6838cd0 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f3748a2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bd679a0c0 con 0x560bd6843570 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f373f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6838790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f373ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6843030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f373f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6838790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38916/0 (socket says 172.21.15.71:38916) 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.658+0000 7f373f400640 1 -- 172.21.15.71:0/4006141575 learned_addr learned my addr 172.21.15.71:0/4006141575 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373e000640 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6845960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373ea00640 1 -- 172.21.15.71:0/4006141575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 msgr2=0x560bd6845960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373ea00640 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6845960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373ea00640 1 -- 172.21.15.71:0/4006141575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 msgr2=0x560bd6838790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373ea00640 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6838790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373ea00640 1 -- 172.21.15.71:0/4006141575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd64b1210 con 0x560bd6838cd0 2024-09-05T12:28:54.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373f400640 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6838790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f373ea00640 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6843030 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3730009fd0 tx=0x7f37300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5749a5216acb4051 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f3737400640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37240a2e10 0x7f37240a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f372be00640 1 -- 172.21.15.71:0/4006141575 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730004030 con 0x560bd6838cd0 2024-09-05T12:28:54.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:54 smithi071 ceph-mon[29845]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-05T12:28:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f372be00640 1 -- 172.21.15.71:0/4006141575 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3730005500 con 0x560bd6838cd0 2024-09-05T12:28:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.659+0000 7f372be00640 1 -- 172.21.15.71:0/4006141575 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730004d50 con 0x560bd6838cd0 2024-09-05T12:28:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.660+0000 7f3737400640 1 -- 172.21.15.71:0/1144195501 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f372c0eff10 con 0x7f3740103400 2024-09-05T12:28:54.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.660+0000 7f373f400640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37240a2e10 0x7f37240a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.660+0000 7f3748a2e740 1 -- 172.21.15.71:0/4006141575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 msgr2=0x560bd6843030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.660+0000 7f3748a2e740 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6843030 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3730009fd0 tx=0x7f37300118d0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.660+0000 7f373f400640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37240a2e10 0x7f37240a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3738005f40 tx=0x7f3738009600 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 -- 172.21.15.71:0/4006141575 shutdown_connections 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6845960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6843030 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 --2- 172.21.15.71:0/4006141575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6838790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 -- 172.21.15.71:0/4006141575 >> 172.21.15.71:0/4006141575 conn(0x560bd6834920 msgr2=0x560bd6834d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 -- 172.21.15.71:0/4006141575 shutdown_connections 2024-09-05T12:28:54.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 -- 172.21.15.71:0/4006141575 wait complete. 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 -- start start 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.661+0000 7f3748a2e740 1 Processor -- start 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6840600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6840b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6841080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560bd6846050 con 0x560bd68383b0 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560bd6727090 con 0x560bd6838cd0 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bd68461d0 con 0x560bd6843570 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f3748a2e740 10 client.0 _ll_register_callbacks cb 0x560bd677bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f373ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6840600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f373f400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6841080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f373ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6840600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38918/0 (socket says 172.21.15.71:38918) 2024-09-05T12:28:54.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f373ea00640 1 -- 172.21.15.71:0/483472292 learned_addr learned my addr 172.21.15.71:0/483472292 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.662+0000 7f373e000640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6840b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373ea00640 1 -- 172.21.15.71:0/483472292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 msgr2=0x560bd6841080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373ea00640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6841080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373ea00640 1 -- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 msgr2=0x560bd6840b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373ea00640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6840b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373ea00640 1 -- 172.21.15.71:0/483472292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd657d620 con 0x560bd68383b0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373f400640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6841080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f373ea00640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6840600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3730004600 tx=0x7f373000fc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1b0779b4a76c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3728c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3728c00640 1 -- 172.21.15.71:0/483472292 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730028070 con 0x560bd68383b0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3730005500 con 0x560bd68383b0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3728c00640 1 -- 172.21.15.71:0/483472292 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f3730019660 con 0x560bd68383b0 2024-09-05T12:28:54.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3728c00640 1 -- 172.21.15.71:0/483472292 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3730034050 con 0x560bd68383b0 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3748a2e740 10 client.15507 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.663+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bd6851e10 con 0x560bd68383b0 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3728c00640 1 -- 172.21.15.71:0/483472292 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3730005500 con 0x560bd68383b0 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3728c00640 1 -- 172.21.15.71:0/483472292 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f37300245d0 con 0x560bd68383b0 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3748a2e740 10 client.15507 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3748a2e740 10 client.15507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3748a2e740 20 client.15507 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3748a2e740 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd6853220 0x560bd6855610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3748a2e740 4 client.15507 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bd67ab830 con 0x560bd6853220 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.664+0000 7f373e000640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd6853220 0x560bd6855610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.665+0000 7f373e000640 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd6853220 0x560bd6855610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b90e40ecc95ad685 server_cookie=6600b2ac57a35a97 in_seq=0 out_seq=0 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.665+0000 7f3728c00640 10 client.15507 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3728c00640 1 -- 172.21.15.71:0/483472292 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bd67ab830 con 0x560bd6853220 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3728c00640 10 client.15507 handle_command_reply: tid=0 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 client.15507 shutdown 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 msgr2=0x560bd6840600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6840600 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3730004600 tx=0x7f373000fc90 comp rx=0 tx=0).stop 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 shutdown_connections 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd6853220 0x560bd6855610 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd6841080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd6840b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 --2- 172.21.15.71:0/483472292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd6840600 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 >> 172.21.15.71:0/483472292 conn(0x560bd6834920 msgr2=0x560bd683a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:54.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3728c00640 0 client.15507 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 shutdown_connections 2024-09-05T12:28:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 1 -- 172.21.15.71:0/483472292 wait complete. 2024-09-05T12:28:54.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.666+0000 7f3748a2e740 20 client.15507 trim_cache size 0 max 16384 2024-09-05T12:28:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:54 smithi123 ceph-mon[34321]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.681+0000 7f3748a2e740 1 Processor -- start 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.681+0000 7f3748a2e740 1 -- start start 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd685ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd685b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd683d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f3748a2e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37401b0ef0 con 0x560bd68383b0 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f3748a2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37401b2a30 con 0x560bd6838cd0 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f3748a2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37401b15c0 con 0x560bd6843570 2024-09-05T12:28:54.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd685ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd685b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd685ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38928/0 (socket says 172.21.15.71:38928) 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373ea00640 1 -- 172.21.15.71:0/2509513499 learned_addr learned my addr 172.21.15.71:0/2509513499 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd685b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54514/0 (socket says 172.21.15.71:54514) 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373f400640 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd683d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 -- 172.21.15.71:0/2509513499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 msgr2=0x560bd683d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd683d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 -- 172.21.15.71:0/2509513499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 msgr2=0x560bd685ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd685ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373e000640 1 -- 172.21.15.71:0/2509513499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bd68403d0 con 0x560bd6838cd0 2024-09-05T12:28:54.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.682+0000 7f373ea00640 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd685ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:28:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.683+0000 7f373e000640 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd685b270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f372c0ed140 tx=0x7f372c03b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0a9ad53e4c0d8af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:54.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.683+0000 7f372b400640 1 -- 172.21.15.71:0/2509513499 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c049040 con 0x560bd6838cd0 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.683+0000 7f372b400640 1 -- 172.21.15.71:0/2509513499 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c0bc080 con 0x560bd6838cd0 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.683+0000 7f372b400640 1 -- 172.21.15.71:0/2509513499 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c0c4c40 con 0x560bd6838cd0 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 -- 172.21.15.71:0/2509513499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 msgr2=0x560bd685b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd685b270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f372c0ed140 tx=0x7f372c03b5d0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 -- 172.21.15.71:0/2509513499 shutdown_connections 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd683d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd6838cd0 0x560bd685b270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 --2- 172.21.15.71:0/2509513499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd68383b0 0x560bd685ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 -- 172.21.15.71:0/2509513499 >> 172.21.15.71:0/2509513499 conn(0x560bd6834920 msgr2=0x560bd683b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 -- 172.21.15.71:0/2509513499 shutdown_connections 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 -- 172.21.15.71:0/2509513499 wait complete. 2024-09-05T12:28:54.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 -- start start 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.684+0000 7f3748a2e740 1 Processor -- start 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 0x560bd6846a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd6838cd0 0x560bd6846f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd68bbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37401b2360 con 0x560bd6838cd0 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37401b1c90 con 0x560bd68383b0 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37401aeed0 con 0x560bd6843570 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f3748a2e740 10 client.0 _ll_register_callbacks cb 0x7f3738009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f373ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 0x560bd6846a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f373e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd6838cd0 0x560bd6846f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f373ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 0x560bd6846a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54518/0 (socket says 172.21.15.71:54518) 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f373ea00640 1 -- 172.21.15.71:0/1100531202 learned_addr learned my addr 172.21.15.71:0/1100531202 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:28:54.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f373f400640 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd68bbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.685+0000 7f373ea00640 1 -- 172.21.15.71:0/1100531202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 msgr2=0x560bd68bbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f373ea00640 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd68bbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f373ea00640 1 -- 172.21.15.71:0/1100531202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd6838cd0 msgr2=0x560bd6846f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f373ea00640 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd6838cd0 0x560bd6846f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f373ea00640 1 -- 172.21.15.71:0/1100531202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bd68bc300 con 0x560bd68383b0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f373ea00640 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 0x560bd6846a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3730011910 tx=0x7f3730009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbb004f3d36d59f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3728c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3728c00640 1 -- 172.21.15.71:0/1100531202 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730028040 con 0x560bd68383b0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3728c00640 1 -- 172.21.15.71:0/1100531202 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f3730024440 con 0x560bd68383b0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c0bc080 con 0x560bd68383b0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3728c00640 1 -- 172.21.15.71:0/1100531202 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3730041050 con 0x560bd68383b0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3748a2e740 10 client.25559 fetch_fsmap learned FSMap version 30 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.686+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560bd68bd5f0 con 0x560bd68383b0 2024-09-05T12:28:54.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3728c00640 1 -- 172.21.15.71:0/1100531202 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f372c0bc080 con 0x560bd68383b0 2024-09-05T12:28:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3728c00640 1 -- 172.21.15.71:0/1100531202 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f3730019820 con 0x560bd68383b0 2024-09-05T12:28:54.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3748a2e740 10 client.25559 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3748a2e740 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3748a2e740 20 client.25559 populate_metadata read hostname 'smithi071' 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3748a2e740 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd68bdc50 0x560bd68c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3748a2e740 4 client.25559 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.687+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bd67a5b80 con 0x560bd68bdc50 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.688+0000 7f373e000640 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd68bdc50 0x560bd68c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.688+0000 7f3728c00640 10 client.25559 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:54.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.688+0000 7f373e000640 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd68bdc50 0x560bd68c0070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d718331c0f298737 server_cookie=267518aa0978c9e3 in_seq=0 out_seq=0 2024-09-05T12:28:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.691+0000 7f3728c00640 1 -- 172.21.15.71:0/1100531202 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bd67a5b80 con 0x560bd68bdc50 2024-09-05T12:28:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.691+0000 7f3728c00640 10 client.25559 handle_command_reply: tid=0 2024-09-05T12:28:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 client.25559 shutdown 2024-09-05T12:28:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 msgr2=0x560bd6846a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 0x560bd6846a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3730011910 tx=0x7f3730009e90 comp rx=0 tx=0).stop 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 shutdown_connections 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560bd68bdc50 0x560bd68c0070 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bd6843570 0x560bd68bbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3728c00640 0 client.25559 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560bd6838cd0 0x560bd6846f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 --2- 172.21.15.71:0/1100531202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560bd68383b0 0x560bd6846a20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 >> 172.21.15.71:0/1100531202 conn(0x560bd6834920 msgr2=0x560bd68331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 shutdown_connections 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 1 -- 172.21.15.71:0/1100531202 wait complete. 2024-09-05T12:28:54.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.692+0000 7f3748a2e740 20 client.25559 trim_cache size 0 max 16384 2024-09-05T12:28:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:28:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:28:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:28:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:28:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37240a2e10 msgr2=0x7f37240a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.695 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37240a2e10 0x7f37240a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3738005f40 tx=0x7f3738009600 comp rx=0 tx=0).stop 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 msgr2=0x7f374019a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374019a960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f372c006010 tx=0x7f372c0042f0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 shutdown_connections 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f374010bf00 0x7f374019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37240a2e10 0x7f37240a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3740103400 0x7f374019a960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 --2- 172.21.15.71:0/1144195501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3740102ae0 0x7f374019a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 >> 172.21.15.71:0/1144195501 conn(0x7f37400777c0 msgr2=0x7f37401006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 shutdown_connections 2024-09-05T12:28:54.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:28:54.693+0000 7f373fe00640 1 -- 172.21.15.71:0/1144195501 wait complete. 2024-09-05T12:28:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:54 smithi174 ceph-mon[34313]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.1 KiB/s wr, 13 op/s 2024-09-05T12:28:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:56 smithi071 ceph-mon[29845]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-05T12:28:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:56 smithi123 ceph-mon[34321]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-05T12:28:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:56 smithi174 ceph-mon[34313]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-05T12:28:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:28:58 smithi071 ceph-mon[29845]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:28:58 smithi123 ceph-mon[34321]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:28:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:28:58 smithi174 ceph-mon[34313]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-05T12:29:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:00 smithi071 ceph-mon[29845]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:29:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:00 smithi123 ceph-mon[34321]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:29:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:00 smithi174 ceph-mon[34313]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:29:01.622 INFO:tasks.workunit.client.0.smithi071.stderr:Updating files: 66% (7980/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-05T12:29:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:02 smithi071 ceph-mon[29845]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:29:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:02 smithi123 ceph-mon[34321]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:29:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:02 smithi174 ceph-mon[34313]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:29:02.784 INFO:tasks.workunit.client.0.smithi071.stderr:Note: switching to 'e856d6b2462509a377ef39149a05475ed006ebac'. 2024-09-05T12:29:02.784 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.784 INFO:tasks.workunit.client.0.smithi071.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr:state without impacting any branches by switching back to a branch. 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: git switch -c 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr:Or undo this operation with: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: git switch - 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.785 INFO:tasks.workunit.client.0.smithi071.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-05T12:29:02.786 INFO:tasks.workunit.client.0.smithi071.stderr: 2024-09-05T12:29:02.786 INFO:tasks.workunit.client.0.smithi071.stderr:HEAD is now at e856d6b2462 Merge branch 'wip-67307-squid' of https://github.com/cbodley/ceph into wip-bharath6-testing-2024-09-05-1413-squid 2024-09-05T12:29:02.796 DEBUG:teuthology.orchestra.run.smithi071:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-05T12:29:02.890 INFO:tasks.workunit.client.0.smithi071.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-05T12:29:02.895 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T12:29:02.895 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-05T12:29:03.693 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-05T12:29:03.791 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-05T12:29:03.856 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-05T12:29:03.860 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T12:29:03.860 INFO:tasks.workunit.client.0.smithi071.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-05T12:29:03.924 INFO:tasks.workunit.client.0.smithi071.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-05T12:29:03.931 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:29:03.932 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-05T12:29:03.968 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-05T12:29:03.970 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-05T12:29:03.971 DEBUG:teuthology.orchestra.run.smithi071:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e856d6b2462509a377ef39149a05475ed006ebac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-05T12:29:04.058 INFO:tasks.workunit.client.0.smithi071.stderr:+ dbench 1 2024-09-05T12:29:04.077 INFO:tasks.workunit.client.0.smithi071.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-05T12:29:04.077 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-05T12:29:04.077 INFO:tasks.workunit.client.0.smithi071.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-05T12:29:04.088 INFO:tasks.workunit.client.0.smithi071.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-05T12:29:04.188 INFO:tasks.workunit.client.0.smithi071.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-05T12:29:04.188 INFO:tasks.workunit.client.0.smithi071.stdout:releasing clients 2024-09-05T12:29:04.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:04 smithi071 ceph-mon[29845]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:29:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:04 smithi123 ceph-mon[34321]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:29:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:04 smithi174 ceph-mon[34313]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-05T12:29:05.188 INFO:tasks.workunit.client.0.smithi071.stdout: 1 3626 38.60 MB/sec warmup 1 sec latency 8.291 ms 2024-09-05T12:29:06.188 INFO:tasks.workunit.client.0.smithi071.stdout: 1 6851 27.24 MB/sec warmup 2 sec latency 11.953 ms 2024-09-05T12:29:06.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:06 smithi071 ceph-mon[29845]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:29:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:06 smithi123 ceph-mon[34321]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:29:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:06 smithi174 ceph-mon[34313]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-05T12:29:07.188 INFO:tasks.workunit.client.0.smithi071.stdout: 1 10002 23.69 MB/sec warmup 3 sec latency 11.874 ms 2024-09-05T12:29:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 -- 172.21.15.71:0/2680752270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c1078c0 msgr2=0x7f911c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 --2- 172.21.15.71:0/2680752270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c1078c0 0x7f911c107ca0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9110009900 tx=0x7f911002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 -- 172.21.15.71:0/2680752270 shutdown_connections 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 --2- 172.21.15.71:0/2680752270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c102250 0x7f911c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 --2- 172.21.15.71:0/2680752270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911c1018b0 0x7f911c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 --2- 172.21.15.71:0/2680752270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c1078c0 0x7f911c107ca0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 -- 172.21.15.71:0/2680752270 >> 172.21.15.71:0/2680752270 conn(0x7f911c0fd5d0 msgr2=0x7f911c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 -- 172.21.15.71:0/2680752270 shutdown_connections 2024-09-05T12:29:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.842+0000 7f9122000640 1 -- 172.21.15.71:0/2680752270 wait complete. 2024-09-05T12:29:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.843+0000 7f9122000640 1 Processor -- start 2024-09-05T12:29:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.843+0000 7f9122000640 1 -- start start 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9122000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c1018b0 0x7f911c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9122000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c102250 0x7f911c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9122000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911c1078c0 0x7f911c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9122000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f911c111d20 con 0x7f911c1078c0 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9122000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f911c111ba0 con 0x7f911c102250 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9122000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f911c111ea0 con 0x7f911c1018b0 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9120c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c1018b0 0x7f911c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f911be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c102250 0x7f911c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9120c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c1018b0 0x7f911c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39112/0 (socket says 172.21.15.71:39112) 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9120c00640 1 -- 172.21.15.71:0/402021963 learned_addr learned my addr 172.21.15.71:0/402021963 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.844+0000 7f9121600640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911c1078c0 0x7f911c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f911be00640 1 -- 172.21.15.71:0/402021963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c1018b0 msgr2=0x7f911c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f911be00640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c1018b0 0x7f911c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f911be00640 1 -- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911c1078c0 msgr2=0x7f911c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f911be00640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911c1078c0 0x7f911c19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f911be00640 1 -- 172.21.15.71:0/402021963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c19f400 con 0x7f911c102250 2024-09-05T12:29:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f911be00640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c102250 0x7f911c19a950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f910800d6d0 tx=0x7f910800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f9119600640 1 -- 172.21.15.71:0/402021963 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108017070 con 0x7f911c102250 2024-09-05T12:29:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f9119600640 1 -- 172.21.15.71:0/402021963 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9108009820 con 0x7f911c102250 2024-09-05T12:29:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f9119600640 1 -- 172.21.15.71:0/402021963 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108004d00 con 0x7f911c102250 2024-09-05T12:29:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c19f6f0 con 0x7f911c102250 2024-09-05T12:29:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.845+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c1a6f80 con 0x7f911c102250 2024-09-05T12:29:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.848+0000 7f91265f2740 1 Processor -- start 2024-09-05T12:29:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 -- start start 2024-09-05T12:29:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43e92790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e9d570 0x557d43e9f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557d43df8100 con 0x557d43e923b0 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d43da01b0 con 0x557d43e92cd0 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f91265f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d43df40c0 con 0x557d43e9d570 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f9119600640 1 -- 172.21.15.71:0/402021963 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 25) ==== 138066+0+0 (secure 0 0 0) 0x7f91080099d0 con 0x7f911c102250 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f9120c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f9120c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48816/0 (socket says 172.21.15.71:48816) 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f9120c00640 1 -- 172.21.15.71:0/521069189 learned_addr learned my addr 172.21.15.71:0/521069189 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f9121600640 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43e92790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.849+0000 7f911be00640 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e9d570 0x557d43e9f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f9120c00640 1 -- 172.21.15.71:0/521069189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e9d570 msgr2=0x557d43e9f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f9120c00640 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e9d570 0x557d43e9f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f9120c00640 1 -- 172.21.15.71:0/521069189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 msgr2=0x557d43e92790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f9120c00640 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43e92790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f9120c00640 1 -- 172.21.15.71:0/521069189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d43b0b210 con 0x557d43e92cd0 2024-09-05T12:29:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f9120c00640 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9d030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9110005e90 tx=0x7f9110004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e1f6843e786f85 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.850+0000 7f910dc00640 1 -- 172.21.15.71:0/521069189 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110040070 con 0x557d43e92cd0 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.851+0000 7f910dc00640 1 -- 172.21.15.71:0/521069189 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110005500 con 0x557d43e92cd0 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.851+0000 7f910dc00640 1 -- 172.21.15.71:0/521069189 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110035040 con 0x557d43e92cd0 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.851+0000 7f9119600640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91040a2e90 0x7f91040a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.851+0000 7f9119600640 1 -- 172.21.15.71:0/402021963 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f91080cf3a0 con 0x7f911c102250 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.851+0000 7f91265f2740 1 -- 172.21.15.71:0/521069189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 msgr2=0x557d43e9d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.851+0000 7f91265f2740 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9d030 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9110005e90 tx=0x7f9110004040 comp rx=0 tx=0).stop 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f9121600640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91040a2e90 0x7f91040a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 -- 172.21.15.71:0/521069189 shutdown_connections 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e9d570 0x557d43e9f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9d030 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 --2- 172.21.15.71:0/521069189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43e92790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 -- 172.21.15.71:0/521069189 >> 172.21.15.71:0/521069189 conn(0x557d43e8e920 msgr2=0x557d43e8ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 -- 172.21.15.71:0/521069189 shutdown_connections 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 -- 172.21.15.71:0/521069189 wait complete. 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f9121600640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91040a2e90 0x7f91040a5350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9114005fd0 tx=0x7f9114009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 -- start start 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.852+0000 7f91265f2740 1 Processor -- start 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43e9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e9d570 0x557d43e9b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557d43e9fff0 con 0x557d43e9d570 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d43d81090 con 0x557d43e92cd0 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d43ea0170 con 0x557d43e923b0 2024-09-05T12:29:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f91265f2740 10 client.0 _ll_register_callbacks cb 0x557d43dd5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f9120c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43e9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f9120c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43e9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39130/0 (socket says 172.21.15.71:39130) 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f9120c00640 1 -- 172.21.15.71:0/324296687 learned_addr learned my addr 172.21.15.71:0/324296687 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f9121600640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e9d570 0x557d43e9b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.853+0000 7f911be00640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9120c00640 1 -- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 msgr2=0x557d43e9ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9120c00640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9120c00640 1 -- 172.21.15.71:0/324296687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e9d570 msgr2=0x557d43e9b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9120c00640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e9d570 0x557d43e9b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9120c00640 1 -- 172.21.15.71:0/324296687 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d43bd7620 con 0x557d43e923b0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9120c00640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43e9a730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f9110005890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96291c9c7188c04f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9102000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:07.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9102000640 1 -- 172.21.15.71:0/324296687 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110040070 con 0x557d43e923b0 2024-09-05T12:29:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9102000640 1 -- 172.21.15.71:0/324296687 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f9110005000 con 0x557d43e923b0 2024-09-05T12:29:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9110005500 con 0x557d43e923b0 2024-09-05T12:29:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f9102000640 1 -- 172.21.15.71:0/324296687 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9110056050 con 0x557d43e923b0 2024-09-05T12:29:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f91265f2740 10 client.25579 fetch_fsmap learned FSMap version 30 2024-09-05T12:29:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.854+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d43e9fff0 con 0x557d43e923b0 2024-09-05T12:29:07.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f9102000640 1 -- 172.21.15.71:0/324296687 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9110005500 con 0x557d43e923b0 2024-09-05T12:29:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f9102000640 1 -- 172.21.15.71:0/324296687 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9110031540 con 0x557d43e923b0 2024-09-05T12:29:07.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f91265f2740 10 client.25579 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f91265f2740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f91265f2740 20 client.25579 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f91265f2740 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43ead1d0 0x557d43eaf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f91265f2740 4 client.25579 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.855+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d43e05830 con 0x557d43ead1d0 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.856+0000 7f911be00640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43ead1d0 0x557d43eaf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.856+0000 7f911be00640 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43ead1d0 0x557d43eaf5e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b53e1c072c4eb7c5 server_cookie=6520cec8054dcdca in_seq=0 out_seq=0 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.856+0000 7f9102000640 10 client.25579 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:07.861 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f9102000640 1 -- 172.21.15.71:0/324296687 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d43e05830 con 0x557d43ead1d0 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f9102000640 10 client.25579 handle_command_reply: tid=0 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 client.25579 shutdown 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 msgr2=0x557d43e9a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43e9a730 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f9110005890 comp rx=0 tx=0).stop 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 shutdown_connections 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43ead1d0 0x557d43eaf5e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e9d570 0x557d43e9b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e92cd0 0x557d43e9ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 --2- 172.21.15.71:0/324296687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43e9a730 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f9102000640 0 client.25579 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 >> 172.21.15.71:0/324296687 conn(0x557d43e8e920 msgr2=0x557d43e94680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 shutdown_connections 2024-09-05T12:29:07.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.858+0000 7f91265f2740 1 -- 172.21.15.71:0/324296687 wait complete. 2024-09-05T12:29:07.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.859+0000 7f91265f2740 20 client.25579 trim_cache size 0 max 16384 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 Processor -- start 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 -- start start 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43eb4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 0x557d43eb51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e979b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1b0f40 con 0x557d43e923b0 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1b2a80 con 0x557d43e9d570 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f91265f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1b1610 con 0x557d43e92cd0 2024-09-05T12:29:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f911be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 0x557d43eb51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.875+0000 7f9120c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43eb4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 0x557d43eb51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39136/0 (socket says 172.21.15.71:39136) 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f9121600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 -- 172.21.15.71:0/239151561 learned_addr learned my addr 172.21.15.71:0/239151561 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 -- 172.21.15.71:0/239151561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 msgr2=0x557d43e979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 -- 172.21.15.71:0/239151561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 msgr2=0x557d43eb4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43eb4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 -- 172.21.15.71:0/239151561 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d43ea91a0 con 0x557d43e92cd0 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f9120c00640 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43eb4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f911be00640 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 0x557d43eb51d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f910809afd0 tx=0x7f91080d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75649930e97a1bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f910d200640 1 -- 172.21.15.71:0/239151561 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108020040 con 0x557d43e92cd0 2024-09-05T12:29:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f910d200640 1 -- 172.21.15.71:0/239151561 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910809b9d0 con 0x557d43e92cd0 2024-09-05T12:29:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.876+0000 7f910d200640 1 -- 172.21.15.71:0/239151561 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9108009d40 con 0x557d43e92cd0 2024-09-05T12:29:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 -- 172.21.15.71:0/239151561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 msgr2=0x557d43eb51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 0x557d43eb51d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f910809afd0 tx=0x7f91080d1040 comp rx=0 tx=0).stop 2024-09-05T12:29:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 -- 172.21.15.71:0/239151561 shutdown_connections 2024-09-05T12:29:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e92cd0 0x557d43eb51d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 --2- 172.21.15.71:0/239151561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e923b0 0x557d43eb4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 -- 172.21.15.71:0/239151561 >> 172.21.15.71:0/239151561 conn(0x557d43e8e920 msgr2=0x557d43e959d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 -- 172.21.15.71:0/239151561 shutdown_connections 2024-09-05T12:29:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.877+0000 7f91265f2740 1 -- 172.21.15.71:0/239151561 wait complete. 2024-09-05T12:29:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 -- start start 2024-09-05T12:29:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 Processor -- start 2024-09-05T12:29:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43ea5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 0x557d43ea5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e96fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1b23b0 con 0x557d43e92cd0 2024-09-05T12:29:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1b1ce0 con 0x557d43e9d570 2024-09-05T12:29:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1aef20 con 0x557d43e923b0 2024-09-05T12:29:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.878+0000 7f91265f2740 10 client.0 _ll_register_callbacks cb 0x7f91140099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43ea5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f911be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 0x557d43ea5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f911be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 0x557d43ea5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37120/0 (socket says 172.21.15.71:37120) 2024-09-05T12:29:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f911be00640 1 -- 172.21.15.71:0/1127049967 learned_addr learned my addr 172.21.15.71:0/1127049967 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9121600640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e96fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 -- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 msgr2=0x557d43e96fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e96fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 -- 172.21.15.71:0/1127049967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 msgr2=0x557d43ea5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 0x557d43ea5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 -- 172.21.15.71:0/1127049967 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d43e97690 con 0x557d43e923b0 2024-09-05T12:29:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f911be00640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 0x557d43ea5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9120c00640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43ea5450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9110031b00 tx=0x7f9110004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eae4c692f7c26292 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.879+0000 7f9102000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.880+0000 7f9102000640 1 -- 172.21.15.71:0/1127049967 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f911005f060 con 0x557d43e923b0 2024-09-05T12:29:07.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.880+0000 7f9102000640 1 -- 172.21.15.71:0/1127049967 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f9110031b80 con 0x557d43e923b0 2024-09-05T12:29:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.880+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910809b9d0 con 0x557d43e923b0 2024-09-05T12:29:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.880+0000 7f9102000640 1 -- 172.21.15.71:0/1127049967 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9110040040 con 0x557d43e923b0 2024-09-05T12:29:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.880+0000 7f91265f2740 10 client.25585 fetch_fsmap learned FSMap version 30 2024-09-05T12:29:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.880+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d43f175d0 con 0x557d43e923b0 2024-09-05T12:29:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f9102000640 1 -- 172.21.15.71:0/1127049967 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f910809b9d0 con 0x557d43e923b0 2024-09-05T12:29:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f9102000640 1 -- 172.21.15.71:0/1127049967 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f91100035f0 con 0x557d43e923b0 2024-09-05T12:29:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f91265f2740 10 client.25585 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:29:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f91265f2740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f91265f2740 20 client.25585 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f91265f2740 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43f17be0 0x557d43f19fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f91265f2740 4 client.25585 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:29:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d43dffb80 con 0x557d43f17be0 2024-09-05T12:29:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.881+0000 7f911be00640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43f17be0 0x557d43f19fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.882+0000 7f9102000640 10 client.25585 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.882+0000 7f911be00640 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43f17be0 0x557d43f19fd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f6d61bd4eaf0b2 server_cookie=a31ef9b680bf46c4 in_seq=0 out_seq=0 2024-09-05T12:29:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f9102000640 1 -- 172.21.15.71:0/1127049967 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d43dffb80 con 0x557d43f17be0 2024-09-05T12:29:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f9102000640 10 client.25585 handle_command_reply: tid=0 2024-09-05T12:29:07.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 client.25585 shutdown 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 msgr2=0x557d43ea5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43ea5450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9110031b00 tx=0x7f9110004cf0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 shutdown_connections 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557d43f17be0 0x557d43f19fd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d43e9d570 0x557d43e96fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d43e92cd0 0x557d43ea5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 --2- 172.21.15.71:0/1127049967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d43e923b0 0x557d43ea5450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f9102000640 0 client.25585 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 >> 172.21.15.71:0/1127049967 conn(0x557d43e8e920 msgr2=0x557d43e9a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 shutdown_connections 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 1 -- 172.21.15.71:0/1127049967 wait complete. 2024-09-05T12:29:07.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.995+0000 7f91265f2740 20 client.25585 trim_cache size 0 max 16384 2024-09-05T12:29:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:29:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:29:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:29:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.996+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91040a2e90 msgr2=0x7f91040a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.996+0000 7f9122000640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91040a2e90 0x7f91040a5350 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9114005fd0 tx=0x7f9114009500 comp rx=0 tx=0).stop 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.996+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c102250 msgr2=0x7f911c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.996+0000 7f9122000640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c102250 0x7f911c19a950 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f910800d6d0 tx=0x7f910800db90 comp rx=0 tx=0).stop 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 shutdown_connections 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91040a2e90 0x7f91040a5350 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f911c1078c0 0x7f911c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f911c102250 0x7f911c19a950 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 --2- 172.21.15.71:0/402021963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f911c1018b0 0x7f911c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 >> 172.21.15.71:0/402021963 conn(0x7f911c0fd5d0 msgr2=0x7f911c10ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 shutdown_connections 2024-09-05T12:29:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:07.997+0000 7f9122000640 1 -- 172.21.15.71:0/402021963 wait complete. 2024-09-05T12:29:08.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 12308 20.89 MB/sec warmup 4 sec latency 57.372 ms 2024-09-05T12:29:08.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:08 smithi071 ceph-mon[29845]: pgmap v44: 129 pgs: 129 active+clean; 3.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 232 KiB/s wr, 51 op/s 2024-09-05T12:29:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:08 smithi123 ceph-mon[34321]: pgmap v44: 129 pgs: 129 active+clean; 3.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 232 KiB/s wr, 51 op/s 2024-09-05T12:29:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:08 smithi174 ceph-mon[34313]: pgmap v44: 129 pgs: 129 active+clean; 3.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 232 KiB/s wr, 51 op/s 2024-09-05T12:29:09.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 15188 19.68 MB/sec warmup 5 sec latency 16.554 ms 2024-09-05T12:29:10.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 18429 19.46 MB/sec warmup 6 sec latency 13.298 ms 2024-09-05T12:29:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:10 smithi071 ceph-mon[29845]: pgmap v45: 129 pgs: 129 active+clean; 3.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 232 KiB/s wr, 46 op/s 2024-09-05T12:29:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:10 smithi123 ceph-mon[34321]: pgmap v45: 129 pgs: 129 active+clean; 3.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 232 KiB/s wr, 46 op/s 2024-09-05T12:29:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:10 smithi174 ceph-mon[34313]: pgmap v45: 129 pgs: 129 active+clean; 3.6 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 232 KiB/s wr, 46 op/s 2024-09-05T12:29:11.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 21701 19.39 MB/sec warmup 7 sec latency 12.509 ms 2024-09-05T12:29:12.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 25031 19.19 MB/sec warmup 8 sec latency 10.786 ms 2024-09-05T12:29:12.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:12 smithi071 ceph-mon[29845]: pgmap v46: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1017 KiB/s wr, 211 op/s 2024-09-05T12:29:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:12 smithi123 ceph-mon[34321]: pgmap v46: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1017 KiB/s wr, 211 op/s 2024-09-05T12:29:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:12 smithi174 ceph-mon[34313]: pgmap v46: 129 pgs: 129 active+clean; 11 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 1017 KiB/s wr, 211 op/s 2024-09-05T12:29:13.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 27736 18.45 MB/sec warmup 9 sec latency 12.051 ms 2024-09-05T12:29:14.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 30786 18.09 MB/sec warmup 10 sec latency 19.253 ms 2024-09-05T12:29:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:14 smithi071 ceph-mon[29845]: pgmap v47: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1023 KiB/s wr, 214 op/s 2024-09-05T12:29:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:14 smithi123 ceph-mon[34321]: pgmap v47: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1023 KiB/s wr, 214 op/s 2024-09-05T12:29:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:14 smithi174 ceph-mon[34313]: pgmap v47: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1023 KiB/s wr, 214 op/s 2024-09-05T12:29:15.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 33228 17.75 MB/sec warmup 11 sec latency 15.727 ms 2024-09-05T12:29:16.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 36404 17.74 MB/sec warmup 12 sec latency 11.184 ms 2024-09-05T12:29:16.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:16 smithi071 ceph-mon[29845]: pgmap v48: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1023 KiB/s wr, 213 op/s 2024-09-05T12:29:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:16 smithi123 ceph-mon[34321]: pgmap v48: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1023 KiB/s wr, 213 op/s 2024-09-05T12:29:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:16 smithi174 ceph-mon[34313]: pgmap v48: 129 pgs: 129 active+clean; 11 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1023 KiB/s wr, 213 op/s 2024-09-05T12:29:17.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 39497 17.79 MB/sec warmup 13 sec latency 44.271 ms 2024-09-05T12:29:18.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 42689 17.79 MB/sec warmup 14 sec latency 19.522 ms 2024-09-05T12:29:18.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:18 smithi071 ceph-mon[29845]: pgmap v49: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.7 MiB/s wr, 317 op/s 2024-09-05T12:29:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:18 smithi071 ceph-mon[29845]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-05T12:29:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:18 smithi123 ceph-mon[34321]: pgmap v49: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.7 MiB/s wr, 317 op/s 2024-09-05T12:29:18.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:18 smithi123 ceph-mon[34321]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-05T12:29:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:18 smithi174 ceph-mon[34313]: pgmap v49: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 1.7 MiB/s wr, 317 op/s 2024-09-05T12:29:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:18 smithi174 ceph-mon[34313]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-05T12:29:19.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 45570 17.53 MB/sec warmup 15 sec latency 10.899 ms 2024-09-05T12:29:20.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 48114 17.19 MB/sec warmup 16 sec latency 24.424 ms 2024-09-05T12:29:20.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:20 smithi071 ceph-mon[29845]: pgmap v50: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-05T12:29:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:20 smithi123 ceph-mon[34321]: pgmap v50: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-05T12:29:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:20 smithi174 ceph-mon[34313]: pgmap v50: 129 pgs: 129 active+clean; 18 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 804 KiB/s rd, 1.5 MiB/s wr, 275 op/s 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.131+0000 7fbe2f200640 1 -- 172.21.15.71:0/1138829331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe30103b00 msgr2=0x7fbe30103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.131+0000 7fbe2f200640 1 --2- 172.21.15.71:0/1138829331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe30103b00 0x7fbe30103f60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c009930 tx=0x7fbe1c02a420 comp rx=0 tx=0).stop 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 -- 172.21.15.71:0/1138829331 shutdown_connections 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 --2- 172.21.15.71:0/1138829331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe301044a0 0x7fbe301109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 --2- 172.21.15.71:0/1138829331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe30103b00 0x7fbe30103f60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 --2- 172.21.15.71:0/1138829331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe30109ba0 0x7fbe30109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 -- 172.21.15.71:0/1138829331 >> 172.21.15.71:0/1138829331 conn(0x7fbe300ff7c0 msgr2=0x7fbe30101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 -- 172.21.15.71:0/1138829331 shutdown_connections 2024-09-05T12:29:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.132+0000 7fbe2f200640 1 -- 172.21.15.71:0/1138829331 wait complete. 2024-09-05T12:29:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 Processor -- start 2024-09-05T12:29:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 -- start start 2024-09-05T12:29:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe30103b00 0x7fbe3019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe301044a0 0x7fbe3019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe30109ba0 0x7fbe3019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30113f60 con 0x7fbe30103b00 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe30113de0 con 0x7fbe30109ba0 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.133+0000 7fbe2f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301140e0 con 0x7fbe301044a0 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe301044a0 0x7fbe3019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe301044a0 0x7fbe3019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:54154/0 (socket says 172.21.15.71:54154) 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe30109ba0 0x7fbe3019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2d400640 1 -- 172.21.15.71:0/662698798 learned_addr learned my addr 172.21.15.71:0/662698798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2de00640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe30103b00 0x7fbe3019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 -- 172.21.15.71:0/662698798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe301044a0 msgr2=0x7fbe3019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe301044a0 0x7fbe3019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 -- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe30103b00 msgr2=0x7fbe3019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe30103b00 0x7fbe3019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 -- 172.21.15.71:0/662698798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe3019f480 con 0x7fbe30109ba0 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.134+0000 7fbe2e800640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe30109ba0 0x7fbe3019ed50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fd0 tx=0x7fbe2800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.135+0000 7fbe26a00640 1 -- 172.21.15.71:0/662698798 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800ce20 con 0x7fbe30109ba0 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.135+0000 7fbe26a00640 1 -- 172.21.15.71:0/662698798 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe28005020 con 0x7fbe30109ba0 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.135+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe3019f770 con 0x7fbe30109ba0 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.135+0000 7fbe26a00640 1 -- 172.21.15.71:0/662698798 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28010720 con 0x7fbe30109ba0 2024-09-05T12:29:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.135+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe301a7000 con 0x7fbe30109ba0 2024-09-05T12:29:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 Processor -- start 2024-09-05T12:29:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe26a00640 1 -- 172.21.15.71:0/662698798 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbe28005540 con 0x7fbe30109ba0 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 -- start start 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c52399d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c523903100 con 0x55c5239a8570 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c5238ab1b0 con 0x55c52399dcd0 2024-09-05T12:29:21.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe381ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c5238ff0c0 con 0x55c52399d3b0 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe2d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c52399d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36722/0 (socket says 172.21.15.71:36722) 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe2de00640 1 -- 172.21.15.71:0/3572779504 learned_addr learned my addr 172.21.15.71:0/3572779504 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.138+0000 7fbe2d400640 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2de00640 1 -- 172.21.15.71:0/3572779504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 msgr2=0x55c52399d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2de00640 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c52399d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2de00640 1 -- 172.21.15.71:0/3572779504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 msgr2=0x55c5239aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2de00640 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2de00640 1 -- 172.21.15.71:0/3572779504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c523616210 con 0x55c52399dcd0 2024-09-05T12:29:21.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe26a00640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe140a2d70 0x7fbe140a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2de00640 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a8030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bc50 tx=0x7fbe200107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9821dfd350cf4d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe26a00640 1 -- 172.21.15.71:0/662698798 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbe280cfe70 con 0x7fbe30109ba0 2024-09-05T12:29:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe2d400640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe140a2d70 0x7fbe140a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe1b400640 1 -- 172.21.15.71:0/3572779504 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2001c480 con 0x55c52399dcd0 2024-09-05T12:29:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.139+0000 7fbe1b400640 1 -- 172.21.15.71:0/3572779504 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2001cb00 con 0x55c52399dcd0 2024-09-05T12:29:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe1b400640 1 -- 172.21.15.71:0/3572779504 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20011c70 con 0x55c52399dcd0 2024-09-05T12:29:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 -- 172.21.15.71:0/3572779504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 msgr2=0x55c5239a8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe2d400640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe140a2d70 0x7fbe140a5230 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0096c0 tx=0x7fbe1c041040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a8030 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bc50 tx=0x7fbe200107a0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 -- 172.21.15.71:0/3572779504 shutdown_connections 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a8030 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3572779504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c52399d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.140+0000 7fbe381ff740 1 -- 172.21.15.71:0/3572779504 >> 172.21.15.71:0/3572779504 conn(0x55c523999920 msgr2=0x55c523999d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.141+0000 7fbe381ff740 1 -- 172.21.15.71:0/3572779504 shutdown_connections 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.141+0000 7fbe381ff740 1 -- 172.21.15.71:0/3572779504 wait complete. 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.141+0000 7fbe381ff740 1 -- start start 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.141+0000 7fbe381ff740 1 Processor -- start 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 0x55c5239a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5239a8570 0x55c5239a6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c5239aaff0 con 0x55c52399d3b0 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c52388c090 con 0x55c52399dcd0 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c5239ab170 con 0x55c5239a8570 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe381ff740 10 client.0 _ll_register_callbacks cb 0x55c5238e0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 0x55c5239a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 0x55c5239a5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45670/0 (socket says 172.21.15.71:45670) 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2de00640 1 -- 172.21.15.71:0/1542106024 learned_addr learned my addr 172.21.15.71:0/1542106024 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2d400640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2e800640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5239a8570 0x55c5239a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2de00640 1 -- 172.21.15.71:0/1542106024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5239a8570 msgr2=0x55c5239a6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2de00640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5239a8570 0x55c5239a6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.142+0000 7fbe2de00640 1 -- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 msgr2=0x55c5239a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe2de00640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe2de00640 1 -- 172.21.15.71:0/1542106024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5236e2620 con 0x55c52399d3b0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe2de00640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 0x55c5239a5710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bc20 tx=0x7fbe200114f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37aac4e27a47dd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1542106024 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2001c690 con 0x55c52399d3b0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1542106024 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbe2001c830 con 0x55c52399d3b0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2001cb00 con 0x55c52399d3b0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1542106024 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe20005030 con 0x55c52399d3b0 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe381ff740 10 client.15543 fetch_fsmap learned FSMap version 30 2024-09-05T12:29:21.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.143+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c5239b6de0 con 0x55c52399d3b0 2024-09-05T12:29:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1542106024 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe2001cb00 con 0x55c52399d3b0 2024-09-05T12:29:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1542106024 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbe200046c0 con 0x55c52399d3b0 2024-09-05T12:29:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe381ff740 10 client.15543 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:29:21.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe381ff740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe381ff740 20 client.15543 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c5239b81e0 0x55c5239ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe381ff740 4 client.15543 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c523910830 con 0x55c5239b81e0 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.144+0000 7fbe2d400640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c5239b81e0 0x55c5239ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.145+0000 7fbe2d400640 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c5239b81e0 0x55c5239ba5d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4cebc694e3beb4 server_cookie=39382aa94811cf7d in_seq=0 out_seq=0 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.145+0000 7fbe0fe00640 10 client.15543 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1542106024 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c523910830 con 0x55c5239b81e0 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe0fe00640 10 client.15543 handle_command_reply: tid=0 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 client.15543 shutdown 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 msgr2=0x55c5239a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 0x55c5239a5710 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bc20 tx=0x7fbe200114f0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 shutdown_connections 2024-09-05T12:29:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c5239b81e0 0x55c5239ba5d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c5239a8570 0x55c5239a6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239a5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1542106024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c52399d3b0 0x55c5239a5710 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 >> 172.21.15.71:0/1542106024 conn(0x55c523999920 msgr2=0x55c52399f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe0fe00640 0 client.15543 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 shutdown_connections 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.146+0000 7fbe381ff740 1 -- 172.21.15.71:0/1542106024 wait complete. 2024-09-05T12:29:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.147+0000 7fbe381ff740 20 client.15543 trim_cache size 0 max 16384 2024-09-05T12:29:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.164+0000 7fbe381ff740 1 Processor -- start 2024-09-05T12:29:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.164+0000 7fbe381ff740 1 -- start start 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.164+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 0x55c5239a7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399dcd0 0x55c5239bfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe381ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301b0f20 con 0x55c5239a8570 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe381ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301b2a60 con 0x55c52399d3b0 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe381ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301b15f0 con 0x55c52399dcd0 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 0x55c5239a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 0x55c5239a7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36736/0 (socket says 172.21.15.71:36736) 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399dcd0 0x55c5239bfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 -- 172.21.15.71:0/3890731629 learned_addr learned my addr 172.21.15.71:0/3890731629 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:21.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 -- 172.21.15.71:0/3890731629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399dcd0 msgr2=0x55c5239bfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399dcd0 0x55c5239bfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 -- 172.21.15.71:0/3890731629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 msgr2=0x55c5239c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 -- 172.21.15.71:0/3890731629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5239a54e0 con 0x55c52399d3b0 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2d400640 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399dcd0 0x55c5239bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2e800640 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.165+0000 7fbe2de00640 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 0x55c5239a7570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bc50 tx=0x7fbe200058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef76e9c71216656 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.166+0000 7fbe1aa00640 1 -- 172.21.15.71:0/3890731629 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe20010ca0 con 0x55c52399d3b0 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.166+0000 7fbe1aa00640 1 -- 172.21.15.71:0/3890731629 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe20011560 con 0x55c52399d3b0 2024-09-05T12:29:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.166+0000 7fbe1aa00640 1 -- 172.21.15.71:0/3890731629 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe200048c0 con 0x55c52399d3b0 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 -- 172.21.15.71:0/3890731629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 msgr2=0x55c5239a7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 0x55c5239a7570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000bc50 tx=0x7fbe200058d0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 -- 172.21.15.71:0/3890731629 shutdown_connections 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399dcd0 0x55c5239bfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 --2- 172.21.15.71:0/3890731629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399d3b0 0x55c5239a7570 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 -- 172.21.15.71:0/3890731629 >> 172.21.15.71:0/3890731629 conn(0x55c523999920 msgr2=0x55c5239a6e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 -- 172.21.15.71:0/3890731629 shutdown_connections 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 -- 172.21.15.71:0/3890731629 wait complete. 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 -- start start 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.167+0000 7fbe381ff740 1 Processor -- start 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c5239b0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239abb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301b2390 con 0x55c5239a8570 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301b1cc0 con 0x55c52399dcd0 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe301aef00 con 0x55c52399d3b0 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe381ff740 10 client.0 _ll_register_callbacks cb 0x7fbe1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c5239b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c5239b0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:54186/0 (socket says 172.21.15.71:54186) 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36748/0 (socket says 172.21.15.71:36748) 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2de00640 1 -- 172.21.15.71:0/1799882332 learned_addr learned my addr 172.21.15.71:0/1799882332 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.168+0000 7fbe2e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239abb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45682/0 (socket says 172.21.15.71:45682) 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2de00640 1 -- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 msgr2=0x55c5239b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2de00640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2de00640 1 -- 172.21.15.71:0/1799882332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 msgr2=0x55c5239abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2de00640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2de00640 1 -- 172.21.15.71:0/1799882332 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5239ac3b0 con 0x55c52399d3b0 2024-09-05T12:29:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2d400640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2e800640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe2de00640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c5239b0610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe20005d00 tx=0x7fbe200059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e482bf9175d69117 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe0fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1799882332 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2001cb00 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe20011560 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1799882332 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbe20004660 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1799882332 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe20004860 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe381ff740 10 client.25603 fetch_fsmap learned FSMap version 30 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.169+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c523a22980 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1799882332 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe20011560 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1799882332 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbe20011cd0 con 0x55c52399d3b0 2024-09-05T12:29:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe381ff740 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe381ff740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe381ff740 20 client.25603 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c523a22eb0 0x55c523a252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe381ff740 4 client.25603 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c52390ab80 con 0x55c523a22eb0 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.170+0000 7fbe2d400640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c523a22eb0 0x55c523a252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.171+0000 7fbe2d400640 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c523a22eb0 0x55c523a252a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b45d0349bb949797 server_cookie=d86474117471f003 in_seq=0 out_seq=0 2024-09-05T12:29:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.171+0000 7fbe0fe00640 10 client.25603 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:21.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 50622 17.03 MB/sec warmup 17 sec latency 16.139 ms 2024-09-05T12:29:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.406+0000 7fbe0fe00640 1 -- 172.21.15.71:0/1799882332 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c52390ab80 con 0x55c523a22eb0 2024-09-05T12:29:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.406+0000 7fbe0fe00640 10 client.25603 handle_command_reply: tid=0 2024-09-05T12:29:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.406+0000 7fbe381ff740 1 client.25603 shutdown 2024-09-05T12:29:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 msgr2=0x55c5239b0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c5239b0610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbe20005d00 tx=0x7fbe200059b0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 shutdown_connections 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c523a22eb0 0x55c523a252a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5239a8570 0x55c5239abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c52399dcd0 0x55c5239b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 --2- 172.21.15.71:0/1799882332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52399d3b0 0x55c5239b0610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 >> 172.21.15.71:0/1799882332 conn(0x55c523999920 msgr2=0x55c52399f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe0fe00640 0 client.25603 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 shutdown_connections 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 1 -- 172.21.15.71:0/1799882332 wait complete. 2024-09-05T12:29:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.407+0000 7fbe381ff740 20 client.25603 trim_cache size 0 max 16384 2024-09-05T12:29:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:29:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:29:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:29:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:29:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe140a2d70 msgr2=0x7fbe140a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe140a2d70 0x7fbe140a5230 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c0096c0 tx=0x7fbe1c041040 comp rx=0 tx=0).stop 2024-09-05T12:29:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe30109ba0 msgr2=0x7fbe3019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe30109ba0 0x7fbe3019ed50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009fd0 tx=0x7fbe2800ef10 comp rx=0 tx=0).stop 2024-09-05T12:29:21.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 shutdown_connections 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe30109ba0 0x7fbe3019ed50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe301044a0 0x7fbe3019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe140a2d70 0x7fbe140a5230 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 --2- 172.21.15.71:0/662698798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe30103b00 0x7fbe3019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 >> 172.21.15.71:0/662698798 conn(0x7fbe300ff7c0 msgr2=0x7fbe301000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 shutdown_connections 2024-09-05T12:29:21.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:21.408+0000 7fbe2f200640 1 -- 172.21.15.71:0/662698798 wait complete. 2024-09-05T12:29:22.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 53530 17.08 MB/sec warmup 18 sec latency 139.624 ms 2024-09-05T12:29:22.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:22 smithi071 ceph-mon[29845]: pgmap v51: 129 pgs: 129 active+clean; 25 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 364 op/s 2024-09-05T12:29:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:22 smithi123 ceph-mon[34321]: pgmap v51: 129 pgs: 129 active+clean; 25 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 364 op/s 2024-09-05T12:29:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:22 smithi174 ceph-mon[34313]: pgmap v51: 129 pgs: 129 active+clean; 25 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 364 op/s 2024-09-05T12:29:23.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 56779 17.12 MB/sec warmup 19 sec latency 16.604 ms 2024-09-05T12:29:24.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 59381 16.75 MB/sec warmup 20 sec latency 19.608 ms 2024-09-05T12:29:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:24 smithi071 ceph-mon[29845]: pgmap v52: 129 pgs: 129 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-09-05T12:29:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:24 smithi123 ceph-mon[34321]: pgmap v52: 129 pgs: 129 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-09-05T12:29:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:24 smithi174 ceph-mon[34313]: pgmap v52: 129 pgs: 129 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-09-05T12:29:24.922 INFO:tasks.quiescer.fs.[cephfs]:Total duration (35.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:29:24.923 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-05T12:29:24.923 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:29:25.189 INFO:tasks.workunit.client.0.smithi071.stdout: 1 62250 16.74 MB/sec warmup 21 sec latency 11.497 ms 2024-09-05T12:29:25.330 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 -- 172.21.15.71:0/1562560221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2041047e0 msgr2=0x7fd204108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 --2- 172.21.15.71:0/1562560221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2041047e0 0x7fd204108830 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009900 tx=0x7fd1f002a410 comp rx=0 tx=0).stop 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 -- 172.21.15.71:0/1562560221 shutdown_connections 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 --2- 172.21.15.71:0/1562560221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204108f60 0x7fd204110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 --2- 172.21.15.71:0/1562560221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2041047e0 0x7fd204108830 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 --2- 172.21.15.71:0/1562560221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204103e30 0x7fd204104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 -- 172.21.15.71:0/1562560221 >> 172.21.15.71:0/1562560221 conn(0x7fd2040ff910 msgr2=0x7fd204101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 -- 172.21.15.71:0/1562560221 shutdown_connections 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.957+0000 7fd20b000640 1 -- 172.21.15.71:0/1562560221 wait complete. 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.958+0000 7fd20b000640 1 Processor -- start 2024-09-05T12:29:25.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.958+0000 7fd20b000640 1 -- start start 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.958+0000 7fd20b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 0x7fd20419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041047e0 0x7fd20419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20b000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204108f60 0x7fd20419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd204114010 con 0x7fd2041047e0 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd204113e90 con 0x7fd204103e30 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20b000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd204114190 con 0x7fd204108f60 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 0x7fd20419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 0x7fd20419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44986/0 (socket says 172.21.15.71:44986) 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041047e0 0x7fd20419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20a600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204108f60 0x7fd20419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 -- 172.21.15.71:0/2224181428 learned_addr learned my addr 172.21.15.71:0/2224181428 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 -- 172.21.15.71:0/2224181428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204108f60 msgr2=0x7fd20419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204108f60 0x7fd20419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:25.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 -- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041047e0 msgr2=0x7fd20419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041047e0 0x7fd20419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd209c00640 1 -- 172.21.15.71:0/2224181428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20419f400 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.959+0000 7fd20a600640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204108f60 0x7fd20419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.960+0000 7fd209c00640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 0x7fd20419a450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800d950 tx=0x7fd1f800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.960+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8017070 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.960+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f8009c30 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.960+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f800f6b0 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.960+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20419f6f0 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.960+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a6fd0 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.962+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204105310 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.963+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd1f80109d0 con 0x7fd204103e30 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.965+0000 7fd202000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1ec0a2d70 0x7fd1ec0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:25.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.965+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fd1f80cf2d0 con 0x7fd204103e30 2024-09-05T12:29:25.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.965+0000 7fd209200640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1ec0a2d70 0x7fd1ec0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:25.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.966+0000 7fd209200640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1ec0a2d70 0x7fd1ec0a5230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0000c00 tx=0x7fd1f0005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:25.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:25.969+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1f8009de0 con 0x7fd204103e30 2024-09-05T12:29:26.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 65076 16.69 MB/sec warmup 22 sec latency 12.623 ms 2024-09-05T12:29:26.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.223+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd20410c200 con 0x7fd204103e30 2024-09-05T12:29:26.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.224+0000 7fd202000640 1 -- 172.21.15.71:0/2224181428 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11968 (secure 0 0 0) 0x7fd1f809b500 con 0x7fd204103e30 2024-09-05T12:29:26.226 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:29:26.226 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":30,"btime":"2024-09-05T12:27:44:191847+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:27:44.168192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:29:26.227 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 30 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1ec0a2d70 msgr2=0x7fd1ec0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1ec0a2d70 0x7fd1ec0a5230 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0000c00 tx=0x7fd1f0005b50 comp rx=0 tx=0).stop 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 msgr2=0x7fd20419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 0x7fd20419a450 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1f800d950 tx=0x7fd1f800de10 comp rx=0 tx=0).stop 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 shutdown_connections 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204108f60 0x7fd20419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1ec0a2d70 0x7fd1ec0a5230 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:26.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041047e0 0x7fd20419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:26.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 --2- 172.21.15.71:0/2224181428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204103e30 0x7fd20419a450 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:26.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 >> 172.21.15.71:0/2224181428 conn(0x7fd2040ff910 msgr2=0x7fd204100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:26.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 shutdown_connections 2024-09-05T12:29:26.230 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:26.227+0000 7fd20b000640 1 -- 172.21.15.71:0/2224181428 wait complete. 2024-09-05T12:29:26.630 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:26 smithi071 ceph-mon[29845]: pgmap v53: 129 pgs: 129 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-09-05T12:29:26.630 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:26 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2224181428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:26 smithi123 ceph-mon[34321]: pgmap v53: 129 pgs: 129 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-09-05T12:29:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:26 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2224181428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:26 smithi174 ceph-mon[34313]: pgmap v53: 129 pgs: 129 active+clean; 25 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 791 KiB/s rd, 1.5 MiB/s wr, 194 op/s 2024-09-05T12:29:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:26 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2224181428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:27.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 67800 16.71 MB/sec warmup 23 sec latency 108.112 ms 2024-09-05T12:29:27.892 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-05T12:29:27.892 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-05T12:29:28.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 70755 16.61 MB/sec warmup 24 sec latency 163.160 ms 2024-09-05T12:29:28.326 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:29:28.471 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:28 smithi071 ceph-mon[29845]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 463 op/s 2024-09-05T12:29:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:28 smithi123 ceph-mon[34321]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 463 op/s 2024-09-05T12:29:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:28 smithi174 ceph-mon[34313]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 463 op/s 2024-09-05T12:29:29.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 -- 172.21.15.71:0/778536030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c41049c0 msgr2=0x7f07c4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 --2- 172.21.15.71:0/778536030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c41049c0 0x7f07c4108a10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07b0009860 tx=0x7f07b002a340 comp rx=0 tx=0).stop 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 -- 172.21.15.71:0/778536030 shutdown_connections 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 --2- 172.21.15.71:0/778536030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4109140 0x7f07c4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 --2- 172.21.15.71:0/778536030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c41049c0 0x7f07c4108a10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 --2- 172.21.15.71:0/778536030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c4104010 0x7f07c41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 -- 172.21.15.71:0/778536030 >> 172.21.15.71:0/778536030 conn(0x7f07c40ffa50 msgr2=0x7f07c4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 -- 172.21.15.71:0/778536030 shutdown_connections 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.080+0000 7f07c8e00640 1 -- 172.21.15.71:0/778536030 wait complete. 2024-09-05T12:29:29.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.082+0000 7f07c8e00640 1 Processor -- start 2024-09-05T12:29:29.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.082+0000 7f07c8e00640 1 -- start start 2024-09-05T12:29:29.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.082+0000 7f07c8e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4104010 0x7f07c419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.082+0000 7f07c8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 0x7f07c419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.082+0000 7f07c8e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c4109140 0x7f07c419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.082+0000 7f07c8e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4114140 con 0x7f07c4104010 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07c4113fc0 con 0x7f07c41049c0 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c8e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41142c0 con 0x7f07c4109140 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 0x7f07c419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 0x7f07c419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45000/0 (socket says 172.21.15.71:45000) 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 -- 172.21.15.71:0/1009739041 learned_addr learned my addr 172.21.15.71:0/1009739041 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4104010 0x7f07c419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c2800640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c4109140 0x7f07c419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 -- 172.21.15.71:0/1009739041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c4109140 msgr2=0x7f07c419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c4109140 0x7f07c419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 -- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4104010 msgr2=0x7f07c419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4104010 0x7f07c419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.083+0000 7f07c1400640 1 -- 172.21.15.71:0/1009739041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c419f660 con 0x7f07c41049c0 2024-09-05T12:29:29.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.084+0000 7f07c1400640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 0x7f07c419abf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07b0035040 tx=0x7f07b0033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:29.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.084+0000 7f07baa00640 1 -- 172.21.15.71:0/1009739041 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b003f020 con 0x7f07c41049c0 2024-09-05T12:29:29.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.084+0000 7f07baa00640 1 -- 172.21.15.71:0/1009739041 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b0032770 con 0x7f07c41049c0 2024-09-05T12:29:29.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.084+0000 7f07baa00640 1 -- 172.21.15.71:0/1009739041 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b0037040 con 0x7f07c41049c0 2024-09-05T12:29:29.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.084+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c419f8f0 con 0x7f07c41049c0 2024-09-05T12:29:29.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.085+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c419fd80 con 0x7f07c41049c0 2024-09-05T12:29:29.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.088+0000 7f07cb7d3740 1 Processor -- start 2024-09-05T12:29:29.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.088+0000 7f07cb7d3740 1 -- start start 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.088+0000 7f07baa00640 1 -- 172.21.15.71:0/1009739041 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f07b0042020 con 0x7f07c41049c0 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99bddb0 0x5597a99c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5597a9910530 con 0x5597a99bddb0 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597a98b7860 con 0x5597a99aa2c0 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597a990c4f0 con 0x5597a99aabe0 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45004/0 (socket says 172.21.15.71:45004) 2024-09-05T12:29:29.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c2800640 1 -- 172.21.15.71:0/1671399173 learned_addr learned my addr 172.21.15.71:0/1671399173 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c1400640 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99bddb0 0x5597a99c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c2800640 1 -- 172.21.15.71:0/1671399173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 msgr2=0x5597a99bd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c2800640 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.089+0000 7f07c2800640 1 -- 172.21.15.71:0/1671399173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99bddb0 msgr2=0x5597a99c01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.090+0000 7f07c2800640 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99bddb0 0x5597a99c01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.090+0000 7f07c2800640 1 -- 172.21.15.71:0/1671399173 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a9681b90 con 0x5597a99aa2c0 2024-09-05T12:29:29.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.090+0000 7f07c1400640 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99bddb0 0x5597a99c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:29:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.090+0000 7f07baa00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f07a80a2d70 0x7f07a80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.090+0000 7f07c2800640 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99aa6a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009f40 tx=0x7f07bc010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbdb43a7c103f892 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:29.095 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.090+0000 7f07baa00640 1 -- 172.21.15.71:0/1009739041 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f07b00ef8e0 con 0x7f07c41049c0 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.091+0000 7f07af400640 1 -- 172.21.15.71:0/1671399173 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc003b70 con 0x5597a99aa2c0 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.091+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f07a80a2d70 0x7f07a80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.091+0000 7f07af400640 1 -- 172.21.15.71:0/1671399173 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc003d10 con 0x5597a99aa2c0 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.091+0000 7f07af400640 1 -- 172.21.15.71:0/1671399173 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc011a40 con 0x5597a99aa2c0 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1671399173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 msgr2=0x5597a99aa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99aa6a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009f40 tx=0x7f07bc010860 comp rx=0 tx=0).stop 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f07a80a2d70 0x7f07a80a5230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07b4005fd0 tx=0x7f07b4009260 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1671399173 shutdown_connections 2024-09-05T12:29:29.096 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99bddb0 0x5597a99c01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1671399173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99aa6a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1671399173 >> 172.21.15.71:0/1671399173 conn(0x5597a99a6950 msgr2=0x5597a99a6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.092+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1671399173 shutdown_connections 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.093+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1671399173 wait complete. 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.093+0000 7f07cb7d3740 1 -- start start 2024-09-05T12:29:29.097 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.093+0000 7f07cb7d3740 1 Processor -- start 2024-09-05T12:29:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aa2c0 0x5597a99bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99bddb0 0x5597a99bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5597a99c0980 con 0x5597a99aa2c0 2024-09-05T12:29:29.101 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597a98cae60 con 0x5597a99bddb0 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597a99c0b00 con 0x5597a99aabe0 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07cb7d3740 10 client.0 _ll_register_callbacks cb 0x5597a98cfcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07c1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07c1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33674/0 (socket says 172.21.15.71:33674) 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.094+0000 7f07c1400640 1 -- 172.21.15.71:0/1751468069 learned_addr learned my addr 172.21.15.71:0/1751468069 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1400640 1 -- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99bddb0 msgr2=0x5597a99bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aa2c0 0x5597a99bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c2800640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99bddb0 0x5597a99bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1400640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99bddb0 0x5597a99bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1400640 1 -- 172.21.15.71:0/1751468069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aa2c0 msgr2=0x5597a99bacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1400640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aa2c0 0x5597a99bacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.102 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1400640 1 -- 172.21.15.71:0/1751468069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a98d6990 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aa2c0 0x5597a99bacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c2800640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99bddb0 0x5597a99bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07c1400640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bb1f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07b00b7fd0 tx=0x7f07b00035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9112af697f2c1159 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07a3e00640 1 -- 172.21.15.71:0/1751468069 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b00ed110 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07a3e00640 1 -- 172.21.15.71:0/1751468069 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f07b00bbdd0 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.095+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07bc003d10 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07a3e00640 1 -- 172.21.15.71:0/1751468069 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07b00bbb70 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 10 client.25621 fetch_fsmap learned FSMap version 30 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597a99c0980 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07a3e00640 1 -- 172.21.15.71:0/1751468069 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07bc003d10 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07a3e00640 1 -- 172.21.15.71:0/1751468069 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f07b003a030 con 0x5597a99aabe0 2024-09-05T12:29:29.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 10 client.25621 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 10 client.25621 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 20 client.25621 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a99cda30 0x5597a99cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 4 client.25621 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.096+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a98b3310 con 0x5597a99cda30 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.097+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a99cda30 0x5597a99cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.097+0000 7f07c1e00640 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a99cda30 0x5597a99cfeb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac465dedce84539 server_cookie=3abcdff6a134e1f3 in_seq=0 out_seq=0 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.097+0000 7f07a3e00640 10 client.25621 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.098+0000 7f07a3e00640 1 -- 172.21.15.71:0/1751468069 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597a98b3310 con 0x5597a99cda30 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.098+0000 7f07a3e00640 10 client.25621 handle_command_reply: tid=0 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.098+0000 7f07cb7d3740 1 client.25621 shutdown 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.098+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 msgr2=0x5597a99bb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.098+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bb1f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f07b00b7fd0 tx=0x7f07b00035f0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 shutdown_connections 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a99cda30 0x5597a99cfeb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99bddb0 0x5597a99bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99aabe0 0x5597a99bb1f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/1751468069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aa2c0 0x5597a99bacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 >> 172.21.15.71:0/1751468069 conn(0x5597a99a6950 msgr2=0x5597a99ac6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07a3e00640 0 client.25621 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 shutdown_connections 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 1 -- 172.21.15.71:0/1751468069 wait complete. 2024-09-05T12:29:29.105 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.099+0000 7f07cb7d3740 20 client.25621 trim_cache size 0 max 16384 2024-09-05T12:29:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 Processor -- start 2024-09-05T12:29:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 -- start start 2024-09-05T12:29:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99bcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.122 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a99d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41acc60 con 0x5597a99aabe0 2024-09-05T12:29:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41ae7a0 con 0x5597a99aa2c0 2024-09-05T12:29:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.120+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41ad330 con 0x5597a99bddb0 2024-09-05T12:29:29.123 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99bcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99bcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45026/0 (socket says 172.21.15.71:45026) 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 -- 172.21.15.71:0/2747488754 learned_addr learned my addr 172.21.15.71:0/2747488754 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c2800640 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a99d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 -- 172.21.15.71:0/2747488754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 msgr2=0x5597a99d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a99d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1400640 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 -- 172.21.15.71:0/2747488754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 msgr2=0x5597a99d57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99d57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.125 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1e00640 1 -- 172.21.15.71:0/2747488754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a99aaa10 con 0x5597a99aa2c0 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.121+0000 7f07c1400640 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.122+0000 7f07c1e00640 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99bcb10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07b4012cb0 tx=0x7f07b401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42a48362f2caaced server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.122+0000 7f07aea00640 1 -- 172.21.15.71:0/2747488754 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4003b70 con 0x5597a99aa2c0 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.122+0000 7f07aea00640 1 -- 172.21.15.71:0/2747488754 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b402e070 con 0x5597a99aa2c0 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.122+0000 7f07aea00640 1 -- 172.21.15.71:0/2747488754 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b401f680 con 0x5597a99aa2c0 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 -- 172.21.15.71:0/2747488754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 msgr2=0x5597a99bcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99bcb10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f07b4012cb0 tx=0x7f07b401a000 comp rx=0 tx=0).stop 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 -- 172.21.15.71:0/2747488754 shutdown_connections 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a99d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99d57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/2747488754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99bcb10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 -- 172.21.15.71:0/2747488754 >> 172.21.15.71:0/2747488754 conn(0x5597a99a6950 msgr2=0x5597a99bc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 -- 172.21.15.71:0/2747488754 shutdown_connections 2024-09-05T12:29:29.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 -- 172.21.15.71:0/2747488754 wait complete. 2024-09-05T12:29:29.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.123+0000 7f07cb7d3740 1 -- start start 2024-09-05T12:29:29.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 Processor -- start 2024-09-05T12:29:29.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99c1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99c2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a9a37790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41ae0d0 con 0x5597a99aabe0 2024-09-05T12:29:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41ada00 con 0x5597a99aa2c0 2024-09-05T12:29:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07c41aac40 con 0x5597a99bddb0 2024-09-05T12:29:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.124+0000 7f07cb7d3740 10 client.0 _ll_register_callbacks cb 0x7f07b0005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:29.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99c1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99c1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45034/0 (socket says 172.21.15.71:45034) 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 -- 172.21.15.71:0/3992860490 learned_addr learned my addr 172.21.15.71:0/3992860490 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1400640 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99c2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 -- 172.21.15.71:0/3992860490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 msgr2=0x5597a9a37790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a9a37790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 -- 172.21.15.71:0/3992860490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 msgr2=0x5597a99c2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99c2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 -- 172.21.15.71:0/3992860490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a9a37cd0 con 0x5597a99aa2c0 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07c1e00640 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99c1ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f07b40289b0 tx=0x7f07b40289e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95cc9da3020460d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.125+0000 7f07a3e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07a3e00640 1 -- 172.21.15.71:0/3992860490 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4028d40 con 0x5597a99aa2c0 2024-09-05T12:29:29.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07a3e00640 1 -- 172.21.15.71:0/3992860490 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f07b4003b70 con 0x5597a99aa2c0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b402e070 con 0x5597a99aa2c0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07a3e00640 1 -- 172.21.15.71:0/3992860490 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f07b4004150 con 0x5597a99aa2c0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07cb7d3740 10 client.25616 fetch_fsmap learned FSMap version 30 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5597a9a38db0 con 0x5597a99aa2c0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07a3e00640 1 -- 172.21.15.71:0/3992860490 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07b402e070 con 0x5597a99aa2c0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.126+0000 7f07a3e00640 1 -- 172.21.15.71:0/3992860490 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f07b401f990 con 0x5597a99aa2c0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07cb7d3740 10 client.25616 fetch_fsmap finished waiting for FSMap version 30 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07cb7d3740 10 client.25616 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07cb7d3740 20 client.25616 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a9a39490 0x5597a9a3b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07cb7d3740 4 client.25616 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x5597a99a2180 con 0x5597a9a39490 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07c1400640 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a9a39490 0x5597a9a3b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07c1400640 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a9a39490 0x5597a9a3b880 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3d1f91e6d49cdc server_cookie=773e6a8868906784 in_seq=0 out_seq=0 2024-09-05T12:29:29.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.127+0000 7f07a3e00640 10 client.25616 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:29.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72604 16.34 MB/sec warmup 25 sec latency 51.620 ms 2024-09-05T12:29:29.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07a3e00640 1 -- 172.21.15.71:0/3992860490 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5597a99a2180 con 0x5597a9a39490 2024-09-05T12:29:29.236 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07a3e00640 10 client.25616 handle_command_reply: tid=0 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 client.25616 shutdown 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 msgr2=0x5597a99c1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99c1ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f07b40289b0 tx=0x7f07b40289e0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 shutdown_connections 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5597a9a39490 0x5597a9a3b880 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597a99bddb0 0x5597a9a37790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597a99aabe0 0x5597a99c2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07a3e00640 0 client.25616 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 --2- 172.21.15.71:0/3992860490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597a99aa2c0 0x5597a99c1ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 >> 172.21.15.71:0/3992860490 conn(0x5597a99a6950 msgr2=0x5597a99bb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 shutdown_connections 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 1 -- 172.21.15.71:0/3992860490 wait complete. 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.234+0000 7f07cb7d3740 20 client.25616 trim_cache size 0 max 16384 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 30, 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 6, 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout: "37865911": { 2024-09-05T12:29:29.238 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 6, 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 137.7, 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:29:29.239 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:29.240 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f07a80a2d70 msgr2=0x7f07a80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f07a80a2d70 0x7f07a80a5230 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f07b4005fd0 tx=0x7f07b4009260 comp rx=0 tx=0).stop 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 msgr2=0x7f07c419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 0x7f07c419abf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f07b0035040 tx=0x7f07b0033d90 comp rx=0 tx=0).stop 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 shutdown_connections 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c4109140 0x7f07c419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c41049c0 0x7f07c419abf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f07a80a2d70 0x7f07a80a5230 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 --2- 172.21.15.71:0/1009739041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4104010 0x7f07c419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:29.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 >> 172.21.15.71:0/1009739041 conn(0x7f07c40ffa50 msgr2=0x7f07c4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:29.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 shutdown_connections 2024-09-05T12:29:29.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:29.235+0000 7f07c8e00640 1 -- 172.21.15.71:0/1009739041 wait complete. 2024-09-05T12:29:30.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 15.72 MB/sec warmup 26 sec latency 963.081 ms 2024-09-05T12:29:30.368 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '37865911', quiesce took 5.4 seconds. Will release after: 12.299999999999999 2024-09-05T12:29:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:30 smithi071 ceph-mon[29845]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-09-05T12:29:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:30 smithi123 ceph-mon[34321]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-09-05T12:29:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:30 smithi174 ceph-mon[34313]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 758 KiB/s rd, 1.4 MiB/s wr, 359 op/s 2024-09-05T12:29:31.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 15.13 MB/sec warmup 27 sec latency 1963.194 ms 2024-09-05T12:29:32.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 14.59 MB/sec warmup 28 sec latency 2963.267 ms 2024-09-05T12:29:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:32 smithi123 ceph-mon[34321]: pgmap v56: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-09-05T12:29:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:32 smithi174 ceph-mon[34313]: pgmap v56: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-09-05T12:29:32.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:32 smithi071 ceph-mon[29845]: pgmap v56: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 2.3 MiB/s wr, 427 op/s 2024-09-05T12:29:33.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 14.09 MB/sec warmup 29 sec latency 3963.389 ms 2024-09-05T12:29:33.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:33 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:29:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:33 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:29:33.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:33 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:29:34.190 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 13.62 MB/sec warmup 30 sec latency 4963.528 ms 2024-09-05T12:29:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 -- 172.21.15.71:0/2657362156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc1044a0 msgr2=0x7f00bc1109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 --2- 172.21.15.71:0/2657362156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc1044a0 0x7f00bc1109d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009cd0 tx=0x7f00b402a8e0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 -- 172.21.15.71:0/2657362156 shutdown_connections 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 --2- 172.21.15.71:0/2657362156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc1044a0 0x7f00bc1109d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 --2- 172.21.15.71:0/2657362156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00bc103b00 0x7f00bc103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 --2- 172.21.15.71:0/2657362156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc109ba0 0x7f00bc109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 -- 172.21.15.71:0/2657362156 >> 172.21.15.71:0/2657362156 conn(0x7f00bc0ff7c0 msgr2=0x7f00bc101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 -- 172.21.15.71:0/2657362156 shutdown_connections 2024-09-05T12:29:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.533+0000 7f00c1200640 1 -- 172.21.15.71:0/2657362156 wait complete. 2024-09-05T12:29:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.534+0000 7f00c1200640 1 Processor -- start 2024-09-05T12:29:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.534+0000 7f00c1200640 1 -- start start 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00c1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 0x7f00bc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00c1200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc1044a0 0x7f00bc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00c1200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00bc109ba0 0x7f00bc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00c1200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc113f50 con 0x7f00bc103b00 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00c1200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc113dd0 con 0x7f00bc109ba0 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00c1200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1140d0 con 0x7f00bc1044a0 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 0x7f00bc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 0x7f00bc19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49768/0 (socket says 172.21.15.71:49768) 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00bb400640 1 -- 172.21.15.71:0/3831177181 learned_addr learned my addr 172.21.15.71:0/3831177181 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00bbe00640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00bc109ba0 0x7f00bc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.535+0000 7f00b3e00640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc1044a0 0x7f00bc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00bb400640 1 -- 172.21.15.71:0/3831177181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc1044a0 msgr2=0x7f00bc19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00bb400640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc1044a0 0x7f00bc19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00bb400640 1 -- 172.21.15.71:0/3831177181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00bc109ba0 msgr2=0x7f00bc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00bb400640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00bc109ba0 0x7f00bc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00bb400640 1 -- 172.21.15.71:0/3831177181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc19f490 con 0x7f00bc103b00 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00b3e00640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc1044a0 0x7f00bc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00bb400640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 0x7f00bc19a4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e9f0 tx=0x7f00ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00b8c00640 1 -- 172.21.15.71:0/3831177181 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00cd80 con 0x7f00bc103b00 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00b8c00640 1 -- 172.21.15.71:0/3831177181 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac005500 con 0x7f00bc103b00 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc19f780 con 0x7f00bc103b00 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00b8c00640 1 -- 172.21.15.71:0/3831177181 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac010690 con 0x7f00bc103b00 2024-09-05T12:29:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.536+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a7010 con 0x7f00bc103b00 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00b8c00640 1 -- 172.21.15.71:0/3831177181 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f00ac0046c0 con 0x7f00bc103b00 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 Processor -- start 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 -- start start 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d18790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d23030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d25960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562728c7e100 con 0x562728d18cd0 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562728c261b0 con 0x562728d23570 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.540+0000 7f00c5846740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562728c7a0c0 con 0x562728d183b0 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d25960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d25960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45866/0 (socket says 172.21.15.71:45866) 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 -- 172.21.15.71:0/1397030215 learned_addr learned my addr 172.21.15.71:0/1397030215 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00bb400640 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d23030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00bbe00640 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d18790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 -- 172.21.15.71:0/1397030215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 msgr2=0x562728d18790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d18790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 -- 172.21.15.71:0/1397030215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 msgr2=0x562728d23030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d23030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 -- 172.21.15.71:0/1397030215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562728991210 con 0x562728d23570 2024-09-05T12:29:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b8c00640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f00a00a2d70 0x7f00a00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b3e00640 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d25960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00a400bcb0 tx=0x7f00a400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb9be87e4337b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00b8c00640 1 -- 172.21.15.71:0/3831177181 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f00ac0cfb90 con 0x7f00bc103b00 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.541+0000 7f00bbe00640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f00a00a2d70 0x7f00a00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.542+0000 7f00b0c00640 1 -- 172.21.15.71:0/1397030215 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4011440 con 0x562728d23570 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.542+0000 7f00b0c00640 1 -- 172.21.15.71:0/1397030215 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a4011ac0 con 0x562728d23570 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.542+0000 7f00b0c00640 1 -- 172.21.15.71:0/1397030215 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a401bb50 con 0x562728d23570 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.542+0000 7f00c5846740 1 -- 172.21.15.71:0/1397030215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 msgr2=0x562728d25960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.542+0000 7f00bbe00640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f00a00a2d70 0x7f00a00a5230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00b4004ab0 tx=0x7f00b4004a40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.542+0000 7f00c5846740 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d25960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f00a400bcb0 tx=0x7f00a400f730 comp rx=0 tx=0).stop 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 -- 172.21.15.71:0/1397030215 shutdown_connections 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d25960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d23030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 --2- 172.21.15.71:0/1397030215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d18790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 -- 172.21.15.71:0/1397030215 >> 172.21.15.71:0/1397030215 conn(0x562728d14920 msgr2=0x562728d14d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 -- 172.21.15.71:0/1397030215 shutdown_connections 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 -- 172.21.15.71:0/1397030215 wait complete. 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 -- start start 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.543+0000 7f00c5846740 1 Processor -- start 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 0x562728d205f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 0x562728d20b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d23570 0x562728d21070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562728d25ff0 con 0x562728d183b0 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562728c07090 con 0x562728d18cd0 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562728d26170 con 0x562728d23570 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00c5846740 10 client.0 _ll_register_callbacks cb 0x562728c5bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 0x562728d205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 0x562728d20b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00bb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 0x562728d205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49782/0 (socket says 172.21.15.71:49782) 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00bb400640 1 -- 172.21.15.71:0/3075270227 learned_addr learned my addr 172.21.15.71:0/3075270227 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 0x562728d20b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45882/0 (socket says 172.21.15.71:45882) 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00bbe00640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d23570 0x562728d21070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00bb400640 1 -- 172.21.15.71:0/3075270227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d23570 msgr2=0x562728d21070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.544+0000 7f00bb400640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d23570 0x562728d21070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00bb400640 1 -- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 msgr2=0x562728d20b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00bb400640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 0x562728d20b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00bb400640 1 -- 172.21.15.71:0/3075270227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562728a5d620 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00b3e00640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 0x562728d20b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00bb400640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 0x562728d205f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0a4450 tx=0x7f00ac004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d4efabd235c5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00a9600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00a9600640 1 -- 172.21.15.71:0/3075270227 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0d4040 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00a9600640 1 -- 172.21.15.71:0/3075270227 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f00ac0a46f0 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a4011ac0 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00a9600640 1 -- 172.21.15.71:0/3075270227 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00ac09b670 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00c5846740 10 client.15579 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.545+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562728d31f00 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00a9600640 1 -- 172.21.15.71:0/3075270227 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a4011ac0 con 0x562728d183b0 2024-09-05T12:29:34.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00a9600640 1 -- 172.21.15.71:0/3075270227 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f00ac09b160 con 0x562728d183b0 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00c5846740 10 client.15579 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00c5846740 10 client.15579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00c5846740 20 client.15579 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00c5846740 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d33430 0x562728d35820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00c5846740 4 client.15579 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562728c8b830 con 0x562728d33430 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.546+0000 7f00b3e00640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d33430 0x562728d35820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.547+0000 7f00b3e00640 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d33430 0x562728d35820 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d632e005f35b3ff0 server_cookie=3bfdf107c9bf8526 in_seq=0 out_seq=0 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.547+0000 7f00a9600640 10 client.15579 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00a9600640 1 -- 172.21.15.71:0/3075270227 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562728c8b830 con 0x562728d33430 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00a9600640 10 client.15579 handle_command_reply: tid=0 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 client.15579 shutdown 2024-09-05T12:29:34.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 msgr2=0x562728d205f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 0x562728d205f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0a4450 tx=0x7f00ac004ae0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 shutdown_connections 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d33430 0x562728d35820 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d23570 0x562728d21070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d18cd0 0x562728d20b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00a9600640 0 client.15579 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 --2- 172.21.15.71:0/3075270227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d183b0 0x562728d205f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 >> 172.21.15.71:0/3075270227 conn(0x562728d14920 msgr2=0x562728d1a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 shutdown_connections 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 1 -- 172.21.15.71:0/3075270227 wait complete. 2024-09-05T12:29:34.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.548+0000 7f00c5846740 20 client.15579 trim_cache size 0 max 16384 2024-09-05T12:29:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 Processor -- start 2024-09-05T12:29:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 -- start start 2024-09-05T12:29:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d20cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d3b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d3b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b1010 con 0x562728d18cd0 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.563+0000 7f00c5846740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b2bb0 con 0x562728d23570 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00c5846740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b16e0 con 0x562728d183b0 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d3b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d3b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49798/0 (socket says 172.21.15.71:49798) 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 -- 172.21.15.71:0/1310842785 learned_addr learned my addr 172.21.15.71:0/1310842785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00bbe00640 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00bb400640 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d20cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 -- 172.21.15.71:0/1310842785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 msgr2=0x562728d20cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d20cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 -- 172.21.15.71:0/1310842785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 msgr2=0x562728d3b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d3b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 -- 172.21.15.71:0/1310842785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562728d155b0 con 0x562728d18cd0 2024-09-05T12:29:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.564+0000 7f00b3e00640 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d3b0c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f00a400ba40 tx=0x7f00a400b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db100160837a53e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00abe00640 1 -- 172.21.15.71:0/1310842785 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4005240 con 0x562728d18cd0 2024-09-05T12:29:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00abe00640 1 -- 172.21.15.71:0/1310842785 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a401b480 con 0x562728d18cd0 2024-09-05T12:29:34.568 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00abe00640 1 -- 172.21.15.71:0/1310842785 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4003850 con 0x562728d18cd0 2024-09-05T12:29:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 -- 172.21.15.71:0/1310842785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 msgr2=0x562728d3b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d3b0c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f00a400ba40 tx=0x7f00a400b5a0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 -- 172.21.15.71:0/1310842785 shutdown_connections 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d23570 0x562728d3b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d18cd0 0x562728d3b0c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 --2- 172.21.15.71:0/1310842785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d183b0 0x562728d20cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 -- 172.21.15.71:0/1310842785 >> 172.21.15.71:0/1310842785 conn(0x562728d14920 msgr2=0x562728d205b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.565+0000 7f00c5846740 1 -- 172.21.15.71:0/1310842785 shutdown_connections 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 -- 172.21.15.71:0/1310842785 wait complete. 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 -- start start 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 Processor -- start 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 0x562728d2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d18cd0 0x562728d26df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d23570 0x562728d27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b2480 con 0x562728d23570 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1b1db0 con 0x562728d183b0 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1aeff0 con 0x562728d18cd0 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.566+0000 7f00c5846740 10 client.0 _ll_register_callbacks cb 0x7f00ac0d3330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d23570 0x562728d27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 0x562728d2ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 0x562728d2ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45908/0 (socket says 172.21.15.71:45908) 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 -- 172.21.15.71:0/180582932 learned_addr learned my addr 172.21.15.71:0/180582932 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00b3e00640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d18cd0 0x562728d26df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 -- 172.21.15.71:0/180582932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d18cd0 msgr2=0x562728d26df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d18cd0 0x562728d26df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 -- 172.21.15.71:0/180582932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d23570 msgr2=0x562728d27330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:34.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d23570 0x562728d27330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 -- 172.21.15.71:0/180582932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562728d2f390 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bbe00640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d23570 0x562728d27330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00bb400640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 0x562728d2ba80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e9c0 tx=0x7f00ac09abd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e80c41b759385dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.567+0000 7f00a9600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00a9600640 1 -- 172.21.15.71:0/180582932 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac0cd3d0 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a401b480 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00a9600640 1 -- 172.21.15.71:0/180582932 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f00ac0a4dd0 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00a9600640 1 -- 172.21.15.71:0/180582932 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f00ac0aa050 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 10 client.25634 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562728d9dd00 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00a9600640 1 -- 172.21.15.71:0/180582932 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a401b480 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00a9600640 1 -- 172.21.15.71:0/180582932 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f00ac0a1250 con 0x562728d183b0 2024-09-05T12:29:34.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 10 client.25634 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 10 client.25634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 20 client.25634 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d9e480 0x562728da08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 4 client.25634 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.568+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562728c85b80 con 0x562728d9e480 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.569+0000 7f00b3e00640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d9e480 0x562728da08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.569+0000 7f00b3e00640 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d9e480 0x562728da08a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969ef493e11cd4d3 server_cookie=52f6e776d3d22919 in_seq=0 out_seq=0 2024-09-05T12:29:34.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.569+0000 7f00a9600640 10 client.25634 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:34 smithi123 ceph-mon[34321]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 337 op/s 2024-09-05T12:29:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00a9600640 1 -- 172.21.15.71:0/180582932 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562728c85b80 con 0x562728d9e480 2024-09-05T12:29:34.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00a9600640 10 client.25634 handle_command_reply: tid=0 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 client.25634 shutdown 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 msgr2=0x562728d2ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 0x562728d2ba80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e9c0 tx=0x7f00ac09abd0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 shutdown_connections 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562728d9e480 0x562728da08a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562728d23570 0x562728d27330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562728d18cd0 0x562728d26df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 --2- 172.21.15.71:0/180582932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562728d183b0 0x562728d2ba80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 >> 172.21.15.71:0/180582932 conn(0x562728d14920 msgr2=0x562728d2eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.707+0000 7f00a9600640 0 client.25634 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:34.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 shutdown_connections 2024-09-05T12:29:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c5846740 1 -- 172.21.15.71:0/180582932 wait complete. 2024-09-05T12:29:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c5846740 20 client.25634 trim_cache size 0 max 16384 2024-09-05T12:29:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:29:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:29:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:29:34.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:29:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f00a00a2d70 msgr2=0x7f00a00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c1200640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f00a00a2d70 0x7f00a00a5230 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00b4004ab0 tx=0x7f00b4004a40 comp rx=0 tx=0).stop 2024-09-05T12:29:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 msgr2=0x7f00bc19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.708+0000 7f00c1200640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 0x7f00bc19a4a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e9f0 tx=0x7f00ac00edf0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 shutdown_connections 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f00bc109ba0 0x7f00bc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00bc1044a0 0x7f00bc19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f00a00a2d70 0x7f00a00a5230 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 --2- 172.21.15.71:0/3831177181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00bc103b00 0x7f00bc19a4a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 >> 172.21.15.71:0/3831177181 conn(0x7f00bc0ff7c0 msgr2=0x7f00bc100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 shutdown_connections 2024-09-05T12:29:34.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:34.709+0000 7f00c1200640 1 -- 172.21.15.71:0/3831177181 wait complete. 2024-09-05T12:29:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:34 smithi174 ceph-mon[34313]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 337 op/s 2024-09-05T12:29:34.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:34 smithi071 ceph-mon[29845]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 337 op/s 2024-09-05T12:29:35.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 13.18 MB/sec warmup 31 sec latency 5963.593 ms 2024-09-05T12:29:36.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 12.77 MB/sec warmup 32 sec latency 6963.668 ms 2024-09-05T12:29:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:36 smithi174 ceph-mon[34313]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 336 op/s 2024-09-05T12:29:36.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:36 smithi071 ceph-mon[29845]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 336 op/s 2024-09-05T12:29:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:36 smithi123 ceph-mon[34321]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.6 MiB/s wr, 336 op/s 2024-09-05T12:29:37.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 12.38 MB/sec warmup 33 sec latency 7963.715 ms 2024-09-05T12:29:38.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 12.02 MB/sec warmup 34 sec latency 8963.838 ms 2024-09-05T12:29:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:38 smithi174 ceph-mon[34313]: pgmap v59: 129 pgs: 129 active+clean; 37 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 441 op/s 2024-09-05T12:29:38.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:38 smithi071 ceph-mon[29845]: pgmap v59: 129 pgs: 129 active+clean; 37 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 441 op/s 2024-09-05T12:29:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:38 smithi123 ceph-mon[34321]: pgmap v59: 129 pgs: 129 active+clean; 37 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 3.1 MiB/s wr, 441 op/s 2024-09-05T12:29:39.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 11.67 MB/sec warmup 35 sec latency 9963.912 ms 2024-09-05T12:29:40.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 11.35 MB/sec warmup 36 sec latency 10963.966 ms 2024-09-05T12:29:40.507 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:40 smithi071 ceph-mon[29845]: pgmap v60: 129 pgs: 129 active+clean; 37 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-09-05T12:29:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:40 smithi174 ceph-mon[34313]: pgmap v60: 129 pgs: 129 active+clean; 37 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-09-05T12:29:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:40 smithi123 ceph-mon[34321]: pgmap v60: 129 pgs: 129 active+clean; 37 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-09-05T12:29:41.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 11.04 MB/sec warmup 37 sec latency 11964.023 ms 2024-09-05T12:29:42.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 10.75 MB/sec warmup 38 sec latency 12964.093 ms 2024-09-05T12:29:42.526 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:42 smithi071 ceph-mon[29845]: pgmap v61: 129 pgs: 129 active+clean; 40 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.8 MiB/s wr, 187 op/s 2024-09-05T12:29:42.669 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:29:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:42 smithi174 ceph-mon[34313]: pgmap v61: 129 pgs: 129 active+clean; 40 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.8 MiB/s wr, 187 op/s 2024-09-05T12:29:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:42 smithi123 ceph-mon[34321]: pgmap v61: 129 pgs: 129 active+clean; 40 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 2.8 MiB/s wr, 187 op/s 2024-09-05T12:29:43.107 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:29:43.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 10.48 MB/sec warmup 39 sec latency 13964.136 ms 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 -- 172.21.15.71:0/1197360276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 msgr2=0x7f3f6410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 --2- 172.21.15.71:0/1197360276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 0x7f3f6410ea90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f3f5c009ca0 tx=0x7f3f5c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 -- 172.21.15.71:0/1197360276 shutdown_connections 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 --2- 172.21.15.71:0/1197360276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 0x7f3f6410ea90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 --2- 172.21.15.71:0/1197360276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64102790 0x7f3f641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 --2- 172.21.15.71:0/1197360276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f64101de0 0x7f3f641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 -- 172.21.15.71:0/1197360276 >> 172.21.15.71:0/1197360276 conn(0x7f3f640fd8c0 msgr2=0x7f3f640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:43.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 -- 172.21.15.71:0/1197360276 shutdown_connections 2024-09-05T12:29:43.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.974+0000 7f3f6a200640 1 -- 172.21.15.71:0/1197360276 wait complete. 2024-09-05T12:29:43.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 Processor -- start 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 -- start start 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 0x7f3f6419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f64102790 0x7f3f6419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 0x7f3f6419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64111fc0 con 0x7f3f64106f10 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64111e40 con 0x7f3f64102790 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.975+0000 7f3f6a200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f64112140 con 0x7f3f64101de0 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 0x7f3f6419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 0x7f3f6419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 0x7f3f6419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52246/0 (socket says 172.21.15.71:52246) 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 -- 172.21.15.71:0/3332979242 learned_addr learned my addr 172.21.15.71:0/3332979242 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:43.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f5aa00640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f64102790 0x7f3f6419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 -- 172.21.15.71:0/3332979242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f64102790 msgr2=0x7f3f6419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f64102790 0x7f3f6419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:43.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 -- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 msgr2=0x7f3f6419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 0x7f3f6419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.976+0000 7f3f63400640 1 -- 172.21.15.71:0/3332979242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6419f590 con 0x7f3f64101de0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.977+0000 7f3f63400640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 0x7f3f6419a680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400ea60 tx=0x7f3f5400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.977+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5400cde0 con 0x7f3f64101de0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.977+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6419f880 con 0x7f3f64101de0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.977+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f540035f0 con 0x7f3f64101de0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.977+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f54010670 con 0x7f3f64101de0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.977+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f641a7160 con 0x7f3f64101de0 2024-09-05T12:29:43.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.978+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f641021c0 con 0x7f3f64101de0 2024-09-05T12:29:43.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.980+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3f540037a0 con 0x7f3f64101de0 2024-09-05T12:29:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.981+0000 7f3f60c00640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3f480a2cf0 0x7f3f480a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.981+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f3f540cfcc0 con 0x7f3f64101de0 2024-09-05T12:29:43.983 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.982+0000 7f3f5aa00640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3f480a2cf0 0x7f3f480a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:43.984 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.982+0000 7f3f5aa00640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3f480a2cf0 0x7f3f480a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f640ff6a0 tx=0x7f3f4c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:43.985 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:43.983+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f540541b0 con 0x7f3f64101de0 2024-09-05T12:29:44.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 10.22 MB/sec warmup 40 sec latency 14964.203 ms 2024-09-05T12:29:44.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.195+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f6419b9e0 con 0x7f3f64101de0 2024-09-05T12:29:44.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.196+0000 7f3f60c00640 1 -- 172.21.15.71:0/3332979242 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11971 (secure 0 0 0) 0x7f3f5409c540 con 0x7f3f64101de0 2024-09-05T12:29:44.198 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:29:44.199 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":31,"btime":"2024-09-05T12:29:32:411358+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:29:31.941232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:29:44.199 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 31 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3f480a2cf0 msgr2=0x7f3f480a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3f480a2cf0 0x7f3f480a51b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3f640ff6a0 tx=0x7f3f4c005f90 comp rx=0 tx=0).stop 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 msgr2=0x7f3f6419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 0x7f3f6419a680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400ea60 tx=0x7f3f5400ef20 comp rx=0 tx=0).stop 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 shutdown_connections 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3f480a2cf0 0x7f3f480a51b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:44.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f64106f10 0x7f3f6419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f64102790 0x7f3f6419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 --2- 172.21.15.71:0/3332979242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f64101de0 0x7f3f6419a680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 >> 172.21.15.71:0/3332979242 conn(0x7f3f640fd8c0 msgr2=0x7f3f640fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 shutdown_connections 2024-09-05T12:29:44.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:44.199+0000 7f3f6a200640 1 -- 172.21.15.71:0/3332979242 wait complete. 2024-09-05T12:29:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:44 smithi174 ceph-mon[34313]: pgmap v62: 129 pgs: 129 active+clean; 46 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-09-05T12:29:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:44 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3332979242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:44.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:44 smithi071 ceph-mon[29845]: pgmap v62: 129 pgs: 129 active+clean; 46 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-09-05T12:29:44.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:44 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3332979242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:44 smithi123 ceph-mon[34321]: pgmap v62: 129 pgs: 129 active+clean; 46 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-09-05T12:29:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:44 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3332979242' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:45.122 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 37865911 --release --await' 2024-09-05T12:29:45.122 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 37865911 --release --await 2024-09-05T12:29:45.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 9.97 MB/sec warmup 41 sec latency 15964.263 ms 2024-09-05T12:29:45.577 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:29:46.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 9.73 MB/sec warmup 42 sec latency 16964.317 ms 2024-09-05T12:29:46.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 -- 172.21.15.71:0/1758939519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb758074f40 msgr2=0x7fb75810dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 --2- 172.21.15.71:0/1758939519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb758074f40 0x7fb75810dcc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb744009860 tx=0x7fb74402a3d0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 -- 172.21.15.71:0/1758939519 shutdown_connections 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 --2- 172.21.15.71:0/1758939519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb75810e200 0x7fb758110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 --2- 172.21.15.71:0/1758939519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb758074f40 0x7fb75810dcc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 --2- 172.21.15.71:0/1758939519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb758074620 0x7fb758074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 -- 172.21.15.71:0/1758939519 >> 172.21.15.71:0/1758939519 conn(0x7fb7580fd700 msgr2=0x7fb7580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 -- 172.21.15.71:0/1758939519 shutdown_connections 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.410+0000 7fb75d000640 1 -- 172.21.15.71:0/1758939519 wait complete. 2024-09-05T12:29:46.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.411+0000 7fb75d000640 1 Processor -- start 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.411+0000 7fb75d000640 1 -- start start 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb75d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 0x7fb75819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb75d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb758074f40 0x7fb75819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb75d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 0x7fb75819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb75d000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb758113fb0 con 0x7fb758074620 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb75d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb758113e30 con 0x7fb758074f40 2024-09-05T12:29:46.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb75d000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb758114130 con 0x7fb75810e200 2024-09-05T12:29:46.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb757400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 0x7fb75819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb757e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 0x7fb75819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb757e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 0x7fb75819edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52264/0 (socket says 172.21.15.71:52264) 2024-09-05T12:29:46.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb757e00640 1 -- 172.21.15.71:0/3000243522 learned_addr learned my addr 172.21.15.71:0/3000243522 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:46.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.412+0000 7fb757400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 0x7fb75819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60420/0 (socket says 172.21.15.71:60420) 2024-09-05T12:29:46.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb756a00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb758074f40 0x7fb75819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb757e00640 1 -- 172.21.15.71:0/3000243522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb758074f40 msgr2=0x7fb75819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb757e00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb758074f40 0x7fb75819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb757e00640 1 -- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 msgr2=0x7fb75819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb757e00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 0x7fb75819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb757e00640 1 -- 172.21.15.71:0/3000243522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75819f4a0 con 0x7fb75810e200 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.413+0000 7fb757400640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 0x7fb75819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.414+0000 7fb757e00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 0x7fb75819edc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb75000efc0 tx=0x7fb75000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.414+0000 7fb74fe00640 1 -- 172.21.15.71:0/3000243522 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb750003970 con 0x7fb75810e200 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.414+0000 7fb74fe00640 1 -- 172.21.15.71:0/3000243522 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb750004020 con 0x7fb75810e200 2024-09-05T12:29:46.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.414+0000 7fb74fe00640 1 -- 172.21.15.71:0/3000243522 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75000b9f0 con 0x7fb75810e200 2024-09-05T12:29:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.414+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75819f790 con 0x7fb75810e200 2024-09-05T12:29:46.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.414+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7581a7070 con 0x7fb75810e200 2024-09-05T12:29:46.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.417+0000 7fb7610ca740 1 Processor -- start 2024-09-05T12:29:46.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.418+0000 7fb74fe00640 1 -- 172.21.15.71:0/3000243522 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb75001d020 con 0x7fb75810e200 2024-09-05T12:29:46.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.418+0000 7fb7610ca740 1 -- start start 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9a520 0x55bc06c9a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 0x55bc06ca51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06ca7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb7610ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bc06c00650 con 0x55bc06c9ae40 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb7610ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc06ba7f10 con 0x55bc06ca5700 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb7610ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc06bfc610 con 0x55bc06c9a520 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb757400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 0x55bc06ca51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb757400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 0x55bc06ca51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60426/0 (socket says 172.21.15.71:60426) 2024-09-05T12:29:46.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb757400640 1 -- 172.21.15.71:0/3669571876 learned_addr learned my addr 172.21.15.71:0/3669571876 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb756a00640 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06ca7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.419+0000 7fb756a00640 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9a520 0x55bc06c9a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb757400640 1 -- 172.21.15.71:0/3669571876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9a520 msgr2=0x55bc06c9a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb757400640 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9a520 0x55bc06c9a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb757400640 1 -- 172.21.15.71:0/3669571876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 msgr2=0x55bc06ca7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb757400640 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06ca7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb757400640 1 -- 172.21.15.71:0/3669571876 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0690f580 con 0x55bc06c9ae40 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb74fe00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb73c0a2d70 0x7fb73c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb757400640 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 0x55bc06ca51c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb7480115f0 tx=0x7fb748011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded851d59c6b9eef server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb74fe00640 1 -- 172.21.15.71:0/3000243522 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb7500cf570 con 0x7fb75810e200 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb74cc00640 1 -- 172.21.15.71:0/3669571876 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb74801e070 con 0x55bc06c9ae40 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.420+0000 7fb74cc00640 1 -- 172.21.15.71:0/3669571876 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74800fb20 con 0x55bc06c9ae40 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.421+0000 7fb74cc00640 1 -- 172.21.15.71:0/3669571876 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb748019c80 con 0x55bc06c9ae40 2024-09-05T12:29:46.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.421+0000 7fb756a00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb73c0a2d70 0x7fb73c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.421+0000 7fb7610ca740 1 -- 172.21.15.71:0/3669571876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 msgr2=0x55bc06ca51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.421+0000 7fb756a00640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb73c0a2d70 0x7fb73c0a5230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb75810f5f0 tx=0x7fb744041040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:46.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.421+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 0x55bc06ca51c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb7480115f0 tx=0x7fb748011ab0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 -- 172.21.15.71:0/3669571876 shutdown_connections 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06ca7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9ae40 0x55bc06ca51c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3669571876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9a520 0x55bc06c9a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 -- 172.21.15.71:0/3669571876 >> 172.21.15.71:0/3669571876 conn(0x55bc06c96ab0 msgr2=0x55bc06c96e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 -- 172.21.15.71:0/3669571876 shutdown_connections 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 -- 172.21.15.71:0/3669571876 wait complete. 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.422+0000 7fb7610ca740 1 -- start start 2024-09-05T12:29:46.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 Processor -- start 2024-09-05T12:29:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06ca2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06ca2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bc06ca82d0 con 0x55bc06ca5700 2024-09-05T12:29:46.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc06bbb350 con 0x55bc06c9a520 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc06ca8450 con 0x55bc06c9ae40 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.423+0000 7fb7610ca740 10 client.0 _ll_register_callbacks cb 0x55bc06bde290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb757400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06ca2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb757400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06ca2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35092/0 (socket says 172.21.15.71:35092) 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb757400640 1 -- 172.21.15.71:0/2467846519 learned_addr learned my addr 172.21.15.71:0/2467846519 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb756a00640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06ca2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb757e00640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb756a00640 1 -- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 msgr2=0x55bc06ca2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb756a00640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06ca2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb756a00640 1 -- 172.21.15.71:0/2467846519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 msgr2=0x55bc06ca3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb756a00640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.424+0000 7fb756a00640 1 -- 172.21.15.71:0/2467846519 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc069d9960 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb756a00640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06ca2bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb74402a8f0 tx=0x7fb74402a3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6bf60bf620b60e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb741600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb741600640 1 -- 172.21.15.71:0/2467846519 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb744050070 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb741600640 1 -- 172.21.15.71:0/2467846519 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb7440046c0 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb74800fb20 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb741600640 1 -- 172.21.15.71:0/2467846519 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb74405e050 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb7610ca740 10 client.25660 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.425+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bc06cb3fd0 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb741600640 1 -- 172.21.15.71:0/2467846519 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb74800fb20 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb741600640 1 -- 172.21.15.71:0/2467846519 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb74402f8a0 con 0x55bc06c9ae40 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb7610ca740 10 client.25660 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:46.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb7610ca740 10 client.25660 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:29:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb7610ca740 20 client.25660 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb7610ca740 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06cb5490 0x55bc06cb7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb7610ca740 4 client.25660 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.426+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc06c0dbc0 con 0x55bc06cb5490 2024-09-05T12:29:46.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.427+0000 7fb757400640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06cb5490 0x55bc06cb7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.427+0000 7fb757400640 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06cb5490 0x55bc06cb7900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea1b20398d7ef6b server_cookie=8cfebc2f1bf0cec1 in_seq=0 out_seq=0 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.427+0000 7fb741600640 10 client.25660 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.428+0000 7fb741600640 1 -- 172.21.15.71:0/2467846519 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc06c0dbc0 con 0x55bc06cb5490 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.428+0000 7fb741600640 10 client.25660 handle_command_reply: tid=0 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.428+0000 7fb7610ca740 1 client.25660 shutdown 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 msgr2=0x55bc06ca2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06ca2bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb74402a8f0 tx=0x7fb74402a3d0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 shutdown_connections 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06cb5490 0x55bc06cb7900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06ca2bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 --2- 172.21.15.71:0/2467846519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06ca2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb741600640 0 client.25660 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:46.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 >> 172.21.15.71:0/2467846519 conn(0x55bc06c96ab0 msgr2=0x55bc06c95cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:46.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 shutdown_connections 2024-09-05T12:29:46.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.429+0000 7fb7610ca740 1 -- 172.21.15.71:0/2467846519 wait complete. 2024-09-05T12:29:46.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.430+0000 7fb7610ca740 20 client.25660 trim_cache size 0 max 16384 2024-09-05T12:29:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.457+0000 7fb7610ca740 1 Processor -- start 2024-09-05T12:29:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.457+0000 7fb7610ca740 1 -- start start 2024-09-05T12:29:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9a520 0x55bc06c9c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cbd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06cbd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb7610ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b0f50 con 0x55bc06c9a520 2024-09-05T12:29:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb7610ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b2af0 con 0x55bc06ca5700 2024-09-05T12:29:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb7610ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b1620 con 0x55bc06c9ae40 2024-09-05T12:29:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb757400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9a520 0x55bc06c9c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb757e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06cbd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb757e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06cbd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35096/0 (socket says 172.21.15.71:35096) 2024-09-05T12:29:46.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb757e00640 1 -- 172.21.15.71:0/3340405922 learned_addr learned my addr 172.21.15.71:0/3340405922 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:46.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.458+0000 7fb756a00640 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cbd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb757e00640 1 -- 172.21.15.71:0/3340405922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 msgr2=0x55bc06cbd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb757e00640 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cbd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb757e00640 1 -- 172.21.15.71:0/3340405922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9a520 msgr2=0x55bc06c9c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb757e00640 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9a520 0x55bc06c9c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb757e00640 1 -- 172.21.15.71:0/3340405922 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc06c97740 con 0x55bc06ca5700 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb757e00640 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06cbd710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb75009b680 tx=0x7fb750097320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80e596f0bec7112 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.459+0000 7fb743e00640 1 -- 172.21.15.71:0/3340405922 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500b3020 con 0x55bc06ca5700 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb743e00640 1 -- 172.21.15.71:0/3340405922 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75009faa0 con 0x55bc06ca5700 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb743e00640 1 -- 172.21.15.71:0/3340405922 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7500041d0 con 0x55bc06ca5700 2024-09-05T12:29:46.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 -- 172.21.15.71:0/3340405922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 msgr2=0x55bc06cbd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06cbd710 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb75009b680 tx=0x7fb750097320 comp rx=0 tx=0).stop 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 -- 172.21.15.71:0/3340405922 shutdown_connections 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06ca5700 0x55bc06cbd710 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cbd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 --2- 172.21.15.71:0/3340405922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06c9a520 0x55bc06c9c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.460+0000 7fb7610ca740 1 -- 172.21.15.71:0/3340405922 >> 172.21.15.71:0/3340405922 conn(0x55bc06c96ab0 msgr2=0x55bc06c9bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.461+0000 7fb7610ca740 1 -- 172.21.15.71:0/3340405922 shutdown_connections 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.461+0000 7fb7610ca740 1 -- 172.21.15.71:0/3340405922 wait complete. 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.461+0000 7fb7610ca740 1 -- start start 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.461+0000 7fb7610ca740 1 Processor -- start 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06cad770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cadcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b23c0 con 0x55bc06ca5700 2024-09-05T12:29:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581b1cf0 con 0x55bc06c9a520 2024-09-05T12:29:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7581aef30 con 0x55bc06c9ae40 2024-09-05T12:29:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb7610ca740 10 client.0 _ll_register_callbacks cb 0x7fb748005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:46.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb757e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb756a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cadcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb757e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60460/0 (socket says 172.21.15.71:60460) 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb756a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cadcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52298/0 (socket says 172.21.15.71:52298) 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb757e00640 1 -- 172.21.15.71:0/1381580361 learned_addr learned my addr 172.21.15.71:0/1381580361 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.462+0000 7fb757400640 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06cad770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb757e00640 1 -- 172.21.15.71:0/1381580361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 msgr2=0x55bc06cadcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb757e00640 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cadcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb757e00640 1 -- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 msgr2=0x55bc06cad770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:46.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb757e00640 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06cad770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb757e00640 1 -- 172.21.15.71:0/1381580361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc06c96360 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb757e00640 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca9bb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb750005c40 tx=0x7fb75009b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1860cead15be9faa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb741600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb741600640 1 -- 172.21.15.71:0/1381580361 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75007e020 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb741600640 1 -- 172.21.15.71:0/1381580361 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb750097d60 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.463+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb75009faa0 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb741600640 1 -- 172.21.15.71:0/1381580361 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb7500a9050 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 10 client.15609 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bc06d20b10 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb741600640 1 -- 172.21.15.71:0/1381580361 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb75009faa0 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb741600640 1 -- 172.21.15.71:0/1381580361 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7fb75000b040 con 0x55bc06ca5700 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 10 client.15609 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:46.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 10 client.15609 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 20 client.15609 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06d21170 0x55bc06d23590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 4 client.15609 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "37865911", "release": true, "await": true}] 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.464+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "37865911", "release": true, "await": true}) -- 0x55bc06c080d0 con 0x55bc06d21170 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.465+0000 7fb757400640 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06d21170 0x55bc06d23590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.465+0000 7fb757400640 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06d21170 0x55bc06d23590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21e17d9dae768e14 server_cookie=6cac42c48ff79204 in_seq=0 out_seq=0 2024-09-05T12:29:46.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.465+0000 7fb741600640 10 client.15609 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:46.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.475+0000 7fb741600640 1 -- 172.21.15.71:0/1381580361 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55bc06c080d0 con 0x55bc06d21170 2024-09-05T12:29:46.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.475+0000 7fb741600640 10 client.15609 handle_command_reply: tid=0 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.475+0000 7fb7610ca740 1 client.15609 shutdown 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.475+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 msgr2=0x55bc06ca9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.475+0000 7fb7610ca740 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca9bb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb750005c40 tx=0x7fb75009b5b0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 shutdown_connections 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bc06d21170 0x55bc06d23590 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc06ca5700 0x55bc06ca9bb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc06c9ae40 0x55bc06cadcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 --2- 172.21.15.71:0/1381580361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc06c9a520 0x55bc06cad770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 >> 172.21.15.71:0/1381580361 conn(0x55bc06c96ab0 msgr2=0x55bc06c9c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb741600640 0 client.15609 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 shutdown_connections 2024-09-05T12:29:46.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 1 -- 172.21.15.71:0/1381580361 wait complete. 2024-09-05T12:29:46.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.476+0000 7fb7610ca740 20 client.15609 trim_cache size 0 max 16384 2024-09-05T12:29:46.479 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:29:46.479 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 31, 2024-09-05T12:29:46.479 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:29:46.479 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 8, 2024-09-05T12:29:46.479 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "37865911": { 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 8, 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 137.7, 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:29:46.480 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:29:46.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb73c0a2d70 msgr2=0x7fb73c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb73c0a2d70 0x7fb73c0a5230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb75810f5f0 tx=0x7fb744041040 comp rx=0 tx=0).stop 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 msgr2=0x7fb75819edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 0x7fb75819edc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb75000efc0 tx=0x7fb75000c800 comp rx=0 tx=0).stop 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 shutdown_connections 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb75810e200 0x7fb75819edc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb758074f40 0x7fb75819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb73c0a2d70 0x7fb73c0a5230 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 --2- 172.21.15.71:0/3000243522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb758074620 0x7fb75819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 >> 172.21.15.71:0/3000243522 conn(0x7fb7580fd700 msgr2=0x7fb7580fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 shutdown_connections 2024-09-05T12:29:46.482 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:46.477+0000 7fb75d000640 1 -- 172.21.15.71:0/3000243522 wait complete. 2024-09-05T12:29:46.684 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:46 smithi071 ceph-mon[29845]: pgmap v63: 129 pgs: 129 active+clean; 46 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-09-05T12:29:46.731 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:46 smithi123 ceph-mon[34321]: pgmap v63: 129 pgs: 129 active+clean; 46 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-09-05T12:29:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:46 smithi174 ceph-mon[34313]: pgmap v63: 129 pgs: 129 active+clean; 46 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-09-05T12:29:47.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72606 9.50 MB/sec warmup 43 sec latency 17964.382 ms 2024-09-05T12:29:47.535 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '37865911', total seconds elapsed: 22.6 2024-09-05T12:29:47.535 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.7 2024-09-05T12:29:47.535 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:29:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.905+0000 7ff74d800640 1 -- 172.21.15.71:0/3873492733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748112990 msgr2=0x7ff748114d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.905+0000 7ff74d800640 1 --2- 172.21.15.71:0/3873492733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748112990 0x7ff748114d80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff734009860 tx=0x7ff73402a340 comp rx=0 tx=0).stop 2024-09-05T12:29:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.906+0000 7ff74d800640 1 -- 172.21.15.71:0/3873492733 shutdown_connections 2024-09-05T12:29:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.906+0000 7ff74d800640 1 --2- 172.21.15.71:0/3873492733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748112990 0x7ff748114d80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.906+0000 7ff74d800640 1 --2- 172.21.15.71:0/3873492733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748077040 0x7ff748112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.906+0000 7ff74d800640 1 --2- 172.21.15.71:0/3873492733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff748076720 0x7ff748076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.906+0000 7ff74d800640 1 -- 172.21.15.71:0/3873492733 >> 172.21.15.71:0/3873492733 conn(0x7ff7480ff970 msgr2=0x7ff748101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.907+0000 7ff74d800640 1 -- 172.21.15.71:0/3873492733 shutdown_connections 2024-09-05T12:29:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.907+0000 7ff74d800640 1 -- 172.21.15.71:0/3873492733 wait complete. 2024-09-05T12:29:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.910+0000 7ff74d800640 1 Processor -- start 2024-09-05T12:29:47.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.910+0000 7ff74d800640 1 -- start start 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.911+0000 7ff74d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff748076720 0x7ff7481a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.911+0000 7ff74d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748077040 0x7ff7481a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.911+0000 7ff74d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 0x7ff74819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.912+0000 7ff747400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 0x7ff74819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.912+0000 7ff747400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 0x7ff74819e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35128/0 (socket says 172.21.15.71:35128) 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.912+0000 7ff747400640 1 -- 172.21.15.71:0/2731102818 learned_addr learned my addr 172.21.15.71:0/2731102818 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.912+0000 7ff746000640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748077040 0x7ff7481a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.913+0000 7ff746a00640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff748076720 0x7ff7481a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.913+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481184d0 con 0x7ff748077040 2024-09-05T12:29:47.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.913+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff748118350 con 0x7ff748112990 2024-09-05T12:29:47.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.913+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff748118650 con 0x7ff748076720 2024-09-05T12:29:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.917+0000 7ff747400640 1 -- 172.21.15.71:0/2731102818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff748076720 msgr2=0x7ff7481a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.917+0000 7ff747400640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff748076720 0x7ff7481a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.919 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.917+0000 7ff747400640 1 -- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748077040 msgr2=0x7ff7481a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.917+0000 7ff747400640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748077040 0x7ff7481a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.917+0000 7ff747400640 1 -- 172.21.15.71:0/2731102818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff74819ef30 con 0x7ff748112990 2024-09-05T12:29:47.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.923+0000 7ff747400640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 0x7ff74819e720 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff73402a860 tx=0x7ff734033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.923+0000 7ff73f400640 1 -- 172.21.15.71:0/2731102818 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734004260 con 0x7ff748112990 2024-09-05T12:29:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.923+0000 7ff73f400640 1 -- 172.21.15.71:0/2731102818 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff734048040 con 0x7ff748112990 2024-09-05T12:29:47.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.923+0000 7ff73f400640 1 -- 172.21.15.71:0/2731102818 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7340046c0 con 0x7ff748112990 2024-09-05T12:29:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.924+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff74819f1c0 con 0x7ff748112990 2024-09-05T12:29:47.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.924+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff74806bc10 con 0x7ff748112990 2024-09-05T12:29:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.929+0000 7ff7506bb740 1 Processor -- start 2024-09-05T12:29:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.929+0000 7ff73f400640 1 -- 172.21.15.71:0/2731102818 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff734053030 con 0x7ff748112990 2024-09-05T12:29:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.929+0000 7ff7506bb740 1 -- start start 2024-09-05T12:29:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.929+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b6f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b79e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff7506bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3c5ad5100 con 0x55e3c5b6f1e0 2024-09-05T12:29:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff7506bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3c5a7d1b0 con 0x55e3c5b6fb00 2024-09-05T12:29:47.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff7506bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3c5ad10c0 con 0x55e3c5b7a3a0 2024-09-05T12:29:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b79e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b79e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35130/0 (socket says 172.21.15.71:35130) 2024-09-05T12:29:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 -- 172.21.15.71:0/1574474747 learned_addr learned my addr 172.21.15.71:0/1574474747 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 -- 172.21.15.71:0/1574474747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 msgr2=0x55e3c5b7c790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:29:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 -- 172.21.15.71:0/1574474747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 msgr2=0x55e3c5b6f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.930+0000 7ff746a00640 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b6f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.931+0000 7ff746a00640 1 -- 172.21.15.71:0/1574474747 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c57e8210 con 0x55e3c5b6fb00 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.931+0000 7ff746a00640 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b79e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff740009f80 tx=0x7ff7400108a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c27932a63e4127 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.931+0000 7ff733e00640 1 -- 172.21.15.71:0/1574474747 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74000e8c0 con 0x55e3c5b6fb00 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.931+0000 7ff73f400640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff72c0a2f80 0x7ff72c0a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.931+0000 7ff733e00640 1 -- 172.21.15.71:0/1574474747 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff74000c3e0 con 0x55e3c5b6fb00 2024-09-05T12:29:47.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.931+0000 7ff733e00640 1 -- 172.21.15.71:0/1574474747 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740019600 con 0x55e3c5b6fb00 2024-09-05T12:29:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.932+0000 7ff73f400640 1 -- 172.21.15.71:0/2731102818 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7ff7340ffef0 con 0x7ff748112990 2024-09-05T12:29:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.932+0000 7ff746000640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff72c0a2f80 0x7ff72c0a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 -- 172.21.15.71:0/1574474747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 msgr2=0x55e3c5b79e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:47.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b79e60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff740009f80 tx=0x7ff7400108a0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 -- 172.21.15.71:0/1574474747 shutdown_connections 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b79e60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 --2- 172.21.15.71:0/1574474747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b6f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.933+0000 7ff7506bb740 1 -- 172.21.15.71:0/1574474747 >> 172.21.15.71:0/1574474747 conn(0x55e3c5b6b750 msgr2=0x55e3c5b6bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.934+0000 7ff746000640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff72c0a2f80 0x7ff72c0a5440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff738005d90 tx=0x7ff738005e40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.934+0000 7ff7506bb740 1 -- 172.21.15.71:0/1574474747 shutdown_connections 2024-09-05T12:29:47.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.934+0000 7ff7506bb740 1 -- 172.21.15.71:0/1574474747 wait complete. 2024-09-05T12:29:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.935+0000 7ff7506bb740 1 -- start start 2024-09-05T12:29:47.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.935+0000 7ff7506bb740 1 Processor -- start 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b772f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b77830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b77d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff7506bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3c5b7cfd0 con 0x55e3c5b6f1e0 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff7506bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3c5a5e090 con 0x55e3c5b6fb00 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff7506bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3c5b7d150 con 0x55e3c5b7a3a0 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff746a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b772f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.936+0000 7ff746000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b77830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b77830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35142/0 (socket says 172.21.15.71:35142) 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 -- 172.21.15.71:0/345312360 learned_addr learned my addr 172.21.15.71:0/345312360 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff747400640 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b77d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 -- 172.21.15.71:0/345312360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 msgr2=0x55e3c5b77d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b77d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 -- 172.21.15.71:0/345312360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 msgr2=0x55e3c5b772f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b772f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 -- 172.21.15.71:0/345312360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c58b4620 con 0x55e3c5b6fb00 2024-09-05T12:29:47.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff7506bb740 10 client.0 _ll_register_callbacks cb 0x55e3c5ab2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74000c3e0 con 0x55e3c5b6fb00 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.937+0000 7ff746000640 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b77830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7380106e0 tx=0x7ff738004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=131a4b50fa248da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.938+0000 7ff730c00640 10 client.25667 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.938+0000 7ff730c00640 1 -- 172.21.15.71:0/345312360 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73802b050 con 0x55e3c5b6fb00 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.938+0000 7ff730c00640 1 -- 172.21.15.71:0/345312360 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7ff73800fad0 con 0x55e3c5b6fb00 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.938+0000 7ff730c00640 1 -- 172.21.15.71:0/345312360 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff738021070 con 0x55e3c5b6fb00 2024-09-05T12:29:47.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.938+0000 7ff7506bb740 10 client.25667 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.938+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55e3c5b99bc0 con 0x55e3c5b6fb00 2024-09-05T12:29:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.939+0000 7ff730c00640 1 -- 172.21.15.71:0/345312360 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff73801d450 con 0x55e3c5b6fb00 2024-09-05T12:29:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.940+0000 7ff7506bb740 10 client.25667 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.940+0000 7ff7506bb740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.940+0000 7ff7506bb740 20 client.25667 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:47.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.940+0000 7ff7506bb740 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5b9af70 0x55e3c5b9d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.940+0000 7ff7506bb740 4 client.25667 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.940+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3c5ae2830 con 0x55e3c5b9af70 2024-09-05T12:29:47.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.941+0000 7ff746a00640 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5b9af70 0x55e3c5b9d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.942+0000 7ff746a00640 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5b9af70 0x55e3c5b9d360 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5506ef9f233b3a server_cookie=894071c7a6dc527f in_seq=0 out_seq=0 2024-09-05T12:29:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.942+0000 7ff730c00640 10 client.25667 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:47.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.943+0000 7ff730c00640 1 -- 172.21.15.71:0/345312360 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3c5ae2830 con 0x55e3c5b9af70 2024-09-05T12:29:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.943+0000 7ff730c00640 10 client.25667 handle_command_reply: tid=0 2024-09-05T12:29:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.943+0000 7ff7506bb740 1 client.25667 shutdown 2024-09-05T12:29:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 msgr2=0x55e3c5b77830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b77830 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff7380106e0 tx=0x7ff738004730 comp rx=0 tx=0).stop 2024-09-05T12:29:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 shutdown_connections 2024-09-05T12:29:47.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5b9af70 0x55e3c5b9d360 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b77d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b77830 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 --2- 172.21.15.71:0/345312360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b772f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 >> 172.21.15.71:0/345312360 conn(0x55e3c5b6b750 msgr2=0x55e3c5b714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff730c00640 0 client.25667 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 shutdown_connections 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.944+0000 7ff7506bb740 1 -- 172.21.15.71:0/345312360 wait complete. 2024-09-05T12:29:47.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.945+0000 7ff7506bb740 20 client.25667 trim_cache size 0 max 16384 2024-09-05T12:29:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.981+0000 7ff7506bb740 1 Processor -- start 2024-09-05T12:29:48.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.981+0000 7ff7506bb740 1 -- start start 2024-09-05T12:29:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b71c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5ba2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5ba3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff7506bb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff748106600 con 0x55e3c5b6f1e0 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff7506bb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481081a0 con 0x55e3c5b6fb00 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff7506bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff748106cd0 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff747400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5ba3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff747400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5ba3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52358/0 (socket says 172.21.15.71:52358) 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.982+0000 7ff747400640 1 -- 172.21.15.71:0/333938785 learned_addr learned my addr 172.21.15.71:0/333938785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff746a00640 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b71c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff747400640 1 -- 172.21.15.71:0/333938785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 msgr2=0x55e3c5ba2c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff747400640 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5ba2c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff747400640 1 -- 172.21.15.71:0/333938785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 msgr2=0x55e3c5b71c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff747400640 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b71c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff747400640 1 -- 172.21.15.71:0/333938785 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3c5b6f930 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.983+0000 7ff746a00640 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b71c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.984+0000 7ff747400640 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5ba3190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff734005e00 tx=0x7ff7340cbd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ceba57391f05ed server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.984+0000 7ff733400640 1 -- 172.21.15.71:0/333938785 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7340cc910 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.984+0000 7ff733400640 1 -- 172.21.15.71:0/333938785 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7340cc210 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.984+0000 7ff733400640 1 -- 172.21.15.71:0/333938785 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7340ec0c0 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 -- 172.21.15.71:0/333938785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 msgr2=0x55e3c5ba3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5ba3190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff734005e00 tx=0x7ff7340cbd50 comp rx=0 tx=0).stop 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 -- 172.21.15.71:0/333938785 shutdown_connections 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5ba3190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5ba2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 --2- 172.21.15.71:0/333938785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b71c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.985+0000 7ff7506bb740 1 -- 172.21.15.71:0/333938785 >> 172.21.15.71:0/333938785 conn(0x55e3c5b6b750 msgr2=0x55e3c5b763c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.986+0000 7ff7506bb740 1 -- 172.21.15.71:0/333938785 shutdown_connections 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.986+0000 7ff7506bb740 1 -- 172.21.15.71:0/333938785 wait complete. 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.986+0000 7ff7506bb740 1 -- start start 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.986+0000 7ff7506bb740 1 Processor -- start 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b82470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b829b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff7506bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff746a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b82470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff746a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b82470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60508/0 (socket says 172.21.15.71:60508) 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff746a00640 1 -- 172.21.15.71:0/3601952220 learned_addr learned my addr 172.21.15.71:0/3601952220 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff748107a70 con 0x55e3c5b6f1e0 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.987+0000 7ff747400640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff746000640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b829b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7481073a0 con 0x55e3c5b6fb00 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff748076b00 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.036 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff7506bb740 10 client.0 _ll_register_callbacks cb 0x7ff738014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff747400640 1 -- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 msgr2=0x55e3c5b829b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff747400640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b829b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff747400640 1 -- 172.21.15.71:0/3601952220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 msgr2=0x55e3c5b82470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.988+0000 7ff747400640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b82470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.989+0000 7ff747400640 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3c5b7e150 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.989+0000 7ff746a00640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b82470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.989+0000 7ff730c00640 10 client.25681 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.989+0000 7ff747400640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7d940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55e3c5b7c860 tx=0x7ff734032000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56baafcc99edd62c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.989+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7340cc210 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.990+0000 7ff730c00640 1 -- 172.21.15.71:0/3601952220 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7340d05c0 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.990+0000 7ff730c00640 1 -- 172.21.15.71:0/3601952220 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7ff7340cc470 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.037 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.990+0000 7ff730c00640 1 -- 172.21.15.71:0/3601952220 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff7340cc670 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.990+0000 7ff7506bb740 10 client.25681 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.990+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=31}) -- 0x55e3c5c05620 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff730c00640 1 -- 172.21.15.71:0/3601952220 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7ff7340d4960 con 0x55e3c5b7a3a0 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff7506bb740 10 client.25681 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff7506bb740 10 client.25681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff7506bb740 20 client.25681 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff7506bb740 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5c05b40 0x55e3c5c07f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff7506bb740 4 client.25681 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:47.992+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3c5adcb80 con 0x55e3c5c05b40 2024-09-05T12:29:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.013+0000 7ff746a00640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5c05b40 0x55e3c5c07f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.038 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:29:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.061+0000 7ff730c00640 10 client.25681 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.061+0000 7ff746a00640 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5c05b40 0x55e3c5c07f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b323def7237c2de server_cookie=c3e721dfe728828a in_seq=0 out_seq=0 2024-09-05T12:29:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.187+0000 7ff730c00640 1 -- 172.21.15.71:0/3601952220 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3c5adcb80 con 0x55e3c5c05b40 2024-09-05T12:29:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.187+0000 7ff730c00640 10 client.25681 handle_command_reply: tid=0 2024-09-05T12:29:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.187+0000 7ff7506bb740 1 client.25681 shutdown 2024-09-05T12:29:48.190 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 msgr2=0x55e3c5b7d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7d940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55e3c5b7c860 tx=0x7ff734032000 comp rx=0 tx=0).stop 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 shutdown_connections 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3c5c05b40 0x55e3c5c07f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3c5b7a3a0 0x55e3c5b7d940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3c5b6fb00 0x55e3c5b829b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 --2- 172.21.15.71:0/3601952220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3c5b6f1e0 0x55e3c5b82470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 >> 172.21.15.71:0/3601952220 conn(0x55e3c5b6b750 msgr2=0x55e3c5b77290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff730c00640 0 client.25681 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 shutdown_connections 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 1 -- 172.21.15.71:0/3601952220 wait complete. 2024-09-05T12:29:48.191 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.188+0000 7ff7506bb740 20 client.25681 trim_cache size 0 max 16384 2024-09-05T12:29:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:29:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:29:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:29:48.192 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:29:48.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 72631 9.29 MB/sec warmup 44 sec latency 18471.084 ms 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff72c0a2f80 msgr2=0x7ff72c0a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff72c0a2f80 0x7ff72c0a5440 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff738005d90 tx=0x7ff738005e40 comp rx=0 tx=0).stop 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 msgr2=0x7ff74819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 0x7ff74819e720 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff73402a860 tx=0x7ff734033d10 comp rx=0 tx=0).stop 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 shutdown_connections 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff748112990 0x7ff74819e720 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff72c0a2f80 0x7ff72c0a5440 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff748077040 0x7ff7481a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 --2- 172.21.15.71:0/2731102818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff748076720 0x7ff7481a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.190+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 >> 172.21.15.71:0/2731102818 conn(0x7ff7480ff970 msgr2=0x7ff748101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.191+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 shutdown_connections 2024-09-05T12:29:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:29:48.191+0000 7ff74d800640 1 -- 172.21.15.71:0/2731102818 wait complete. 2024-09-05T12:29:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:48 smithi071 ceph-mon[29845]: pgmap v64: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.4 MiB/s wr, 194 op/s 2024-09-05T12:29:48.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 -- 172.21.15.71:0/1885326562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd01047e0 msgr2=0x7febd0108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 --2- 172.21.15.71:0/1885326562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd01047e0 0x7febd0108830 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7febbc009900 tx=0x7febbc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 -- 172.21.15.71:0/1885326562 shutdown_connections 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 --2- 172.21.15.71:0/1885326562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd0108f60 0x7febd0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 --2- 172.21.15.71:0/1885326562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd01047e0 0x7febd0108830 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 --2- 172.21.15.71:0/1885326562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0103e30 0x7febd0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.676+0000 7febd6a00640 1 -- 172.21.15.71:0/1885326562 >> 172.21.15.71:0/1885326562 conn(0x7febd00ff910 msgr2=0x7febd0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.677+0000 7febd6a00640 1 -- 172.21.15.71:0/1885326562 shutdown_connections 2024-09-05T12:29:48.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.677+0000 7febd6a00640 1 -- 172.21.15.71:0/1885326562 wait complete. 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 Processor -- start 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 -- start start 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0103e30 0x7febd019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 0x7febd019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd0108f60 0x7febd019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febd0114010 con 0x7febd01047e0 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febd0113e90 con 0x7febd0108f60 2024-09-05T12:29:48.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd6a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7febd0114190 con 0x7febd0103e30 2024-09-05T12:29:48.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.678+0000 7febd4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 0x7febd019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 0x7febd019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60530/0 (socket says 172.21.15.71:60530) 2024-09-05T12:29:48.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 -- 172.21.15.71:0/3890496589 learned_addr learned my addr 172.21.15.71:0/3890496589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:48.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd5600640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0103e30 0x7febd019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 -- 172.21.15.71:0/3890496589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0103e30 msgr2=0x7febd019a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0103e30 0x7febd019a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 -- 172.21.15.71:0/3890496589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd0108f60 msgr2=0x7febd019ed60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:29:48.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd0108f60 0x7febd019ed60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 -- 172.21.15.71:0/3890496589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd019f440 con 0x7febd01047e0 2024-09-05T12:29:48.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.679+0000 7febd4c00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 0x7febd019a9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7febbc004770 tx=0x7febbc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:48.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.680+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc03f070 con 0x7febd01047e0 2024-09-05T12:29:48.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.680+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd019f6d0 con 0x7febd01047e0 2024-09-05T12:29:48.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.680+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febbc031ca0 con 0x7febd01047e0 2024-09-05T12:29:48.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.680+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a7010 con 0x7febd01047e0 2024-09-05T12:29:48.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.680+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc049060 con 0x7febd01047e0 2024-09-05T12:29:48.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.681+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd0104210 con 0x7febd01047e0 2024-09-05T12:29:48.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.683+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7febbc042070 con 0x7febd01047e0 2024-09-05T12:29:48.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.685+0000 7febce000640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7febb80a2cf0 0x7febb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:48.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.685+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7febbc0efe00 con 0x7febd01047e0 2024-09-05T12:29:48.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.685+0000 7febd5600640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7febb80a2cf0 0x7febb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:48.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.686+0000 7febd5600640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7febb80a2cf0 0x7febb80a51b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febc4009bd0 tx=0x7febc4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:48.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.690+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febbc0bc740 con 0x7febd01047e0 2024-09-05T12:29:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:48 smithi174 ceph-mon[34313]: pgmap v64: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.4 MiB/s wr, 194 op/s 2024-09-05T12:29:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:48 smithi123 ceph-mon[34321]: pgmap v64: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 3.4 MiB/s wr, 194 op/s 2024-09-05T12:29:48.937 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.937+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd010bc10 con 0x7febd01047e0 2024-09-05T12:29:48.938 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.938+0000 7febce000640 1 -- 172.21.15.71:0/3890496589 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11971 (secure 0 0 0) 0x7febbc0bc040 con 0x7febd01047e0 2024-09-05T12:29:48.939 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:29:48.939 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":31,"btime":"2024-09-05T12:29:32:411358+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:29:31.941232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:29:48.939 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 31 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7febb80a2cf0 msgr2=0x7febb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7febb80a2cf0 0x7febb80a51b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7febc4009bd0 tx=0x7febc4005f90 comp rx=0 tx=0).stop 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 msgr2=0x7febd019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 0x7febd019a9d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7febbc004770 tx=0x7febbc003c10 comp rx=0 tx=0).stop 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 shutdown_connections 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febd0108f60 0x7febd019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7febb80a2cf0 0x7febb80a51b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febd01047e0 0x7febd019a9d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.941 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 --2- 172.21.15.71:0/3890496589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7febd0103e30 0x7febd019a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 >> 172.21.15.71:0/3890496589 conn(0x7febd00ff910 msgr2=0x7febd0100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 shutdown_connections 2024-09-05T12:29:48.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:48.940+0000 7febd6a00640 1 -- 172.21.15.71:0/3890496589 wait complete. 2024-09-05T12:29:49.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74658 9.36 MB/sec warmup 45 sec latency 316.408 ms 2024-09-05T12:29:49.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:49 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3890496589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:49.744 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0' 2024-09-05T12:29:49.744 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 137.7 --await-for 45.0 2024-09-05T12:29:49.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:49 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3890496589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:49.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:49 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3890496589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:29:50.118 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:29:50.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74658 9.16 MB/sec warmup 46 sec latency 1316.475 ms 2024-09-05T12:29:50.514 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:50 smithi071 ceph-mon[29845]: pgmap v65: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-09-05T12:29:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:50 smithi174 ceph-mon[34313]: pgmap v65: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-09-05T12:29:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:50 smithi123 ceph-mon[34321]: pgmap v65: 129 pgs: 129 active+clean; 43 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.9 MiB/s wr, 89 op/s 2024-09-05T12:29:51.191 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74658 8.96 MB/sec warmup 47 sec latency 2316.519 ms 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 -- 172.21.15.71:0/1366691031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f438010e260 msgr2=0x7f4380110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 --2- 172.21.15.71:0/1366691031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f438010e260 0x7f4380110650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4378009ca0 tx=0x7f437802a950 comp rx=0 tx=0).stop 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 -- 172.21.15.71:0/1366691031 shutdown_connections 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 --2- 172.21.15.71:0/1366691031 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f438010e260 0x7f4380110650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 --2- 172.21.15.71:0/1366691031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4380074ee0 0x7f438010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 --2- 172.21.15.71:0/1366691031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43800745c0 0x7f43800749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 -- 172.21.15.71:0/1366691031 >> 172.21.15.71:0/1366691031 conn(0x7f43800fd6e0 msgr2=0x7f43800ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 -- 172.21.15.71:0/1366691031 shutdown_connections 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.189+0000 7f4386e00640 1 -- 172.21.15.71:0/1366691031 wait complete. 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.190+0000 7f4386e00640 1 Processor -- start 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.190+0000 7f4386e00640 1 -- start start 2024-09-05T12:29:51.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 0x7f438019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4380074ee0 0x7f438019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f438010e260 0x7f438019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4380113fb0 con 0x7f43800745c0 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4380113e30 con 0x7f438010e260 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4380114130 con 0x7f4380074ee0 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 0x7f438019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 0x7f438019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60546/0 (socket says 172.21.15.71:60546) 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 -- 172.21.15.71:0/614578071 learned_addr learned my addr 172.21.15.71:0/614578071 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4386400640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f438010e260 0x7f438019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385000640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4380074ee0 0x7f438019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 -- 172.21.15.71:0/614578071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4380074ee0 msgr2=0x7f438019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4380074ee0 0x7f438019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 -- 172.21.15.71:0/614578071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f438010e260 msgr2=0x7f438019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f438010e260 0x7f438019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.191+0000 7f4385a00640 1 -- 172.21.15.71:0/614578071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f438019f4c0 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.192+0000 7f4385a00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 0x7f438019a510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f437400ea60 tx=0x7f437400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.192+0000 7f437e000640 1 -- 172.21.15.71:0/614578071 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437400cde0 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.192+0000 7f437e000640 1 -- 172.21.15.71:0/614578071 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43740035f0 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.192+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438019f7b0 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.192+0000 7f437e000640 1 -- 172.21.15.71:0/614578071 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374010670 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.192+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43801a7090 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.195+0000 7f437e000640 1 -- 172.21.15.71:0/614578071 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f43740037a0 con 0x7f43800745c0 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.195+0000 7f438ae3a740 1 Processor -- start 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 -- start start 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966b2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966bd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966bfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558c96618530 con 0x558c966b2e80 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c965bf860 con 0x558c966bd740 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f438ae3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c966144f0 con 0x558c966b2560 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f4385a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966bd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f4386400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f4385a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966bd200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60556/0 (socket says 172.21.15.71:60556) 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.196+0000 7f4385a00640 1 -- 172.21.15.71:0/2741104986 learned_addr learned my addr 172.21.15.71:0/2741104986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4386400640 1 -- 172.21.15.71:0/2741104986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 msgr2=0x558c966bfb30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4386400640 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966bfb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4386400640 1 -- 172.21.15.71:0/2741104986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 msgr2=0x558c966bd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4386400640 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966bd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4386400640 1 -- 172.21.15.71:0/2741104986 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c96389b90 con 0x558c966b2560 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f437e000640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f43680a2cf0 0x7f43680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4386400640 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966b2940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f437802ae70 tx=0x7f43780093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a2f1a21ebfeb967 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f437e000640 1 -- 172.21.15.71:0/614578071 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f43740cfaa0 con 0x7f43800745c0 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f4385a00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f43680a2cf0 0x7f43680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f436ea00640 1 -- 172.21.15.71:0/2741104986 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378048070 con 0x558c966b2560 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f436ea00640 1 -- 172.21.15.71:0/2741104986 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437802f940 con 0x558c966b2560 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.197+0000 7f436ea00640 1 -- 172.21.15.71:0/2741104986 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378034040 con 0x558c966b2560 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 -- 172.21.15.71:0/2741104986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 msgr2=0x558c966b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966b2940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f437802ae70 tx=0x7f43780093e0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f4385a00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f43680a2cf0 0x7f43680a51b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43740cfd40 tx=0x7f43740d34a0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 -- 172.21.15.71:0/2741104986 shutdown_connections 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966bfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966bd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2741104986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966b2940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 -- 172.21.15.71:0/2741104986 >> 172.21.15.71:0/2741104986 conn(0x558c966aeaf0 msgr2=0x558c966aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 -- 172.21.15.71:0/2741104986 shutdown_connections 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.198+0000 7f438ae3a740 1 -- 172.21.15.71:0/2741104986 wait complete. 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 -- start start 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 Processor -- start 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2560 0x558c966ba6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966babf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966bd740 0x558c966bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558c966c02b0 con 0x558c966b2e80 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c965d2e60 con 0x558c966b2560 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.199+0000 7f438ae3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c966c0430 con 0x558c966bd740 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f438ae3a740 10 client.0 _ll_register_callbacks cb 0x558c965d7cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966babf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966babf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60564/0 (socket says 172.21.15.71:60564) 2024-09-05T12:29:51.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 -- 172.21.15.71:0/1156350232 learned_addr learned my addr 172.21.15.71:0/1156350232 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 -- 172.21.15.71:0/1156350232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966bd740 msgr2=0x558c966bb130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966bd740 0x558c966bb130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 -- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2560 msgr2=0x558c966ba6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2560 0x558c966ba6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.200+0000 7f4385000640 1 -- 172.21.15.71:0/1156350232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c965de990 con 0x558c966b2e80 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f4385000640 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966babf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4370011720 tx=0x7f4370011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ca35308fac9124 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f4363400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f4363400640 1 -- 172.21.15.71:0/1156350232 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437000f8c0 con 0x558c966b2e80 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f4363400640 1 -- 172.21.15.71:0/1156350232 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f437000fa60 con 0x558c966b2e80 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437802f940 con 0x558c966b2e80 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f4363400640 1 -- 172.21.15.71:0/1156350232 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f437000fc60 con 0x558c966b2e80 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f438ae3a740 10 client.15642 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:51.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.201+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c966cbfa0 con 0x558c966b2e80 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f4363400640 1 -- 172.21.15.71:0/1156350232 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f437802f940 con 0x558c966b2e80 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f4363400640 1 -- 172.21.15.71:0/1156350232 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4370018070 con 0x558c966b2e80 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f438ae3a740 10 client.15642 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f438ae3a740 10 client.15642 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f438ae3a740 20 client.15642 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f438ae3a740 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c966cd500 0x558c966cf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f438ae3a740 4 client.15642 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:29:51.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.202+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c965bb310 con 0x558c966cd500 2024-09-05T12:29:51.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.203+0000 7f4386400640 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c966cd500 0x558c966cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.203+0000 7f4386400640 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c966cd500 0x558c966cf990 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e7d6dd23f9f89c server_cookie=180f37193d985434 in_seq=0 out_seq=0 2024-09-05T12:29:51.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.203+0000 7f4363400640 10 client.15642 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:51.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f4363400640 1 -- 172.21.15.71:0/1156350232 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c965bb310 con 0x558c966cd500 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f4363400640 10 client.15642 handle_command_reply: tid=0 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 client.15642 shutdown 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 msgr2=0x558c966babf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966babf0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4370011720 tx=0x7f4370011be0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 shutdown_connections 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c966cd500 0x558c966cf990 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966bd740 0x558c966bb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2e80 0x558c966babf0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f4363400640 0 client.15642 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 --2- 172.21.15.71:0/1156350232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2560 0x558c966ba6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 >> 172.21.15.71:0/1156350232 conn(0x558c966aeaf0 msgr2=0x558c966b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.204+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 shutdown_connections 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.205+0000 7f438ae3a740 1 -- 172.21.15.71:0/1156350232 wait complete. 2024-09-05T12:29:51.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.205+0000 7f438ae3a740 20 client.15642 trim_cache size 0 max 16384 2024-09-05T12:29:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.222+0000 7f438ae3a740 1 Processor -- start 2024-09-05T12:29:51.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.222+0000 7f438ae3a740 1 -- start start 2024-09-05T12:29:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.222+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966adb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.222+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2e80 0x558c966d5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966bd740 0x558c966d57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f438ae3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43801b0e60 con 0x558c966bd740 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f438ae3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43801b2a00 con 0x558c966b2e80 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f438ae3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43801b1530 con 0x558c966b2560 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966adb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52438/0 (socket says 172.21.15.71:52438) 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 -- 172.21.15.71:0/2811062276 learned_addr learned my addr 172.21.15.71:0/2811062276 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4385000640 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2e80 0x558c966d5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966bd740 0x558c966d57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 -- 172.21.15.71:0/2811062276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2e80 msgr2=0x558c966d5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2e80 0x558c966d5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 -- 172.21.15.71:0/2811062276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966bd740 msgr2=0x558c966d57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:51.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966bd740 0x558c966d57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4386400640 1 -- 172.21.15.71:0/2811062276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c966aa530 con 0x558c966b2560 2024-09-05T12:29:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.223+0000 7f4385000640 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2e80 0x558c966d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:29:51.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.224+0000 7f4386400640 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966adb60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4378009bf0 tx=0x7f4378045040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ede91ccb0f21f9da server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.224+0000 7f436e000640 1 -- 172.21.15.71:0/2811062276 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378048070 con 0x558c966b2560 2024-09-05T12:29:51.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.224+0000 7f436e000640 1 -- 172.21.15.71:0/2811062276 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4378003970 con 0x558c966b2560 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.224+0000 7f436e000640 1 -- 172.21.15.71:0/2811062276 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437804de80 con 0x558c966b2560 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.224+0000 7f438ae3a740 1 -- 172.21.15.71:0/2811062276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 msgr2=0x558c966adb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.224+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966adb60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4378009bf0 tx=0x7f4378045040 comp rx=0 tx=0).stop 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 -- 172.21.15.71:0/2811062276 shutdown_connections 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966bd740 0x558c966d57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966b2e80 0x558c966d5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 --2- 172.21.15.71:0/2811062276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2560 0x558c966adb60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 -- 172.21.15.71:0/2811062276 >> 172.21.15.71:0/2811062276 conn(0x558c966aeaf0 msgr2=0x558c966ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 -- 172.21.15.71:0/2811062276 shutdown_connections 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 -- 172.21.15.71:0/2811062276 wait complete. 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 -- start start 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.225+0000 7f438ae3a740 1 Processor -- start 2024-09-05T12:29:51.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2560 0x558c966c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 0x558c966c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966c1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43801b22d0 con 0x558c966b2560 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43801b1c00 con 0x558c966bd740 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43801aee40 con 0x558c966b2e80 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f438ae3a740 10 client.0 _ll_register_callbacks cb 0x7f43700097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f4386400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2560 0x558c966c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.229 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f4385000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 0x558c966c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f4385000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 0x558c966c5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52452/0 (socket says 172.21.15.71:52452) 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f4385000640 1 -- 172.21.15.71:0/3222229325 learned_addr learned my addr 172.21.15.71:0/3222229325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.226+0000 7f4386400640 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966c1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4385000640 1 -- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 msgr2=0x558c966c1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4385000640 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966c1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4385000640 1 -- 172.21.15.71:0/3222229325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2560 msgr2=0x558c966c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4385000640 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2560 0x558c966c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4385000640 1 -- 172.21.15.71:0/3222229325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c966c27f0 con 0x558c966b2e80 2024-09-05T12:29:51.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4385000640 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 0x558c966c5f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f437001a630 tx=0x7f4370005390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=794716226ee35c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4363400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4363400640 1 -- 172.21.15.71:0/3222229325 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437000f990 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f4363400640 1 -- 172.21.15.71:0/3222229325 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f43700035f0 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.227+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4378003970 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f4363400640 1 -- 172.21.15.71:0/3222229325 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4370036050 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 10 client.25702 fetch_fsmap learned FSMap version 31 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558c96738ec0 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f4363400640 1 -- 172.21.15.71:0/3222229325 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4378003970 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f4363400640 1 -- 172.21.15.71:0/3222229325 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8378+0+0 (secure 0 0 0) 0x7f4370018050 con 0x558c966b2e80 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 10 client.25702 fetch_fsmap finished waiting for FSMap version 31 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 10 client.25702 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 20 client.25702 populate_metadata read hostname 'smithi071' 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c96739790 0x558c9673bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:29:51.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 4 client.25702 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}] 2024-09-05T12:29:51.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.7, "await_for": 45.0}) -- 0x558c96625c60 con 0x558c96739790 2024-09-05T12:29:51.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.228+0000 7f4386400640 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c96739790 0x558c9673bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:29:51.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.229+0000 7f4386400640 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c96739790 0x558c9673bbb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d2f1cd0ab9391a server_cookie=13af1c9acc41eba6 in_seq=0 out_seq=0 2024-09-05T12:29:51.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:51.229+0000 7f4363400640 10 client.25702 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:52.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74658 8.78 MB/sec warmup 48 sec latency 3316.565 ms 2024-09-05T12:29:52.738 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:52 smithi071 ceph-mon[29845]: pgmap v66: 129 pgs: 129 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 150 op/s 2024-09-05T12:29:52.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:52 smithi174 ceph-mon[34313]: pgmap v66: 129 pgs: 129 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 150 op/s 2024-09-05T12:29:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:52 smithi123 ceph-mon[34321]: pgmap v66: 129 pgs: 129 active+clean; 44 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 150 op/s 2024-09-05T12:29:53.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74658 8.60 MB/sec warmup 49 sec latency 4316.607 ms 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f4363400640 1 -- 172.21.15.71:0/3222229325 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558c96625c60 con 0x558c96739790 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f4363400640 10 client.25702 handle_command_reply: tid=0 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 client.25702 shutdown 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 msgr2=0x558c966c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 0x558c966c5f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f437001a630 tx=0x7f4370005390 comp rx=0 tx=0).stop 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 shutdown_connections 2024-09-05T12:29:53.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c96739790 0x558c9673bbb0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c966bd740 0x558c966c1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c966b2e80 0x558c966c5f70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f438ae3a740 1 --2- 172.21.15.71:0/3222229325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c966b2560 0x558c966c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.495+0000 7f4363400640 0 client.25702 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.496+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 >> 172.21.15.71:0/3222229325 conn(0x558c966aeaf0 msgr2=0x558c966adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.496+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 shutdown_connections 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.496+0000 7f438ae3a740 1 -- 172.21.15.71:0/3222229325 wait complete. 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.496+0000 7f438ae3a740 20 client.25702 trim_cache size 0 max 16384 2024-09-05T12:29:53.498 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 32, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 10, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "72401df4": { 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 10, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 137.7, 2024-09-05T12:29:53.499 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:29:53.500 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.497+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f43680a2cf0 msgr2=0x7f43680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.497+0000 7f4386e00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f43680a2cf0 0x7f43680a51b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43740cfd40 tx=0x7f43740d34a0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.497+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 msgr2=0x7f438019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.497+0000 7f4386e00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 0x7f438019a510 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f437400ea60 tx=0x7f437400ef20 comp rx=0 tx=0).stop 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 shutdown_connections 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f438010e260 0x7f438019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4380074ee0 0x7f438019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f43680a2cf0 0x7f43680a51b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 --2- 172.21.15.71:0/614578071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43800745c0 0x7f438019a510 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 >> 172.21.15.71:0/614578071 conn(0x7f43800fd6e0 msgr2=0x7f43800fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:29:53.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 shutdown_connections 2024-09-05T12:29:53.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:29:53.498+0000 7f4386e00640 1 -- 172.21.15.71:0/614578071 wait complete. 2024-09-05T12:29:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:53 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:29:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:29:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:29:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:29:53.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:29:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:53 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:29:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:29:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:29:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:29:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:29:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:53 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:29:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:29:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:29:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:29:53.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:29:54.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 8.42 MB/sec warmup 50 sec latency 4542.344 ms 2024-09-05T12:29:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:54 smithi174 ceph-mon[34313]: pgmap v67: 129 pgs: 129 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 2.7 MiB/s wr, 164 op/s 2024-09-05T12:29:54.890 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:54 smithi071 ceph-mon[29845]: pgmap v67: 129 pgs: 129 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 2.7 MiB/s wr, 164 op/s 2024-09-05T12:29:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:54 smithi123 ceph-mon[34321]: pgmap v67: 129 pgs: 129 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 2.7 MiB/s wr, 164 op/s 2024-09-05T12:29:54.934 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '72401df4', quiesce took 7.4 seconds. Will release after: 10.299999999999999 2024-09-05T12:29:55.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 8.26 MB/sec warmup 51 sec latency 1774.161 ms 2024-09-05T12:29:55.770 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:55 smithi071 ceph-mon[29845]: pgmap v68: 129 pgs: 129 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-09-05T12:29:55.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:55 smithi123 ceph-mon[34321]: pgmap v68: 129 pgs: 129 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-09-05T12:29:56.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:55 smithi174 ceph-mon[34313]: pgmap v68: 129 pgs: 129 active+clean; 44 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-09-05T12:29:56.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 8.10 MB/sec warmup 52 sec latency 2774.251 ms 2024-09-05T12:29:57.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.95 MB/sec warmup 53 sec latency 3774.319 ms 2024-09-05T12:29:57.714 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:57 smithi071 ceph-mon[29845]: pgmap v69: 129 pgs: 129 active+clean; 35 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-09-05T12:29:57.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:57 smithi123 ceph-mon[34321]: pgmap v69: 129 pgs: 129 active+clean; 35 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-09-05T12:29:58.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:57 smithi174 ceph-mon[34313]: pgmap v69: 129 pgs: 129 active+clean; 35 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 2.7 MiB/s wr, 206 op/s 2024-09-05T12:29:58.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.80 MB/sec warmup 54 sec latency 4774.424 ms 2024-09-05T12:29:59.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.66 MB/sec warmup 55 sec latency 5774.525 ms 2024-09-05T12:29:59.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:29:59 smithi071 ceph-mon[29845]: pgmap v70: 129 pgs: 129 active+clean; 35 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-05T12:29:59.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:29:59 smithi123 ceph-mon[34321]: pgmap v70: 129 pgs: 129 active+clean; 35 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-05T12:30:00.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:29:59 smithi174 ceph-mon[34313]: pgmap v70: 129 pgs: 129 active+clean; 35 MiB data, 559 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-09-05T12:30:00.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.52 MB/sec warmup 56 sec latency 6774.646 ms 2024-09-05T12:30:00.885 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:00 smithi071 ceph-mon[29845]: overall HEALTH_OK 2024-09-05T12:30:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:00 smithi123 ceph-mon[34321]: overall HEALTH_OK 2024-09-05T12:30:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:00 smithi174 ceph-mon[34313]: overall HEALTH_OK 2024-09-05T12:30:01.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.39 MB/sec warmup 57 sec latency 7774.711 ms 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3421861267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41018b0 msgr2=0x7ff2c4101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3421861267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41018b0 0x7ff2c4101d10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009930 tx=0x7ff2ac02a420 comp rx=0 tx=0).stop 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3421861267 shutdown_connections 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3421861267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c4102250 0x7ff2c410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3421861267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41018b0 0x7ff2c4101d10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3421861267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c41078c0 0x7ff2c4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3421861267 >> 172.21.15.71:0/3421861267 conn(0x7ff2c40fd5d0 msgr2=0x7ff2c40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:01.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3421861267 shutdown_connections 2024-09-05T12:30:01.321 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.319+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3421861267 wait complete. 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 Processor -- start 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 -- start start 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c41018b0 0x7ff2c40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 0x7ff2c4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41078c0 0x7ff2c4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4111d20 con 0x7ff2c41078c0 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4111ba0 con 0x7ff2c41018b0 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.320+0000 7ff2c3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c4111ea0 con 0x7ff2c4102250 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 0x7ff2c4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41078c0 0x7ff2c4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 0x7ff2c4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48788/0 (socket says 172.21.15.71:48788) 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3917353316 learned_addr learned my addr 172.21.15.71:0/3917353316 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c41018b0 0x7ff2c40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3917353316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c41018b0 msgr2=0x7ff2c40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c41018b0 0x7ff2c40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41078c0 msgr2=0x7ff2c4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41078c0 0x7ff2c4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 -- 172.21.15.71:0/3917353316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c41a6770 con 0x7ff2c4102250 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c41018b0 0x7ff2c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.321+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 0x7ff2c4074a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009230 tx=0x7ff2ac004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.322+0000 7ff2bb400640 1 -- 172.21.15.71:0/3917353316 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac03f030 con 0x7ff2c4102250 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.322+0000 7ff2bb400640 1 -- 172.21.15.71:0/3917353316 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac004440 con 0x7ff2c4102250 2024-09-05T12:30:01.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.322+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c41a6a00 con 0x7ff2c4102250 2024-09-05T12:30:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.322+0000 7ff2bb400640 1 -- 172.21.15.71:0/3917353316 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac035070 con 0x7ff2c4102250 2024-09-05T12:30:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.322+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c41a6ef0 con 0x7ff2c4102250 2024-09-05T12:30:01.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2bb400640 1 -- 172.21.15.71:0/3917353316 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff2ac042030 con 0x7ff2c4102250 2024-09-05T12:30:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 Processor -- start 2024-09-05T12:30:01.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 -- start start 2024-09-05T12:30:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cd5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6ce0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6ce2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5558f6c3b100 con 0x5558f6cd53b0 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558f6be31b0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.325+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5558f6c370c0 con 0x5558f6ce0570 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cd5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6ce0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cd5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60756/0 (socket says 172.21.15.71:60756) 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c3400640 1 -- 172.21.15.71:0/3618517790 learned_addr learned my addr 172.21.15.71:0/3618517790 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6ce2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 -- 172.21.15.71:0/3618517790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 msgr2=0x5558f6ce2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6ce2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 -- 172.21.15.71:0/3618517790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 msgr2=0x5558f6cd5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cd5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 -- 172.21.15.71:0/3618517790 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558f694e210 con 0x5558f6cd5cd0 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2bb400640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2a80a2cf0 0x7ff2a80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.326+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6ce0030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009fb0 tx=0x7ff2b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44788711782fff5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2bb400640 1 -- 172.21.15.71:0/3917353316 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7ff2ac0efad0 con 0x7ff2c4102250 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2b3400640 1 -- 172.21.15.71:0/3618517790 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4004310 con 0x5558f6cd5cd0 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2b3400640 1 -- 172.21.15.71:0/3618517790 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b40044b0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2c3400640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2a80a2cf0 0x7ff2a80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2b3400640 1 -- 172.21.15.71:0/3618517790 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b40035f0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/3618517790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 msgr2=0x5558f6ce0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2c3400640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2a80a2cf0 0x7ff2a80a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc012680 tx=0x7ff2bc009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.327+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6ce0030 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4009fb0 tx=0x7ff2b40118d0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/3618517790 shutdown_connections 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6ce2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6ce0030 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/3618517790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cd5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/3618517790 >> 172.21.15.71:0/3618517790 conn(0x5558f6cd1920 msgr2=0x5558f6cd1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/3618517790 shutdown_connections 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/3618517790 wait complete. 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 -- start start 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.328+0000 7ff2cc3d9740 1 Processor -- start 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cdd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6cddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6cde070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5558f6ce2ff0 con 0x5558f6cd53b0 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558f6bc4090 con 0x5558f6cd5cd0 2024-09-05T12:30:01.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5558f6ce3170 con 0x5558f6ce0570 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2cc3d9740 10 client.0 _ll_register_callbacks cb 0x5558f6c18f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cdd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6cddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cdd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60768/0 (socket says 172.21.15.71:60768) 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2c2a00640 1 -- 172.21.15.71:0/675584656 learned_addr learned my addr 172.21.15.71:0/675584656 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6cddb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:56962/0 (socket says 172.21.15.71:56962) 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 -- 172.21.15.71:0/675584656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 msgr2=0x5558f6cde070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6cde070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 -- 172.21.15.71:0/675584656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 msgr2=0x5558f6cdd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cdd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.329+0000 7ff2bbe00640 1 -- 172.21.15.71:0/675584656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558f6a1a620 con 0x5558f6cd5cd0 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6cddb30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0ece50 tx=0x7ff2ac005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a192cb96d2a368b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:30:01.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2a7e00640 1 -- 172.21.15.71:0/675584656 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac049070 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b40044b0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2a7e00640 1 -- 172.21.15.71:0/675584656 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7ff2ac031bd0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2a7e00640 1 -- 172.21.15.71:0/675584656 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff2ac0bb790 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2cc3d9740 10 client.25697 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558f6ceedd0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.330+0000 7ff2a7e00640 1 -- 172.21.15.71:0/675584656 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2b40044b0 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2a7e00640 1 -- 172.21.15.71:0/675584656 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff2ac0c4430 con 0x5558f6cd5cd0 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2cc3d9740 10 client.25697 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2cc3d9740 10 client.25697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:01.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2cc3d9740 20 client.25697 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6cf0300 0x5558f6cf27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2cc3d9740 4 client.25697 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558f6c48830 con 0x5558f6cf0300 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6cf0300 0x5558f6cf27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6cf0300 0x5558f6cf27c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ebe998e2bad125a server_cookie=68afce30b9561f25 in_seq=0 out_seq=0 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.331+0000 7ff2a7e00640 10 client.25697 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2a7e00640 1 -- 172.21.15.71:0/675584656 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558f6c48830 con 0x5558f6cf0300 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2a7e00640 10 client.25697 handle_command_reply: tid=0 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 client.25697 shutdown 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 msgr2=0x5558f6cddb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6cddb30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac0ece50 tx=0x7ff2ac005c90 comp rx=0 tx=0).stop 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 shutdown_connections 2024-09-05T12:30:01.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6cf0300 0x5558f6cf27c0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6ce0570 0x5558f6cde070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd5cd0 0x5558f6cddb30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/675584656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6cdd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 >> 172.21.15.71:0/675584656 conn(0x5558f6cd1920 msgr2=0x5558f6cd7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2a7e00640 0 client.25697 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.332+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 shutdown_connections 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.333+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/675584656 wait complete. 2024-09-05T12:30:01.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.333+0000 7ff2cc3d9740 20 client.25697 trim_cache size 0 max 16384 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.347+0000 7ff2cc3d9740 1 Processor -- start 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.347+0000 7ff2cc3d9740 1 -- start start 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd53b0 0x5558f6cddcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cf80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 0x5558f6cf8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c41b0e60 con 0x5558f6ce0570 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c41b2a60 con 0x5558f6cd53b0 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c41b1530 con 0x5558f6cd5cd0 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 0x5558f6cf8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 0x5558f6cf8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60772/0 (socket says 172.21.15.71:60772) 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 -- 172.21.15.71:0/4197752993 learned_addr learned my addr 172.21.15.71:0/4197752993 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:01.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd53b0 0x5558f6cddcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cf80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 -- 172.21.15.71:0/4197752993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 msgr2=0x5558f6cf80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cf80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 -- 172.21.15.71:0/4197752993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd53b0 msgr2=0x5558f6cddcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd53b0 0x5558f6cddcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c3400640 1 -- 172.21.15.71:0/4197752993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558f6cd5130 con 0x5558f6ce0570 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.348+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd53b0 0x5558f6cddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2c3400640 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 0x5558f6cf8610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc003cc0 tx=0x7ff2bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61f02f1837c3ce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2b2a00640 1 -- 172.21.15.71:0/4197752993 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc003cf0 con 0x5558f6ce0570 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2b2a00640 1 -- 172.21.15.71:0/4197752993 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2bc00faa0 con 0x5558f6ce0570 2024-09-05T12:30:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2b2a00640 1 -- 172.21.15.71:0/4197752993 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2bc029b00 con 0x5558f6ce0570 2024-09-05T12:30:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/4197752993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 msgr2=0x5558f6cf8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 0x5558f6cf8610 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc003cc0 tx=0x7ff2bc017000 comp rx=0 tx=0).stop 2024-09-05T12:30:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/4197752993 shutdown_connections 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6ce0570 0x5558f6cf8610 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cf80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.349+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/4197752993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6cd53b0 0x5558f6cddcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/4197752993 >> 172.21.15.71:0/4197752993 conn(0x5558f6cd1920 msgr2=0x5558f6cdd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/4197752993 shutdown_connections 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/4197752993 wait complete. 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 -- start start 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 Processor -- start 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6ce87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cda1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.350+0000 7ff2cc3d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6ce0570 0x5558f6cda720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c41b22d0 con 0x5558f6cd53b0 2024-09-05T12:30:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c41b1c00 con 0x5558f6ce0570 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2cc3d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c41aee40 con 0x5558f6cd5cd0 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2cc3d9740 10 client.0 _ll_register_callbacks cb 0x7ff2ac0f3270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6ce87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6ce87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60780/0 (socket says 172.21.15.71:60780) 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 -- 172.21.15.71:0/2766401264 learned_addr learned my addr 172.21.15.71:0/2766401264 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c3400640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6ce0570 0x5558f6cda720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 -- 172.21.15.71:0/2766401264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 msgr2=0x5558f6cda1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cda1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 -- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6ce0570 msgr2=0x5558f6cda720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6ce0570 0x5558f6cda720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 -- 172.21.15.71:0/2766401264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558f6cdad60 con 0x5558f6cd53b0 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c3400640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6ce0570 0x5558f6cda720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:30:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2c2a00640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6ce87d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4004b70 tx=0x7ff2b40045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce94d371df6d993 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.351+0000 7ff2a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2a7e00640 1 -- 172.21.15.71:0/2766401264 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4032020 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2a7e00640 1 -- 172.21.15.71:0/2766401264 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7ff2b40035f0 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2bc00faa0 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2a7e00640 1 -- 172.21.15.71:0/2766401264 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff2b403d050 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 10 client.15666 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558f6d5aec0 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2a7e00640 1 -- 172.21.15.71:0/2766401264 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2bc00faa0 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2a7e00640 1 -- 172.21.15.71:0/2766401264 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7ff2b402e640 con 0x5558f6cd53b0 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 10 client.15666 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 10 client.15666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 20 client.15666 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:01.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6d5b520 0x5558f6d5d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 4 client.15666 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:30:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.352+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558f6c42b80 con 0x5558f6d5b520 2024-09-05T12:30:01.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.368+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6d5b520 0x5558f6d5d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.380+0000 7ff2bbe00640 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6d5b520 0x5558f6d5d940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd43ce0fa817ec2 server_cookie=d13639cdb94444fd in_seq=0 out_seq=0 2024-09-05T12:30:01.381 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.380+0000 7ff2a7e00640 10 client.15666 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:01.711 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:01 smithi071 ceph-mon[29845]: pgmap v71: 129 pgs: 129 active+clean; 31 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-05T12:30:01.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:01 smithi123 ceph-mon[34321]: pgmap v71: 129 pgs: 129 active+clean; 31 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-05T12:30:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.984+0000 7ff2a7e00640 1 -- 172.21.15.71:0/2766401264 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558f6c42b80 con 0x5558f6d5b520 2024-09-05T12:30:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.984+0000 7ff2a7e00640 10 client.15666 handle_command_reply: tid=0 2024-09-05T12:30:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.984+0000 7ff2cc3d9740 1 client.15666 shutdown 2024-09-05T12:30:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.984+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 msgr2=0x5558f6ce87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.984+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6ce87d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff2b4004b70 tx=0x7ff2b40045f0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 shutdown_connections 2024-09-05T12:30:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5558f6d5b520 0x5558f6d5d940 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558f6ce0570 0x5558f6cda720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558f6cd5cd0 0x5558f6cda1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 --2- 172.21.15.71:0/2766401264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558f6cd53b0 0x5558f6ce87d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 >> 172.21.15.71:0/2766401264 conn(0x5558f6cd1920 msgr2=0x5558f6cdddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2a7e00640 0 client.15666 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 shutdown_connections 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 1 -- 172.21.15.71:0/2766401264 wait complete. 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.985+0000 7ff2cc3d9740 20 client.15666 trim_cache size 0 max 16384 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:30:01.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:30:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.986+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2a80a2cf0 msgr2=0x7ff2a80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.986+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2a80a2cf0 0x7ff2a80a51b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc012680 tx=0x7ff2bc009500 comp rx=0 tx=0).stop 2024-09-05T12:30:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.986+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 msgr2=0x7ff2c4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.986+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 0x7ff2c4074a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009230 tx=0x7ff2ac004040 comp rx=0 tx=0).stop 2024-09-05T12:30:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 shutdown_connections 2024-09-05T12:30:01.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2a80a2cf0 0x7ff2a80a51b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2c41078c0 0x7ff2c4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2c4102250 0x7ff2c4074a30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 --2- 172.21.15.71:0/3917353316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2c41018b0 0x7ff2c40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 >> 172.21.15.71:0/3917353316 conn(0x7ff2c40fd5d0 msgr2=0x7ff2c40fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 shutdown_connections 2024-09-05T12:30:01.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:01.987+0000 7ff2c3e00640 1 -- 172.21.15.71:0/3917353316 wait complete. 2024-09-05T12:30:02.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:01 smithi174 ceph-mon[34313]: pgmap v71: 129 pgs: 129 active+clean; 31 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 2.2 MiB/s wr, 183 op/s 2024-09-05T12:30:02.192 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.26 MB/sec warmup 58 sec latency 8774.782 ms 2024-09-05T12:30:03.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.14 MB/sec warmup 59 sec latency 9774.888 ms 2024-09-05T12:30:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:03 smithi123 ceph-mon[34321]: pgmap v72: 129 pgs: 129 active+clean; 28 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-05T12:30:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:03 smithi174 ceph-mon[34313]: pgmap v72: 129 pgs: 129 active+clean; 28 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-05T12:30:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:03 smithi071 ceph-mon[29845]: pgmap v72: 129 pgs: 129 active+clean; 28 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-05T12:30:04.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 7.02 MB/sec warmup 60 sec latency 10775.019 ms 2024-09-05T12:30:05.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 6.91 MB/sec warmup 61 sec latency 11775.185 ms 2024-09-05T12:30:05.235 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:30:05.621 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:30:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:05 smithi123 ceph-mon[34321]: pgmap v73: 129 pgs: 129 active+clean; 28 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-05T12:30:05.958 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:05 smithi071 ceph-mon[29845]: pgmap v73: 129 pgs: 129 active+clean; 28 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-05T12:30:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:05 smithi174 ceph-mon[34313]: pgmap v73: 129 pgs: 129 active+clean; 28 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-05T12:30:06.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 6.79 MB/sec warmup 62 sec latency 12775.277 ms 2024-09-05T12:30:06.277 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 -- 172.21.15.71:0/4053431600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26901049a0 msgr2=0x7f26901089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 --2- 172.21.15.71:0/4053431600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26901049a0 0x7f26901089f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f267c009900 tx=0x7f267c02a410 comp rx=0 tx=0).stop 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 -- 172.21.15.71:0/4053431600 shutdown_connections 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 --2- 172.21.15.71:0/4053431600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690109120 0x7f2690110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 --2- 172.21.15.71:0/4053431600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26901049a0 0x7f26901089f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 --2- 172.21.15.71:0/4053431600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2690103ff0 0x7f26901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 -- 172.21.15.71:0/4053431600 >> 172.21.15.71:0/4053431600 conn(0x7f26900ffa90 msgr2=0x7f2690101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 -- 172.21.15.71:0/4053431600 shutdown_connections 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.237+0000 7f2695e00640 1 -- 172.21.15.71:0/4053431600 wait complete. 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.238+0000 7f2695e00640 1 Processor -- start 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.238+0000 7f2695e00640 1 -- start start 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f2695e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 0x7f269019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f2695e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26901049a0 0x7f269019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:06.278 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f2695e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2690109120 0x7f269019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f2695e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2690114140 con 0x7f2690109120 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f2695e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2690113fc0 con 0x7f2690103ff0 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f2695e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26901142c0 con 0x7f26901049a0 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f268ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 0x7f269019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f268f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2690109120 0x7f269019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f268ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 0x7f269019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35520/0 (socket says 172.21.15.71:35520) 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f268ea00640 1 -- 172.21.15.71:0/147503078 learned_addr learned my addr 172.21.15.71:0/147503078 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.239+0000 7f268e000640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26901049a0 0x7f269019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268ea00640 1 -- 172.21.15.71:0/147503078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26901049a0 msgr2=0x7f269019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268ea00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26901049a0 0x7f269019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268ea00640 1 -- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2690109120 msgr2=0x7f269019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268ea00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2690109120 0x7f269019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268ea00640 1 -- 172.21.15.71:0/147503078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269019f5d0 con 0x7f2690103ff0 2024-09-05T12:30:06.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268f400640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2690109120 0x7f269019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f268ea00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 0x7f269019a620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f268000e9f0 tx=0x7f268000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268000cd80 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f269019f8c0 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2680005500 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680010690 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.240+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26901a71a0 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.241+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26901043d0 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.243+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f26800046c0 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.244+0000 7f2694a00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f26740a2d70 0x7f26740a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.245+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f26800cfd50 con 0x7f2690103ff0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.245+0000 7f268e000640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f26740a2d70 0x7f26740a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:06.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.245+0000 7f268e000640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f26740a2d70 0x7f26740a5230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f267c009230 tx=0x7f267c0058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:06.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.247+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f26800541b0 con 0x7f2690103ff0 2024-09-05T12:30:06.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.459+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f269010c3c0 con 0x7f2690103ff0 2024-09-05T12:30:06.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.460+0000 7f2694a00640 1 -- 172.21.15.71:0/147503078 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11975 (secure 0 0 0) 0x7f268009c5d0 con 0x7f2690103ff0 2024-09-05T12:30:06.461 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:30:06.462 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"btime":"2024-09-05T12:29:52:485449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:29:51.941852+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:30:06.462 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f26740a2d70 msgr2=0x7f26740a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f26740a2d70 0x7f26740a5230 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f267c009230 tx=0x7f267c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 msgr2=0x7f269019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 0x7f269019a620 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f268000e9f0 tx=0x7f268000edf0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 shutdown_connections 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f26740a2d70 0x7f26740a5230 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2690109120 0x7f269019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26901049a0 0x7f269019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 --2- 172.21.15.71:0/147503078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2690103ff0 0x7f269019a620 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 >> 172.21.15.71:0/147503078 conn(0x7f26900ffa90 msgr2=0x7f2690100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:06.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 shutdown_connections 2024-09-05T12:30:06.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:06.463+0000 7f2695e00640 1 -- 172.21.15.71:0/147503078 wait complete. 2024-09-05T12:30:06.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:06 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/147503078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:30:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:06 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/147503078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:30:07.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:06 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/147503078' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:30:07.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 6.69 MB/sec warmup 63 sec latency 13775.372 ms 2024-09-05T12:30:07.455 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 72401df4 --release --await' 2024-09-05T12:30:07.455 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 72401df4 --release --await 2024-09-05T12:30:07.822 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:30:07.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:07 smithi123 ceph-mon[34321]: pgmap v74: 129 pgs: 129 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-09-05T12:30:07.961 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:07 smithi071 ceph-mon[29845]: pgmap v74: 129 pgs: 129 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-09-05T12:30:08.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:07 smithi174 ceph-mon[34313]: pgmap v74: 129 pgs: 129 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-09-05T12:30:08.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74662 6.58 MB/sec warmup 64 sec latency 14775.469 ms 2024-09-05T12:30:08.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 -- 172.21.15.71:0/1845588081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 msgr2=0x7fa3c8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 --2- 172.21.15.71:0/1845588081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c8076b00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009900 tx=0x7fa3bc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 -- 172.21.15.71:0/1845588081 shutdown_connections 2024-09-05T12:30:08.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 --2- 172.21.15.71:0/1845588081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c8112870 0x7fa3c8114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 --2- 172.21.15.71:0/1845588081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 0x7fa3c8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 --2- 172.21.15.71:0/1845588081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c8076b00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 -- 172.21.15.71:0/1845588081 >> 172.21.15.71:0/1845588081 conn(0x7fa3c80ff8d0 msgr2=0x7fa3c8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 -- 172.21.15.71:0/1845588081 shutdown_connections 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.446+0000 7fa3cf000640 1 -- 172.21.15.71:0/1845588081 wait complete. 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.447+0000 7fa3cf000640 1 Processor -- start 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.447+0000 7fa3cf000640 1 -- start start 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cf000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c81a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cf000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 0x7fa3c81a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cf000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c8112870 0x7fa3c819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cf000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8118430 con 0x7fa3c8076720 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cf000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81182b0 con 0x7fa3c8112870 2024-09-05T12:30:08.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cf000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81185b0 con 0x7fa3c8077040 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c81a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cd200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 0x7fa3c81a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c81a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38336/0 (socket says 172.21.15.71:38336) 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cdc00640 1 -- 172.21.15.71:0/3828684564 learned_addr learned my addr 172.21.15.71:0/3828684564 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3ce600640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c8112870 0x7fa3c819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cd200640 1 -- 172.21.15.71:0/3828684564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c8112870 msgr2=0x7fa3c819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cd200640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c8112870 0x7fa3c819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cd200640 1 -- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 msgr2=0x7fa3c81a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cd200640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c81a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.448+0000 7fa3cd200640 1 -- 172.21.15.71:0/3828684564 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c819efd0 con 0x7fa3c8077040 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.449+0000 7fa3cd200640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 0x7fa3c81a4bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400e9f0 tx=0x7fa3b400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.449+0000 7fa3c6000640 1 -- 172.21.15.71:0/3828684564 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400ccc0 con 0x7fa3c8077040 2024-09-05T12:30:08.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.449+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c819f2c0 con 0x7fa3c8077040 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.449+0000 7fa3c6000640 1 -- 172.21.15.71:0/3828684564 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b4010910 con 0x7fa3c8077040 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.449+0000 7fa3c6000640 1 -- 172.21.15.71:0/3828684564 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4018960 con 0x7fa3c8077040 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.449+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c81ab610 con 0x7fa3c8077040 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.452+0000 7fa3d30a4740 1 Processor -- start 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.452+0000 7fa3d30a4740 1 -- start start 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.452+0000 7fa3c6000640 1 -- 172.21.15.71:0/3828684564 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa3b4010ac0 con 0x7fa3c8077040 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d58900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d63700 0x563225d65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563225cbe650 con 0x563225d63700 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563225c65f10 con 0x563225d58520 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563225cba610 con 0x563225d58e40 2024-09-05T12:30:08.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d63700 0x563225d65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53690/0 (socket says 172.21.15.71:53690) 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1566723430 learned_addr learned my addr 172.21.15.71:0/1566723430 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3ce600640 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d58900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1566723430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 msgr2=0x563225d58900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d58900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1566723430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d63700 msgr2=0x563225d65af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d63700 0x563225d65af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.453+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1566723430 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632259cd580 con 0x563225d58e40 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.454+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d631c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005dd0 tx=0x7fa3bc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a83ba3d326150dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.454+0000 7fa3baa00640 1 -- 172.21.15.71:0/1566723430 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc040070 con 0x563225d58e40 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.454+0000 7fa3baa00640 1 -- 172.21.15.71:0/1566723430 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc004400 con 0x563225d58e40 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.454+0000 7fa3baa00640 1 -- 172.21.15.71:0/1566723430 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc035040 con 0x563225d58e40 2024-09-05T12:30:08.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.454+0000 7fa3c6000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa3b00a2cf0 0x7fa3b00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.454+0000 7fa3c6000640 1 -- 172.21.15.71:0/3828684564 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fa3b40cfb80 con 0x7fa3c8077040 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3ce600640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa3b00a2cf0 0x7fa3b00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1566723430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 msgr2=0x563225d631c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d631c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc005dd0 tx=0x7fa3bc005c00 comp rx=0 tx=0).stop 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1566723430 shutdown_connections 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d63700 0x563225d65af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d631c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1566723430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d58900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1566723430 >> 172.21.15.71:0/1566723430 conn(0x563225d54ab0 msgr2=0x563225d54e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1566723430 shutdown_connections 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1566723430 wait complete. 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3ce600640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa3b00a2cf0 0x7fa3b00a51b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3c806b3d0 tx=0x7fa3c0009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:08.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.455+0000 7fa3d30a4740 1 -- start start 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 Processor -- start 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 0x563225d60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225d610f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563225d66270 con 0x563225d58e40 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563225c79350 con 0x563225d63700 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563225d663f0 con 0x563225d58520 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3d30a4740 10 client.0 _ll_register_callbacks cb 0x563225c9c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3cd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 0x563225d60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 0x563225d60670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53692/0 (socket says 172.21.15.71:53692) 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3ce600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225d610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.456+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1285289426 learned_addr learned my addr 172.21.15.71:0/1285289426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 msgr2=0x563225d610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225d610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1285289426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 msgr2=0x563225d60bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d60bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3cdc00640 1 -- 172.21.15.71:0/1285289426 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563225a97960 con 0x563225d58520 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 0x563225d60670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0045b0 tx=0x7fa3bc0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fd2421b8aca1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3af400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3af400640 1 -- 172.21.15.71:0/1285289426 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc040070 con 0x563225d58520 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3bc004400 con 0x563225d58520 2024-09-05T12:30:08.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.457+0000 7fa3af400640 1 -- 172.21.15.71:0/1285289426 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fa3bc005230 con 0x563225d58520 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3af400640 1 -- 172.21.15.71:0/1285289426 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa3bc056050 con 0x563225d58520 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 10 client.25732 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563225d66270 con 0x563225d58520 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3af400640 1 -- 172.21.15.71:0/1285289426 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc004400 con 0x563225d58520 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3af400640 1 -- 172.21.15.71:0/1285289426 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa3bc031cc0 con 0x563225d58520 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 10 client.25732 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 10 client.25732 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 20 client.25732 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225d73080 0x563225d75470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 4 client.25732 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.458+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563225ccbbc0 con 0x563225d73080 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.459+0000 7fa3cd200640 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225d73080 0x563225d75470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.459+0000 7fa3cd200640 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225d73080 0x563225d75470 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6101cc48760f908f server_cookie=695ed2d47b4d6341 in_seq=0 out_seq=0 2024-09-05T12:30:08.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.459+0000 7fa3af400640 10 client.25732 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3af400640 1 -- 172.21.15.71:0/1285289426 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563225ccbbc0 con 0x563225d73080 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3af400640 10 client.25732 handle_command_reply: tid=0 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 client.25732 shutdown 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 msgr2=0x563225d60670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 0x563225d60670 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc0045b0 tx=0x7fa3bc0045e0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 shutdown_connections 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225d73080 0x563225d75470 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225d610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1285289426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58520 0x563225d60670 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3af400640 0 client.25732 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:08.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 >> 172.21.15.71:0/1285289426 conn(0x563225d54ab0 msgr2=0x563225d53cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:08.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 shutdown_connections 2024-09-05T12:30:08.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1285289426 wait complete. 2024-09-05T12:30:08.481 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.460+0000 7fa3d30a4740 20 client.25732 trim_cache size 0 max 16384 2024-09-05T12:30:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.478+0000 7fa3d30a4740 1 Processor -- start 2024-09-05T12:30:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.478+0000 7fa3d30a4740 1 -- start start 2024-09-05T12:30:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d53b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d7ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d63700 0x563225d7b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81b5350 con 0x563225d58e40 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81b6f50 con 0x563225d58520 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81b5a20 con 0x563225d63700 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3cd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d7ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3cd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d7ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38360/0 (socket says 172.21.15.71:38360) 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3cd200640 1 -- 172.21.15.71:0/1942727726 learned_addr learned my addr 172.21.15.71:0/1942727726 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d53b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.479+0000 7fa3ce600640 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d63700 0x563225d7b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3cd200640 1 -- 172.21.15.71:0/1942727726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d63700 msgr2=0x563225d7b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3cd200640 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d63700 0x563225d7b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3cd200640 1 -- 172.21.15.71:0/1942727726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 msgr2=0x563225d53b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3cd200640 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d53b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3cd200640 1 -- 172.21.15.71:0/1942727726 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563225d75a30 con 0x563225d58e40 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3cd200640 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d7ac60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c6a0 tx=0x7fa3b40d1fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a195dffe6a593e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3ba000640 1 -- 172.21.15.71:0/1942727726 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b409bb80 con 0x563225d58e40 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3ba000640 1 -- 172.21.15.71:0/1942727726 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b409bdb0 con 0x563225d58e40 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.480+0000 7fa3ba000640 1 -- 172.21.15.71:0/1942727726 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b409b710 con 0x563225d58e40 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1942727726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 msgr2=0x563225d7ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d7ac60 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400c6a0 tx=0x7fa3b40d1fd0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1942727726 shutdown_connections 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d63700 0x563225d7b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58e40 0x563225d7ac60 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/1942727726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d58520 0x563225d53b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1942727726 >> 172.21.15.71:0/1942727726 conn(0x563225d54ab0 msgr2=0x563225d625e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1942727726 shutdown_connections 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 -- 172.21.15.71:0/1942727726 wait complete. 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.481+0000 7fa3d30a4740 1 -- start start 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 Processor -- start 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58520 0x563225d67b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d68040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225ddcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81b67c0 con 0x563225d58520 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81b60f0 con 0x563225d63700 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c81b3330 con 0x563225d58e40 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3d30a4740 10 client.0 _ll_register_callbacks cb 0x7fa3b40d3500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:08.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58520 0x563225d67b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3cdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58520 0x563225d67b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38374/0 (socket says 172.21.15.71:38374) 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3ce600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225ddcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3cdc00640 1 -- 172.21.15.71:0/3067556290 learned_addr learned my addr 172.21.15.71:0/3067556290 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.482+0000 7fa3cd200640 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d68040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3ce600640 1 -- 172.21.15.71:0/3067556290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 msgr2=0x563225d68040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3ce600640 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d68040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3ce600640 1 -- 172.21.15.71:0/3067556290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58520 msgr2=0x563225d67b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3ce600640 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58520 0x563225d67b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3ce600640 1 -- 172.21.15.71:0/3067556290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563225ddd170 con 0x563225d63700 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3ce600640 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225ddcc30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0005fd0 tx=0x7fa3c0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeac5f2bb289856f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3af400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3af400640 1 -- 172.21.15.71:0/3067556290 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c002e060 con 0x563225d63700 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3af400640 1 -- 172.21.15.71:0/3067556290 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fa3c000f890 con 0x563225d63700 2024-09-05T12:30:08.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.483+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b409bdb0 con 0x563225d63700 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3af400640 1 -- 172.21.15.71:0/3067556290 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa3c0035050 con 0x563225d63700 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 10 client.25727 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563225dde4e0 con 0x563225d63700 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3af400640 1 -- 172.21.15.71:0/3067556290 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3b409bdb0 con 0x563225d63700 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3af400640 1 -- 172.21.15.71:0/3067556290 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fa3c001ecc0 con 0x563225d63700 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 10 client.25727 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 10 client.25727 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 20 client.25727 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225dded20 0x563225de1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:08.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 4 client.25727 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "72401df4", "release": true, "await": true}] 2024-09-05T12:30:08.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.484+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "72401df4", "release": true, "await": true}) -- 0x563225cc60d0 con 0x563225dded20 2024-09-05T12:30:08.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.485+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225dded20 0x563225de1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:08.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.485+0000 7fa3cdc00640 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225dded20 0x563225de1140 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3397693ee4a0616e server_cookie=ee587e352e2d60f0 in_seq=0 out_seq=0 2024-09-05T12:30:08.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.485+0000 7fa3af400640 10 client.25727 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:08.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.494+0000 7fa3af400640 1 -- 172.21.15.71:0/3067556290 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563225cc60d0 con 0x563225dded20 2024-09-05T12:30:08.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.494+0000 7fa3af400640 10 client.25727 handle_command_reply: tid=0 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.494+0000 7fa3d30a4740 1 client.25727 shutdown 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.494+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 msgr2=0x563225ddcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.494+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225ddcc30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0005fd0 tx=0x7fa3c0004a40 comp rx=0 tx=0).stop 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 shutdown_connections 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563225dded20 0x563225de1140 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563225d63700 0x563225ddcc30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563225d58e40 0x563225d68040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 --2- 172.21.15.71:0/3067556290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563225d58520 0x563225d67b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3af400640 0 client.25727 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:08.496 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 >> 172.21.15.71:0/3067556290 conn(0x563225d54ab0 msgr2=0x563225d61cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:08.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 shutdown_connections 2024-09-05T12:30:08.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 1 -- 172.21.15.71:0/3067556290 wait complete. 2024-09-05T12:30:08.497 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.495+0000 7fa3d30a4740 20 client.25727 trim_cache size 0 max 16384 2024-09-05T12:30:08.497 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 32, 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 12, 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "72401df4": { 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 12, 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:30:08.498 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 137.7, 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:30:08.499 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:30:08.500 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:30:08.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa3b00a2cf0 msgr2=0x7fa3b00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa3b00a2cf0 0x7fa3b00a51b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa3c806b3d0 tx=0x7fa3c0009500 comp rx=0 tx=0).stop 2024-09-05T12:30:08.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 msgr2=0x7fa3c81a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:08.500 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 0x7fa3c81a4bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400e9f0 tx=0x7fa3b400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 shutdown_connections 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3c8112870 0x7fa3c819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3c8077040 0x7fa3c81a4bd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa3b00a2cf0 0x7fa3b00a51b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 --2- 172.21.15.71:0/3828684564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8076720 0x7fa3c81a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 >> 172.21.15.71:0/3828684564 conn(0x7fa3c80ff8d0 msgr2=0x7fa3c8101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.496+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 shutdown_connections 2024-09-05T12:30:08.501 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:30:08.497+0000 7fa3cf000640 1 -- 172.21.15.71:0/3828684564 wait complete. 2024-09-05T12:30:09.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 74738 6.50 MB/sec warmup 65 sec latency 15079.429 ms 2024-09-05T12:30:09.460 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '72401df4', total seconds elapsed: 21.9 2024-09-05T12:30:10.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:09 smithi174 ceph-mon[34313]: pgmap v75: 129 pgs: 129 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-05T12:30:10.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:09 smithi071 ceph-mon[29845]: pgmap v75: 129 pgs: 129 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-05T12:30:10.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:09 smithi123 ceph-mon[34321]: pgmap v75: 129 pgs: 129 active+clean; 23 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.2 MiB/s wr, 124 op/s 2024-09-05T12:30:10.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 76075 6.48 MB/sec warmup 66 sec latency 42.524 ms 2024-09-05T12:30:11.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78095 6.57 MB/sec warmup 67 sec latency 355.102 ms 2024-09-05T12:30:12.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:11 smithi174 ceph-mon[34313]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-09-05T12:30:12.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:11 smithi071 ceph-mon[29845]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-09-05T12:30:12.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:11 smithi123 ceph-mon[34321]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-09-05T12:30:12.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78095 6.47 MB/sec warmup 68 sec latency 1355.207 ms 2024-09-05T12:30:13.193 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78095 6.38 MB/sec warmup 69 sec latency 2355.313 ms 2024-09-05T12:30:14.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:13 smithi174 ceph-mon[34313]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 950 KiB/s wr, 137 op/s 2024-09-05T12:30:14.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:13 smithi071 ceph-mon[29845]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 950 KiB/s wr, 137 op/s 2024-09-05T12:30:14.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:13 smithi123 ceph-mon[34321]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 950 KiB/s wr, 137 op/s 2024-09-05T12:30:14.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78095 6.29 MB/sec warmup 70 sec latency 3355.400 ms 2024-09-05T12:30:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 -- 172.21.15.71:0/373366060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408107b80 msgr2=0x7f7408107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 --2- 172.21.15.71:0/373366060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408107b80 0x7f7408107f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009890 tx=0x7f73f802a370 comp rx=0 tx=0).stop 2024-09-05T12:30:15.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 -- 172.21.15.71:0/373366060 shutdown_connections 2024-09-05T12:30:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 --2- 172.21.15.71:0/373366060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408102480 0x7f740810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 --2- 172.21.15.71:0/373366060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408101ae0 0x7f7408101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 --2- 172.21.15.71:0/373366060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408107b80 0x7f7408107f60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 -- 172.21.15.71:0/373366060 >> 172.21.15.71:0/373366060 conn(0x7f74080fd7e0 msgr2=0x7f74080ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 -- 172.21.15.71:0/373366060 shutdown_connections 2024-09-05T12:30:15.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.114+0000 7f740e600640 1 -- 172.21.15.71:0/373366060 wait complete. 2024-09-05T12:30:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 Processor -- start 2024-09-05T12:30:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 -- start start 2024-09-05T12:30:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408101ae0 0x7f7408198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 0x7f74081988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408107b80 0x7f740819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111f30 con 0x7f7408107b80 2024-09-05T12:30:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7408111db0 con 0x7f7408102480 2024-09-05T12:30:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.115+0000 7f740e600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74081120b0 con 0x7f7408101ae0 2024-09-05T12:30:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f7407e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408107b80 0x7f740819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 0x7f74081988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 0x7f74081988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:43830/0 (socket says 172.21.15.71:43830) 2024-09-05T12:30:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 -- 172.21.15.71:0/1088638674 learned_addr learned my addr 172.21.15.71:0/1088638674 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f7407400640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408101ae0 0x7f7408198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 -- 172.21.15.71:0/1088638674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408101ae0 msgr2=0x7f7408198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408101ae0 0x7f7408198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 -- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408107b80 msgr2=0x7f740819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408107b80 0x7f740819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 -- 172.21.15.71:0/1088638674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740819d370 con 0x7f7408102480 2024-09-05T12:30:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f7407400640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408101ae0 0x7f7408198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:30:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.116+0000 7f73ffe00640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 0x7f74081988c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73f000e9f0 tx=0x7f73f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.117+0000 7f7404c00640 1 -- 172.21.15.71:0/1088638674 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f000cd80 con 0x7f7408102480 2024-09-05T12:30:15.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.117+0000 7f7404c00640 1 -- 172.21.15.71:0/1088638674 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0010910 con 0x7f7408102480 2024-09-05T12:30:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.117+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740819d660 con 0x7f7408102480 2024-09-05T12:30:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.117+0000 7f7404c00640 1 -- 172.21.15.71:0/1088638674 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0018990 con 0x7f7408102480 2024-09-05T12:30:15.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.117+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081a4ef0 con 0x7f7408102480 2024-09-05T12:30:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 Processor -- start 2024-09-05T12:30:15.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f7404c00640 1 -- 172.21.15.71:0/1088638674 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f73f0010ac0 con 0x7f7408102480 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 -- start start 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532a0e0 0x55e18532a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185339270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1853397b0 0x55e18533bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e185290100 con 0x55e18532aa00 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e1852381b0 con 0x55e1853397b0 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f74114b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e18528c0c0 con 0x55e18532a0e0 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.120+0000 7f7407400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185339270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1853397b0 0x55e18533bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185339270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41870/0 (socket says 172.21.15.71:41870) 2024-09-05T12:30:15.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 -- 172.21.15.71:0/63201200 learned_addr learned my addr 172.21.15.71:0/63201200 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407e00640 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532a0e0 0x55e18532a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 -- 172.21.15.71:0/63201200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532a0e0 msgr2=0x55e18532a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532a0e0 0x55e18532a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 -- 172.21.15.71:0/63201200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1853397b0 msgr2=0x55e18533bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1853397b0 0x55e18533bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 -- 172.21.15.71:0/63201200 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e184fa3210 con 0x55e18532aa00 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7407400640 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185339270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73f8035040 tx=0x7f73f8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bbae01d24b43f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.121+0000 7f7404c00640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2e20 0x7f73ec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f7404c00640 1 -- 172.21.15.71:0/1088638674 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f73f00cfa60 con 0x7f7408102480 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f73fcc00640 1 -- 172.21.15.71:0/63201200 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8037040 con 0x55e18532aa00 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f7407e00640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2e20 0x7f73ec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f73fcc00640 1 -- 172.21.15.71:0/63201200 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f8030780 con 0x55e18532aa00 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f73fcc00640 1 -- 172.21.15.71:0/63201200 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8003d80 con 0x55e18532aa00 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 -- 172.21.15.71:0/63201200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 msgr2=0x55e185339270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185339270 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73f8035040 tx=0x7f73f8009510 comp rx=0 tx=0).stop 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 -- 172.21.15.71:0/63201200 shutdown_connections 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1853397b0 0x55e18533bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185339270 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 --2- 172.21.15.71:0/63201200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532a0e0 0x55e18532a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f7407e00640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2e20 0x7f73ec0a52e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7400005fd0 tx=0x7f7400010040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.122+0000 7f74114b9740 1 -- 172.21.15.71:0/63201200 >> 172.21.15.71:0/63201200 conn(0x55e185326750 msgr2=0x55e185326b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.123+0000 7f74114b9740 1 -- 172.21.15.71:0/63201200 shutdown_connections 2024-09-05T12:30:15.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.123+0000 7f74114b9740 1 -- 172.21.15.71:0/63201200 wait complete. 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.123+0000 7f74114b9740 1 -- start start 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.123+0000 7f74114b9740 1 Processor -- start 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185336680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185336bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e185337100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e18533c380 con 0x55e18532aa00 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e185219090 con 0x55e18532a0e0 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e18533c500 con 0x55e1853397b0 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f74114b9740 10 client.0 _ll_register_callbacks cb 0x55e18526df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185336680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185336bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185336680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:43850/0 (socket says 172.21.15.71:43850) 2024-09-05T12:30:15.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 -- 172.21.15.71:0/413104456 learned_addr learned my addr 172.21.15.71:0/413104456 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407e00640 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e185337100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 -- 172.21.15.71:0/413104456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 msgr2=0x55e185337100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e185337100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 -- 172.21.15.71:0/413104456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 msgr2=0x55e185336bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185336bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.124+0000 7f7407400640 1 -- 172.21.15.71:0/413104456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18506f620 con 0x55e18532a0e0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f7407400640 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185336680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73f8005b60 tx=0x7f73f80039a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9f009bd94765ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f73f5600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f73f5600640 1 -- 172.21.15.71:0/413104456 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8037040 con 0x55e18532a0e0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73f8030780 con 0x55e18532a0e0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f73f5600640 1 -- 172.21.15.71:0/413104456 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f73f8003d20 con 0x55e18532a0e0 2024-09-05T12:30:15.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f73f5600640 1 -- 172.21.15.71:0/413104456 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73f8003f20 con 0x55e18532a0e0 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f74114b9740 10 client.25739 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e185348020 con 0x55e18532a0e0 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.125+0000 7f73f5600640 1 -- 172.21.15.71:0/413104456 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f8030780 con 0x55e18532a0e0 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f73f5600640 1 -- 172.21.15.71:0/413104456 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f73f804ebf0 con 0x55e18532a0e0 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f74114b9740 10 client.25739 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f74114b9740 10 client.25739 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f74114b9740 20 client.25739 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f74114b9740 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e185349370 0x55e18534b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f74114b9740 4 client.25739 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e18529d830 con 0x55e185349370 2024-09-05T12:30:15.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.126+0000 7f73ffe00640 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e185349370 0x55e18534b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.127+0000 7f73f5600640 10 client.25739 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.127+0000 7f73ffe00640 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e185349370 0x55e18534b760 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04fe4f6d9455671 server_cookie=fd5a01d24af7145c in_seq=0 out_seq=0 2024-09-05T12:30:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f73f5600640 1 -- 172.21.15.71:0/413104456 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e18529d830 con 0x55e185349370 2024-09-05T12:30:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f73f5600640 10 client.25739 handle_command_reply: tid=0 2024-09-05T12:30:15.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 client.25739 shutdown 2024-09-05T12:30:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 msgr2=0x55e185336680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185336680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73f8005b60 tx=0x7f73f80039a0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 shutdown_connections 2024-09-05T12:30:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e185349370 0x55e18534b760 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e185337100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532aa00 0x55e185336bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f73f5600640 0 client.25739 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 --2- 172.21.15.71:0/413104456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185336680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 >> 172.21.15.71:0/413104456 conn(0x55e185326750 msgr2=0x55e18532c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 shutdown_connections 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 1 -- 172.21.15.71:0/413104456 wait complete. 2024-09-05T12:30:15.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.128+0000 7f74114b9740 20 client.25739 trim_cache size 0 max 16384 2024-09-05T12:30:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.143+0000 7f74114b9740 1 Processor -- start 2024-09-05T12:30:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.143+0000 7f74114b9740 1 -- start start 2024-09-05T12:30:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.143+0000 7f74114b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185338de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f74114b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532aa00 0x55e1853358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f74114b9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 0x55e185350df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f74114b9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74081acba0 con 0x55e1853397b0 2024-09-05T12:30:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f74114b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74081ae7a0 con 0x55e18532a0e0 2024-09-05T12:30:15.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f74114b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74081ad270 con 0x55e18532aa00 2024-09-05T12:30:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 0x55e185350df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 0x55e185350df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41886/0 (socket says 172.21.15.71:41886) 2024-09-05T12:30:15.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 -- 172.21.15.71:0/139314320 learned_addr learned my addr 172.21.15.71:0/139314320 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407400640 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185338de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f73ffe00640 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532aa00 0x55e1853358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 -- 172.21.15.71:0/139314320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532aa00 msgr2=0x55e1853358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532aa00 0x55e1853358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 -- 172.21.15.71:0/139314320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 msgr2=0x55e185338de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185338de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f7407e00640 1 -- 172.21.15.71:0/139314320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e185331440 con 0x55e1853397b0 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.144+0000 7f73ffe00640 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532aa00 0x55e1853358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.145+0000 7f7407e00640 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 0x55e185350df0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7400003d70 tx=0x7f7400016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4eb0436af1a75bb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.145+0000 7f73f7e00640 1 -- 172.21.15.71:0/139314320 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400003e60 con 0x55e1853397b0 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.145+0000 7f73f7e00640 1 -- 172.21.15.71:0/139314320 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740002b070 con 0x55e1853397b0 2024-09-05T12:30:15.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.145+0000 7f73f7e00640 1 -- 172.21.15.71:0/139314320 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740001ad60 con 0x55e1853397b0 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 -- 172.21.15.71:0/139314320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 msgr2=0x55e185350df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 0x55e185350df0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7400003d70 tx=0x7f7400016000 comp rx=0 tx=0).stop 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 -- 172.21.15.71:0/139314320 shutdown_connections 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1853397b0 0x55e185350df0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18532aa00 0x55e1853358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 --2- 172.21.15.71:0/139314320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532a0e0 0x55e185338de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 -- 172.21.15.71:0/139314320 >> 172.21.15.71:0/139314320 conn(0x55e185326750 msgr2=0x55e185336e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 -- 172.21.15.71:0/139314320 shutdown_connections 2024-09-05T12:30:15.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 -- 172.21.15.71:0/139314320 wait complete. 2024-09-05T12:30:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 -- start start 2024-09-05T12:30:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.146+0000 7f74114b9740 1 Processor -- start 2024-09-05T12:30:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 0x55e1853ac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.152 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532aa00 0x55e1853ac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e18533cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74081ae010 con 0x55e18532a0e0 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74081ad940 con 0x55e18532aa00 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74081aab80 con 0x55e1853397b0 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f74114b9740 10 client.0 _ll_register_callbacks cb 0x7f73f00d33e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 0x55e1853ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 0x55e1853ac130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41902/0 (socket says 172.21.15.71:41902) 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 -- 172.21.15.71:0/249859784 learned_addr learned my addr 172.21.15.71:0/249859784 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f73ffe00640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532aa00 0x55e1853ac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407e00640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e18533cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 -- 172.21.15.71:0/249859784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 msgr2=0x55e18533cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e18533cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 -- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532aa00 msgr2=0x55e1853ac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532aa00 0x55e1853ac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.147+0000 7f7407400640 1 -- 172.21.15.71:0/249859784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18533d5e0 con 0x55e18532a0e0 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f7407400640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 0x55e1853ac130 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73f8030520 tx=0x7f73f8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa87bdb0e1523b48 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f73f5600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:30:15.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f73f5600640 1 -- 172.21.15.71:0/249859784 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8037070 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f73f5600640 1 -- 172.21.15.71:0/249859784 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f73f803d600 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f740002b070 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f73f5600640 1 -- 172.21.15.71:0/249859784 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f73f8062050 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f74114b9740 10 client.15708 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e1853b3930 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.148+0000 7f73f5600640 1 -- 172.21.15.71:0/249859784 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f740002b070 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f73f5600640 1 -- 172.21.15.71:0/249859784 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f73f804e650 con 0x55e18532a0e0 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f74114b9740 10 client.15708 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f74114b9740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f74114b9740 20 client.15708 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f74114b9740 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e1853b40e0 0x55e1853b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f74114b9740 4 client.15708 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:30:15.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e185297b80 con 0x55e1853b40e0 2024-09-05T12:30:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f73ffe00640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e1853b40e0 0x55e1853b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f73ffe00640 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e1853b40e0 0x55e1853b6500 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe58d47efa773a7 server_cookie=4b5fbe46480c2625 in_seq=0 out_seq=0 2024-09-05T12:30:15.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:15.149+0000 7f73f5600640 10 client.15708 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:15.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78095 6.20 MB/sec warmup 71 sec latency 4355.475 ms 2024-09-05T12:30:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:15 smithi174 ceph-mon[34313]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 828 KiB/s wr, 120 op/s 2024-09-05T12:30:16.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:15 smithi071 ceph-mon[29845]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 828 KiB/s wr, 120 op/s 2024-09-05T12:30:16.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:15 smithi123 ceph-mon[34321]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 754 KiB/s rd, 828 KiB/s wr, 120 op/s 2024-09-05T12:30:16.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78227 6.12 MB/sec warmup 72 sec latency 4933.711 ms 2024-09-05T12:30:17.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78227 6.03 MB/sec warmup 73 sec latency 1342.108 ms 2024-09-05T12:30:17.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:17 smithi071 ceph-mon[29845]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 877 KiB/s wr, 148 op/s 2024-09-05T12:30:17.981 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:17 smithi123 ceph-mon[34321]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 877 KiB/s wr, 148 op/s 2024-09-05T12:30:18.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:17 smithi174 ceph-mon[34313]: pgmap v79: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 877 KiB/s rd, 877 KiB/s wr, 148 op/s 2024-09-05T12:30:18.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78227 5.95 MB/sec warmup 74 sec latency 2342.228 ms 2024-09-05T12:30:19.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78227 5.87 MB/sec warmup 75 sec latency 3342.312 ms 2024-09-05T12:30:19.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:19 smithi071 ceph-mon[29845]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 307 KiB/s wr, 78 op/s 2024-09-05T12:30:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:19 smithi174 ceph-mon[34313]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 307 KiB/s wr, 78 op/s 2024-09-05T12:30:20.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:19 smithi123 ceph-mon[34321]: pgmap v80: 129 pgs: 129 active+clean; 25 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 307 KiB/s wr, 78 op/s 2024-09-05T12:30:20.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78227 5.79 MB/sec warmup 76 sec latency 4342.393 ms 2024-09-05T12:30:21.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78246 5.72 MB/sec warmup 77 sec latency 4923.045 ms 2024-09-05T12:30:22.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.005+0000 7f73f5600640 1 -- 172.21.15.71:0/249859784 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e185297b80 con 0x55e1853b40e0 2024-09-05T12:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.005+0000 7f73f5600640 10 client.15708 handle_command_reply: tid=0 2024-09-05T12:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.005+0000 7f74114b9740 1 client.15708 shutdown 2024-09-05T12:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.005+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 msgr2=0x55e1853ac130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:22.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.005+0000 7f74114b9740 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 0x55e1853ac130 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73f8030520 tx=0x7f73f8003c20 comp rx=0 tx=0).stop 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 shutdown_connections 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e1853b40e0 0x55e1853b6500 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1853397b0 0x55e18533cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e18532aa00 0x55e1853ac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 --2- 172.21.15.71:0/249859784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e18532a0e0 0x55e1853ac130 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 >> 172.21.15.71:0/249859784 conn(0x55e185326750 msgr2=0x55e185344670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f73f5600640 0 client.15708 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 shutdown_connections 2024-09-05T12:30:22.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 1 -- 172.21.15.71:0/249859784 wait complete. 2024-09-05T12:30:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.006+0000 7f74114b9740 20 client.15708 trim_cache size 0 max 16384 2024-09-05T12:30:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:30:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:30:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:30:22.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:30:22.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:21 smithi174 ceph-mon[34313]: pgmap v81: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 321 KiB/s wr, 85 op/s 2024-09-05T12:30:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2e20 msgr2=0x7f73ec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2e20 0x7f73ec0a52e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7400005fd0 tx=0x7f7400010040 comp rx=0 tx=0).stop 2024-09-05T12:30:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 msgr2=0x7f74081988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:22.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 0x7f74081988c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73f000e9f0 tx=0x7f73f000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 shutdown_connections 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2e20 0x7f73ec0a52e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408107b80 0x7f740819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408102480 0x7f74081988c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 --2- 172.21.15.71:0/1088638674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408101ae0 0x7f7408198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 >> 172.21.15.71:0/1088638674 conn(0x7f74080fd7e0 msgr2=0x7f74080feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.008+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 shutdown_connections 2024-09-05T12:30:22.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:22.009+0000 7f740e600640 1 -- 172.21.15.71:0/1088638674 wait complete. 2024-09-05T12:30:22.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:21 smithi071 ceph-mon[29845]: pgmap v81: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 321 KiB/s wr, 85 op/s 2024-09-05T12:30:22.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:21 smithi123 ceph-mon[34321]: pgmap v81: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 321 KiB/s wr, 85 op/s 2024-09-05T12:30:22.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78246 5.65 MB/sec warmup 78 sec latency 1383.304 ms 2024-09-05T12:30:23.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78246 5.57 MB/sec warmup 79 sec latency 2383.416 ms 2024-09-05T12:30:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:23 smithi174 ceph-mon[34313]: pgmap v82: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 80 KiB/s wr, 39 op/s 2024-09-05T12:30:24.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:23 smithi071 ceph-mon[29845]: pgmap v82: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 80 KiB/s wr, 39 op/s 2024-09-05T12:30:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:23 smithi123 ceph-mon[34321]: pgmap v82: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 80 KiB/s wr, 39 op/s 2024-09-05T12:30:24.194 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78246 5.51 MB/sec warmup 80 sec latency 3383.518 ms 2024-09-05T12:30:25.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78246 5.44 MB/sec warmup 81 sec latency 4383.629 ms 2024-09-05T12:30:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:25 smithi174 ceph-mon[34313]: pgmap v83: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-09-05T12:30:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:25 smithi071 ceph-mon[29845]: pgmap v83: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-09-05T12:30:26.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:25 smithi123 ceph-mon[34321]: pgmap v83: 129 pgs: 129 active+clean; 25 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 64 KiB/s wr, 36 op/s 2024-09-05T12:30:26.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 78501 5.39 MB/sec warmup 82 sec latency 4962.316 ms 2024-09-05T12:30:27.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 80528 5.41 MB/sec warmup 83 sec latency 10.338 ms 2024-09-05T12:30:28.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:27 smithi174 ceph-mon[34313]: pgmap v84: 129 pgs: 129 active+clean; 29 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 374 KiB/s wr, 54 op/s 2024-09-05T12:30:28.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:27 smithi071 ceph-mon[29845]: pgmap v84: 129 pgs: 129 active+clean; 29 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 374 KiB/s wr, 54 op/s 2024-09-05T12:30:28.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:27 smithi123 ceph-mon[34321]: pgmap v84: 129 pgs: 129 active+clean; 29 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 374 KiB/s wr, 54 op/s 2024-09-05T12:30:28.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81638 5.47 MB/sec warmup 84 sec latency 540.429 ms 2024-09-05T12:30:29.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81638 5.40 MB/sec warmup 85 sec latency 1540.559 ms 2024-09-05T12:30:30.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:29 smithi174 ceph-mon[34313]: pgmap v85: 129 pgs: 129 active+clean; 29 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 325 KiB/s wr, 25 op/s 2024-09-05T12:30:30.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:29 smithi071 ceph-mon[29845]: pgmap v85: 129 pgs: 129 active+clean; 29 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 325 KiB/s wr, 25 op/s 2024-09-05T12:30:30.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:29 smithi123 ceph-mon[34321]: pgmap v85: 129 pgs: 129 active+clean; 29 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 325 KiB/s wr, 25 op/s 2024-09-05T12:30:30.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81638 5.34 MB/sec warmup 86 sec latency 2540.646 ms 2024-09-05T12:30:31.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81703 5.28 MB/sec warmup 87 sec latency 3119.841 ms 2024-09-05T12:30:32.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:31 smithi174 ceph-mon[34313]: pgmap v86: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 515 KiB/s wr, 64 op/s 2024-09-05T12:30:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:31 smithi071 ceph-mon[29845]: pgmap v86: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 515 KiB/s wr, 64 op/s 2024-09-05T12:30:32.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:31 smithi123 ceph-mon[34321]: pgmap v86: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 515 KiB/s wr, 64 op/s 2024-09-05T12:30:32.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81703 5.22 MB/sec warmup 88 sec latency 1404.088 ms 2024-09-05T12:30:33.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81703 5.16 MB/sec warmup 89 sec latency 2404.181 ms 2024-09-05T12:30:34.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:33 smithi174 ceph-mon[34313]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 524 KiB/s wr, 61 op/s 2024-09-05T12:30:34.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:33 smithi071 ceph-mon[29845]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 524 KiB/s wr, 61 op/s 2024-09-05T12:30:34.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:33 smithi123 ceph-mon[34321]: pgmap v87: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 524 KiB/s wr, 61 op/s 2024-09-05T12:30:34.195 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81703 5.10 MB/sec warmup 90 sec latency 3404.271 ms 2024-09-05T12:30:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 -- 172.21.15.71:0/1200506484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8112960 msgr2=0x7f1ab8114d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 --2- 172.21.15.71:0/1200506484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8112960 0x7f1ab8114d50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0009ca0 tx=0x7f1ab002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 -- 172.21.15.71:0/1200506484 shutdown_connections 2024-09-05T12:30:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 --2- 172.21.15.71:0/1200506484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8112960 0x7f1ab8114d50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 --2- 172.21.15.71:0/1200506484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab8077040 0x7f1ab8112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 --2- 172.21.15.71:0/1200506484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab8076720 0x7f1ab8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 -- 172.21.15.71:0/1200506484 >> 172.21.15.71:0/1200506484 conn(0x7f1ab80ff980 msgr2=0x7f1ab8101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 -- 172.21.15.71:0/1200506484 shutdown_connections 2024-09-05T12:30:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.139+0000 7f1abf200640 1 -- 172.21.15.71:0/1200506484 wait complete. 2024-09-05T12:30:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 Processor -- start 2024-09-05T12:30:35.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 -- start start 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 0x7f1ab81a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab8077040 0x7f1ab81a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab8112960 0x7f1ab819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81184c0 con 0x7f1ab8076720 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8118340 con 0x7f1ab8112960 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.140+0000 7f1abf200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8118640 con 0x7f1ab8077040 2024-09-05T12:30:35.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 0x7f1ab81a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 0x7f1ab81a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50958/0 (socket says 172.21.15.71:50958) 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 -- 172.21.15.71:0/670531041 learned_addr learned my addr 172.21.15.71:0/670531041 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7400640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab8077040 0x7f1ab81a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1abd200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab8112960 0x7f1ab819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 -- 172.21.15.71:0/670531041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab8077040 msgr2=0x7f1ab81a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab8077040 0x7f1ab81a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 -- 172.21.15.71:0/670531041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab8112960 msgr2=0x7f1ab819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab8112960 0x7f1ab819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 -- 172.21.15.71:0/670531041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab819ef80 con 0x7f1ab8076720 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.141+0000 7f1ab7e00640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 0x7f1ab81a4640 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800ea60 tx=0x7f1aa800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.142+0000 7f1ab4c00640 1 -- 172.21.15.71:0/670531041 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa800cde0 con 0x7f1ab8076720 2024-09-05T12:30:35.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.142+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab819f270 con 0x7f1ab8076720 2024-09-05T12:30:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.142+0000 7f1ab4c00640 1 -- 172.21.15.71:0/670531041 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa80035f0 con 0x7f1ab8076720 2024-09-05T12:30:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.142+0000 7f1ab4c00640 1 -- 172.21.15.71:0/670531041 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8010670 con 0x7f1ab8076720 2024-09-05T12:30:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.142+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab81ab490 con 0x7f1ab8076720 2024-09-05T12:30:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 Processor -- start 2024-09-05T12:30:35.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 -- start start 2024-09-05T12:30:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21ba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5590f2120100 con 0x5590f21ba1e0 2024-09-05T12:30:35.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5590f20c81b0 con 0x5590f21c53a0 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ac2254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590f211c0c0 con 0x5590f21bab00 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1ab4c00640 1 -- 172.21.15.71:0/670531041 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1aa80037a0 con 0x7f1ab8076720 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1abd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21ba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1abd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21ba5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50974/0 (socket says 172.21.15.71:50974) 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.146+0000 7f1abd200640 1 -- 172.21.15.71:0/3698561700 learned_addr learned my addr 172.21.15.71:0/3698561700 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1ab7400640 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1abd200640 1 -- 172.21.15.71:0/3698561700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 msgr2=0x5590f21c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1abd200640 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1abd200640 1 -- 172.21.15.71:0/3698561700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 msgr2=0x5590f21c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1abd200640 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1abd200640 1 -- 172.21.15.71:0/3698561700 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f1e33210 con 0x5590f21ba1e0 2024-09-05T12:30:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1ab7400640 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:30:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1abd200640 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21ba5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1ab806b1e0 tx=0x7f1ab0005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3693575aac4b8c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.147+0000 7f1aad600640 1 -- 172.21.15.71:0/3698561700 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0031db0 con 0x5590f21ba1e0 2024-09-05T12:30:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1aad600640 1 -- 172.21.15.71:0/3698561700 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab0036040 con 0x5590f21ba1e0 2024-09-05T12:30:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1aad600640 1 -- 172.21.15.71:0/3698561700 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00042a0 con 0x5590f21ba1e0 2024-09-05T12:30:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ab4c00640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1a9c0a2da0 0x7f1a9c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 -- 172.21.15.71:0/3698561700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 msgr2=0x5590f21ba5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21ba5c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f1ab806b1e0 tx=0x7f1ab0005f10 comp rx=0 tx=0).stop 2024-09-05T12:30:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ab4c00640 1 -- 172.21.15.71:0/670531041 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f1aa80cfae0 con 0x7f1ab8076720 2024-09-05T12:30:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ab7e00640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1a9c0a2da0 0x7f1a9c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 -- 172.21.15.71:0/3698561700 shutdown_connections 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3698561700 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21ba5c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.148+0000 7f1ac2254740 1 -- 172.21.15.71:0/3698561700 >> 172.21.15.71:0/3698561700 conn(0x5590f21b6750 msgr2=0x5590f21b6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.149+0000 7f1ac2254740 1 -- 172.21.15.71:0/3698561700 shutdown_connections 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.149+0000 7f1ac2254740 1 -- 172.21.15.71:0/3698561700 wait complete. 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.149+0000 7f1ab7e00640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1a9c0a2da0 0x7f1a9c0a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ccef0 tx=0x7f1aa809ba80 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.149+0000 7f1ac2254740 1 -- start start 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.149+0000 7f1ac2254740 1 Processor -- start 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5590f21c7df0 con 0x5590f21ba1e0 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5590f20a9090 con 0x5590f21c53a0 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590f21c7f70 con 0x5590f21bab00 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ac2254740 10 client.0 _ll_register_callbacks cb 0x5590f20fdf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21c2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50982/0 (socket says 172.21.15.71:50982) 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 -- 172.21.15.71:0/3645294589 learned_addr learned my addr 172.21.15.71:0/3645294589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1ab7400640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 -- 172.21.15.71:0/3645294589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 msgr2=0x5590f21c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 -- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 msgr2=0x5590f21c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.150+0000 7f1abd200640 1 -- 172.21.15.71:0/3645294589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f1eff620 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1abd200640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21c2280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0006010 tx=0x7f1ab00319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439bbf2a3dd90c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1aa2000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1aa2000640 1 -- 172.21.15.71:0/3645294589 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab0056060 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1aa2000640 1 -- 172.21.15.71:0/3645294589 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f1ab002f810 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ab0036040 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1aa2000640 1 -- 172.21.15.71:0/3645294589 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1ab004a040 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1ac2254740 10 client.15720 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590f21c7df0 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.151+0000 7f1aa2000640 1 -- 172.21.15.71:0/3645294589 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ab0036040 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1aa2000640 1 -- 172.21.15.71:0/3645294589 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1ab0050030 con 0x5590f21ba1e0 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ac2254740 10 client.15720 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ac2254740 10 client.15720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ac2254740 20 client.15720 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f21d4b70 0x5590f21d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ac2254740 4 client.15720 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590f212d830 con 0x5590f21d4b70 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.152+0000 7f1ab7400640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f21d4b70 0x5590f21d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.153+0000 7f1ab7400640 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f21d4b70 0x5590f21d6f90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=308de9c427a593c2 server_cookie=366ca2675282a38b in_seq=0 out_seq=0 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.153+0000 7f1aa2000640 10 client.15720 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1aa2000640 1 -- 172.21.15.71:0/3645294589 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590f212d830 con 0x5590f21d4b70 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1aa2000640 10 client.15720 handle_command_reply: tid=0 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 client.15720 shutdown 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 msgr2=0x5590f21c2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21c2280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0006010 tx=0x7f1ab00319e0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 shutdown_connections 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f21d4b70 0x5590f21d6f90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21c53a0 0x5590f21c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21bab00 0x5590f21c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 --2- 172.21.15.71:0/3645294589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21ba1e0 0x5590f21c2280 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1aa2000640 0 client.15720 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 >> 172.21.15.71:0/3645294589 conn(0x5590f21b6750 msgr2=0x5590f21bc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 shutdown_connections 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 1 -- 172.21.15.71:0/3645294589 wait complete. 2024-09-05T12:30:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.154+0000 7f1ac2254740 20 client.15720 trim_cache size 0 max 16384 2024-09-05T12:30:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.169+0000 7f1ac2254740 1 Processor -- start 2024-09-05T12:30:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.169+0000 7f1ac2254740 1 -- start start 2024-09-05T12:30:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21ba1e0 0x5590f21dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 0x5590f21dcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21c53a0 0x5590f21bbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ac2254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b53f0 con 0x5590f21c53a0 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ac2254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b6ff0 con 0x5590f21bab00 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ac2254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b5ac0 con 0x5590f21ba1e0 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 0x5590f21dcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 0x5590f21dcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37706/0 (socket says 172.21.15.71:37706) 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 -- 172.21.15.71:0/1897353976 learned_addr learned my addr 172.21.15.71:0/1897353976 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1abd200640 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21c53a0 0x5590f21bbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1abd200640 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21ba1e0 0x5590f21dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 -- 172.21.15.71:0/1897353976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21ba1e0 msgr2=0x5590f21dc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21ba1e0 0x5590f21dc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 -- 172.21.15.71:0/1897353976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21c53a0 msgr2=0x5590f21bbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21c53a0 0x5590f21bbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 -- 172.21.15.71:0/1897353976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590f21b1d60 con 0x5590f21bab00 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.170+0000 7f1ab7400640 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 0x5590f21dcdf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8103580 tx=0x7f1aa4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=193008aedc546b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1aacc00640 1 -- 172.21.15.71:0/1897353976 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4022070 con 0x5590f21bab00 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1aacc00640 1 -- 172.21.15.71:0/1897353976 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa40047a0 con 0x5590f21bab00 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1aacc00640 1 -- 172.21.15.71:0/1897353976 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401e7a0 con 0x5590f21bab00 2024-09-05T12:30:35.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 -- 172.21.15.71:0/1897353976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 msgr2=0x5590f21dcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 0x5590f21dcdf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8103580 tx=0x7f1aa4005f90 comp rx=0 tx=0).stop 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 -- 172.21.15.71:0/1897353976 shutdown_connections 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21c53a0 0x5590f21bbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21bab00 0x5590f21dcdf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 --2- 172.21.15.71:0/1897353976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21ba1e0 0x5590f21dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 -- 172.21.15.71:0/1897353976 >> 172.21.15.71:0/1897353976 conn(0x5590f21b6750 msgr2=0x5590f21c1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:35.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 -- 172.21.15.71:0/1897353976 shutdown_connections 2024-09-05T12:30:35.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.171+0000 7f1ac2254740 1 -- 172.21.15.71:0/1897353976 wait complete. 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 -- start start 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 Processor -- start 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21ba1e0 0x5590f21cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 0x5590f21c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21c53a0 0x5590f21c9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b6860 con 0x5590f21bab00 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b6190 con 0x5590f21ba1e0 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.172+0000 7f1ac2254740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab81b33d0 con 0x5590f21c53a0 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ac2254740 10 client.0 _ll_register_callbacks cb 0x7f1ab0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 0x5590f21c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 0x5590f21c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51002/0 (socket says 172.21.15.71:51002) 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 -- 172.21.15.71:0/4273303050 learned_addr learned my addr 172.21.15.71:0/4273303050 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1abd200640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21ba1e0 0x5590f21cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1abd200640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21c53a0 0x5590f21c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 -- 172.21.15.71:0/4273303050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21c53a0 msgr2=0x5590f21c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21c53a0 0x5590f21c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 -- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21ba1e0 msgr2=0x5590f21cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21ba1e0 0x5590f21cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 -- 172.21.15.71:0/4273303050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590f21bd030 con 0x5590f21bab00 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1ab7400640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 0x5590f21c8bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4010f90 tx=0x7f1aa4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90da1567783b81b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.173+0000 7f1aa2000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1aa2000640 1 -- 172.21.15.71:0/4273303050 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4022070 con 0x5590f21bab00 2024-09-05T12:30:35.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1aa2000640 1 -- 172.21.15.71:0/4273303050 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f1aa4005020 con 0x5590f21bab00 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa40047a0 con 0x5590f21bab00 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1aa2000640 1 -- 172.21.15.71:0/4273303050 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1aa4005220 con 0x5590f21bab00 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 10 client.15726 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5590f223f690 con 0x5590f21bab00 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1aa2000640 1 -- 172.21.15.71:0/4273303050 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa40047a0 con 0x5590f21bab00 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1aa2000640 1 -- 172.21.15.71:0/4273303050 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1aa401e420 con 0x5590f21bab00 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 10 client.15726 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 10 client.15726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 20 client.15726 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f223fdc0 0x5590f22421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 4 client.15726 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.174+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5590f2127b80 con 0x5590f223fdc0 2024-09-05T12:30:35.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.175+0000 7f1abd200640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f223fdc0 0x5590f22421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.175+0000 7f1abd200640 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f223fdc0 0x5590f22421e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=368d339806386d31 server_cookie=d106f32afa74b254 in_seq=0 out_seq=0 2024-09-05T12:30:35.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.175+0000 7f1aa2000640 10 client.15726 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:35.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81703 5.05 MB/sec warmup 91 sec latency 4404.421 ms 2024-09-05T12:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.205+0000 7f1aa2000640 1 -- 172.21.15.71:0/4273303050 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5590f2127b80 con 0x5590f223fdc0 2024-09-05T12:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.205+0000 7f1aa2000640 10 client.15726 handle_command_reply: tid=0 2024-09-05T12:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 client.15726 shutdown 2024-09-05T12:30:35.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 msgr2=0x5590f21c8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 0x5590f21c8bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4010f90 tx=0x7f1aa4004ab0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 shutdown_connections 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590f223fdc0 0x5590f22421e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590f21c53a0 0x5590f21c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590f21bab00 0x5590f21c8bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 --2- 172.21.15.71:0/4273303050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590f21ba1e0 0x5590f21cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1aa2000640 0 client.15726 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 >> 172.21.15.71:0/4273303050 conn(0x5590f21b6750 msgr2=0x5590f21c2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:35.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 shutdown_connections 2024-09-05T12:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 1 -- 172.21.15.71:0/4273303050 wait complete. 2024-09-05T12:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.206+0000 7f1ac2254740 20 client.15726 trim_cache size 0 max 16384 2024-09-05T12:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:30:35.210 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:30:35.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1a9c0a2da0 msgr2=0x7f1a9c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1a9c0a2da0 0x7f1a9c0a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ccef0 tx=0x7f1aa809ba80 comp rx=0 tx=0).stop 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 msgr2=0x7f1ab81a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 0x7f1ab81a4640 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800ea60 tx=0x7f1aa800ef20 comp rx=0 tx=0).stop 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 shutdown_connections 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab8112960 0x7f1ab819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ab8077040 0x7f1ab81a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1a9c0a2da0 0x7f1a9c0a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 --2- 172.21.15.71:0/670531041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ab8076720 0x7f1ab81a4640 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 >> 172.21.15.71:0/670531041 conn(0x7f1ab80ff980 msgr2=0x7f1ab8101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 shutdown_connections 2024-09-05T12:30:35.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:35.207+0000 7f1abf200640 1 -- 172.21.15.71:0/670531041 wait complete. 2024-09-05T12:30:36.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:35 smithi174 ceph-mon[34313]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 524 KiB/s wr, 60 op/s 2024-09-05T12:30:36.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:35 smithi071 ceph-mon[29845]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 524 KiB/s wr, 60 op/s 2024-09-05T12:30:36.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:35 smithi123 ceph-mon[34321]: pgmap v88: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 524 KiB/s wr, 60 op/s 2024-09-05T12:30:36.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81770 4.99 MB/sec warmup 92 sec latency 4980.823 ms 2024-09-05T12:30:37.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81770 4.94 MB/sec warmup 93 sec latency 1360.284 ms 2024-09-05T12:30:38.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:37 smithi174 ceph-mon[34313]: pgmap v89: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 549 KiB/s wr, 78 op/s 2024-09-05T12:30:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:37 smithi071 ceph-mon[29845]: pgmap v89: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 549 KiB/s wr, 78 op/s 2024-09-05T12:30:38.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:37 smithi123 ceph-mon[34321]: pgmap v89: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 549 KiB/s wr, 78 op/s 2024-09-05T12:30:38.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81770 4.89 MB/sec warmup 94 sec latency 2360.396 ms 2024-09-05T12:30:39.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81770 4.84 MB/sec warmup 95 sec latency 3360.479 ms 2024-09-05T12:30:40.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:39 smithi174 ceph-mon[34313]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 239 KiB/s wr, 61 op/s 2024-09-05T12:30:40.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:39 smithi071 ceph-mon[29845]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 239 KiB/s wr, 61 op/s 2024-09-05T12:30:40.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:39 smithi123 ceph-mon[34321]: pgmap v90: 129 pgs: 129 active+clean; 31 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 239 KiB/s wr, 61 op/s 2024-09-05T12:30:40.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81770 4.79 MB/sec warmup 96 sec latency 4360.621 ms 2024-09-05T12:30:41.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81789 4.74 MB/sec warmup 97 sec latency 4937.315 ms 2024-09-05T12:30:41.898 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:41 smithi071 ceph-mon[29845]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 247 KiB/s wr, 66 op/s 2024-09-05T12:30:42.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:41 smithi174 ceph-mon[34313]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 247 KiB/s wr, 66 op/s 2024-09-05T12:30:42.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:41 smithi123 ceph-mon[34321]: pgmap v91: 129 pgs: 129 active+clean; 31 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 247 KiB/s wr, 66 op/s 2024-09-05T12:30:42.196 INFO:tasks.workunit.client.0.smithi071.stdout: 1 81789 4.69 MB/sec warmup 98 sec latency 1407.180 ms 2024-09-05T12:30:43.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 82426 4.66 MB/sec warmup 99 sec latency 1713.881 ms 2024-09-05T12:30:44.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:43 smithi174 ceph-mon[34313]: pgmap v92: 129 pgs: 129 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-05T12:30:44.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:43 smithi071 ceph-mon[29845]: pgmap v92: 129 pgs: 129 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-05T12:30:44.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:43 smithi123 ceph-mon[34321]: pgmap v92: 129 pgs: 129 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 57 KiB/s wr, 28 op/s 2024-09-05T12:30:44.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 84997 4.75 MB/sec warmup 100 sec latency 11.470 ms 2024-09-05T12:30:45.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 85267 4.74 MB/sec warmup 101 sec latency 883.487 ms 2024-09-05T12:30:46.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:45 smithi174 ceph-mon[34313]: pgmap v93: 129 pgs: 129 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-09-05T12:30:46.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:45 smithi071 ceph-mon[29845]: pgmap v93: 129 pgs: 129 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-09-05T12:30:46.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:45 smithi123 ceph-mon[34321]: pgmap v93: 129 pgs: 129 active+clean; 31 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-09-05T12:30:46.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 85617 4.71 MB/sec warmup 102 sec latency 1463.106 ms 2024-09-05T12:30:47.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 87667 4.73 MB/sec warmup 103 sec latency 67.622 ms 2024-09-05T12:30:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:47 smithi174 ceph-mon[34313]: pgmap v94: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 824 KiB/s wr, 92 op/s 2024-09-05T12:30:48.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:47 smithi071 ceph-mon[29845]: pgmap v94: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 824 KiB/s wr, 92 op/s 2024-09-05T12:30:48.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:47 smithi123 ceph-mon[34321]: pgmap v94: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 824 KiB/s wr, 92 op/s 2024-09-05T12:30:48.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88840 4.79 MB/sec warmup 104 sec latency 533.387 ms 2024-09-05T12:30:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 -- 172.21.15.71:0/4081894857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53041044a0 msgr2=0x7f53041109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 --2- 172.21.15.71:0/4081894857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53041044a0 0x7f53041109d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009cd0 tx=0x7f52fc02a8f0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 -- 172.21.15.71:0/4081894857 shutdown_connections 2024-09-05T12:30:48.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 --2- 172.21.15.71:0/4081894857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f53041044a0 0x7f53041109d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 --2- 172.21.15.71:0/4081894857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5304103b00 0x7f5304103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 --2- 172.21.15.71:0/4081894857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 0x7f5304109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 -- 172.21.15.71:0/4081894857 >> 172.21.15.71:0/4081894857 conn(0x7f53040ff7c0 msgr2=0x7f5304101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 -- 172.21.15.71:0/4081894857 shutdown_connections 2024-09-05T12:30:48.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.321+0000 7f5303200640 1 -- 172.21.15.71:0/4081894857 wait complete. 2024-09-05T12:30:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 Processor -- start 2024-09-05T12:30:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 -- start start 2024-09-05T12:30:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 0x7f530419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53041044a0 0x7f530419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 0x7f530419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5304113f10 con 0x7f5304103b00 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.322+0000 7f5303200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5304113d90 con 0x7f5304109ba0 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5303200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5304114090 con 0x7f53041044a0 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 0x7f530419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 0x7f530419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34542/0 (socket says 172.21.15.71:34542) 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 -- 172.21.15.71:0/1121996244 learned_addr learned my addr 172.21.15.71:0/1121996244 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301400640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53041044a0 0x7f530419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5302800640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 0x7f530419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 -- 172.21.15.71:0/1121996244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53041044a0 msgr2=0x7f530419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53041044a0 0x7f530419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 -- 172.21.15.71:0/1121996244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 msgr2=0x7f530419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 0x7f530419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 -- 172.21.15.71:0/1121996244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f530419f410 con 0x7f5304103b00 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5302800640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 0x7f530419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.323+0000 7f5301e00640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 0x7f530419a420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f52f400ea60 tx=0x7f52f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.324+0000 7f52faa00640 1 -- 172.21.15.71:0/1121996244 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f400cde0 con 0x7f5304103b00 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.324+0000 7f52faa00640 1 -- 172.21.15.71:0/1121996244 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f40035f0 con 0x7f5304103b00 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.324+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f530419f700 con 0x7f5304103b00 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.324+0000 7f52faa00640 1 -- 172.21.15.71:0/1121996244 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f4010670 con 0x7f5304103b00 2024-09-05T12:30:48.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.324+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53041a6f90 con 0x7f5304103b00 2024-09-05T12:30:48.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.327+0000 7f52faa00640 1 -- 172.21.15.71:0/1121996244 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f52f40037a0 con 0x7f5304103b00 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.327+0000 7f530b9da740 1 Processor -- start 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.327+0000 7f530b9da740 1 -- start start 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.327+0000 7f530b9da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 0x5646091ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f530b9da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091aecd0 0x5646091b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f530b9da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091bb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f530b9da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564609114100 con 0x5646091b9570 2024-09-05T12:30:48.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f530b9da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646090bc1b0 con 0x5646091ae3b0 2024-09-05T12:30:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f530b9da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646091100c0 con 0x5646091aecd0 2024-09-05T12:30:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 0x5646091ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5301400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091bb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 0x5646091ae790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48720/0 (socket says 172.21.15.71:48720) 2024-09-05T12:30:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 -- 172.21.15.71:0/1586724785 learned_addr learned my addr 172.21.15.71:0/1586724785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:48.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5301e00640 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091aecd0 0x5646091b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 -- 172.21.15.71:0/1586724785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091aecd0 msgr2=0x5646091b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091aecd0 0x5646091b9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 -- 172.21.15.71:0/1586724785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 msgr2=0x5646091bb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091bb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.328+0000 7f5302800640 1 -- 172.21.15.71:0/1586724785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564608e27210 con 0x5646091ae3b0 2024-09-05T12:30:48.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f52faa00640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f52e80a2cf0 0x7f52e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f5302800640 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 0x5646091ae790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0093e0 tx=0x7f52fc009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccb09c158aef812a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f5301e00640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f52e80a2cf0 0x7f52e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f52faa00640 1 -- 172.21.15.71:0/1121996244 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f52f40cfae0 con 0x7f5304103b00 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f52ef000640 1 -- 172.21.15.71:0/1586724785 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc030620 con 0x5646091ae3b0 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f52ef000640 1 -- 172.21.15.71:0/1586724785 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc036040 con 0x5646091ae3b0 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.329+0000 7f52ef000640 1 -- 172.21.15.71:0/1586724785 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52fc004480 con 0x5646091ae3b0 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 -- 172.21.15.71:0/1586724785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 msgr2=0x5646091ae790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f5301e00640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f52e80a2cf0 0x7f52e80a51b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52f40a43a0 tx=0x7f52f40d3590 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 0x5646091ae790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0093e0 tx=0x7f52fc009a20 comp rx=0 tx=0).stop 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 -- 172.21.15.71:0/1586724785 shutdown_connections 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091bb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091aecd0 0x5646091b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 --2- 172.21.15.71:0/1586724785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091ae3b0 0x5646091ae790 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 -- 172.21.15.71:0/1586724785 >> 172.21.15.71:0/1586724785 conn(0x5646091aa920 msgr2=0x5646091aad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 -- 172.21.15.71:0/1586724785 shutdown_connections 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 -- 172.21.15.71:0/1586724785 wait complete. 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.330+0000 7f530b9da740 1 -- start start 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 Processor -- start 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5646091bbff0 con 0x5646091b9570 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56460909d090 con 0x5646091aecd0 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646091bc170 con 0x5646091ae3b0 2024-09-05T12:30:48.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f530b9da740 10 client.0 _ll_register_callbacks cb 0x5646090f1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f5301400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f5302800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f5301400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48726/0 (socket says 172.21.15.71:48726) 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f5301400640 1 -- 172.21.15.71:0/2299289264 learned_addr learned my addr 172.21.15.71:0/2299289264 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.331+0000 7f5302800640 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f5301400640 1 -- 172.21.15.71:0/2299289264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 msgr2=0x5646091b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f5301400640 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f5301400640 1 -- 172.21.15.71:0/2299289264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 msgr2=0x5646091b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f5301400640 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f5301400640 1 -- 172.21.15.71:0/2299289264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564608ef3620 con 0x5646091aecd0 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f5301400640 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b6b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52f0012700 tx=0x7f52f0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fabe7610717c659b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:48.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f52e3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f52e3400640 1 -- 172.21.15.71:0/2299289264 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f000f8c0 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f52e3400640 1 -- 172.21.15.71:0/2299289264 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f52f000fa60 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.332+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52fc036040 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f52e3400640 1 -- 172.21.15.71:0/2299289264 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52f0025050 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 10 client.25775 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5646091c7dc0 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f52e3400640 1 -- 172.21.15.71:0/2299289264 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52fc036040 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f52e3400640 1 -- 172.21.15.71:0/2299289264 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f52f0010430 con 0x5646091aecd0 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 10 client.25775 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 10 client.25775 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 20 client.25775 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:48.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5646091c9340 0x5646091cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 4 client.25775 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:30:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564609121830 con 0x5646091c9340 2024-09-05T12:30:48.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.333+0000 7f5302800640 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5646091c9340 0x5646091cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.334+0000 7f5302800640 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5646091c9340 0x5646091cb760 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d329d92b76e64b server_cookie=f898c47e08bb667 in_seq=0 out_seq=0 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.334+0000 7f52e3400640 10 client.25775 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f52e3400640 1 -- 172.21.15.71:0/2299289264 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564609121830 con 0x5646091c9340 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f52e3400640 10 client.25775 handle_command_reply: tid=0 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 client.25775 shutdown 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 msgr2=0x5646091b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b6b80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f52f0012700 tx=0x7f52f0012bc0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 shutdown_connections 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5646091c9340 0x5646091cb760 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091b9570 0x5646091b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b6b80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f52e3400640 0 client.25775 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 --2- 172.21.15.71:0/2299289264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 >> 172.21.15.71:0/2299289264 conn(0x5646091aa920 msgr2=0x5646091b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 shutdown_connections 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 1 -- 172.21.15.71:0/2299289264 wait complete. 2024-09-05T12:30:48.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.335+0000 7f530b9da740 20 client.25775 trim_cache size 0 max 16384 2024-09-05T12:30:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 Processor -- start 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 -- start start 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 0x5646091b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091b9570 0x5646091d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b0f90 con 0x5646091aecd0 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b2b90 con 0x5646091b9570 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f530b9da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b1660 con 0x5646091ae3b0 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.350+0000 7f5301400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 0x5646091b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 0x5646091b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34572/0 (socket says 172.21.15.71:34572) 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 -- 172.21.15.71:0/3362586394 learned_addr learned my addr 172.21.15.71:0/3362586394 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5302800640 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5302800640 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091b9570 0x5646091d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 -- 172.21.15.71:0/3362586394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 msgr2=0x5646091b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 -- 172.21.15.71:0/3362586394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091b9570 msgr2=0x5646091d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091b9570 0x5646091d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 -- 172.21.15.71:0/3362586394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646091af340 con 0x5646091aecd0 2024-09-05T12:30:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5302800640 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091b9570 0x5646091d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f5301400640 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 0x5646091b6ef0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000c00 tx=0x7f52f00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c6147f45c2fcac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f52ee600640 1 -- 172.21.15.71:0/3362586394 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f00104f0 con 0x5646091aecd0 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f52ee600640 1 -- 172.21.15.71:0/3362586394 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f00055b0 con 0x5646091aecd0 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.351+0000 7f52ee600640 1 -- 172.21.15.71:0/3362586394 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f000fac0 con 0x5646091aecd0 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 -- 172.21.15.71:0/3362586394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 msgr2=0x5646091b6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 0x5646091b6ef0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f52f0000c00 tx=0x7f52f00096c0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 -- 172.21.15.71:0/3362586394 shutdown_connections 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091b9570 0x5646091d1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091aecd0 0x5646091b6ef0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 --2- 172.21.15.71:0/3362586394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091ae3b0 0x5646091b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 -- 172.21.15.71:0/3362586394 >> 172.21.15.71:0/3362586394 conn(0x5646091aa920 msgr2=0x5646091b85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 -- 172.21.15.71:0/3362586394 shutdown_connections 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 -- 172.21.15.71:0/3362586394 wait complete. 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.352+0000 7f530b9da740 1 -- start start 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 Processor -- start 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 0x5646091b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091b9570 0x5646091b4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b2400 con 0x5646091ae3b0 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53041b1d30 con 0x5646091aecd0 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53041aef70 con 0x5646091b9570 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f530b9da740 10 client.0 _ll_register_callbacks cb 0x7f52f0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f5302800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 0x5646091b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f5301400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f5302800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 0x5646091b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34584/0 (socket says 172.21.15.71:34584) 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f5302800640 1 -- 172.21.15.71:0/2819485209 learned_addr learned my addr 172.21.15.71:0/2819485209 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.353+0000 7f5302800640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091b9570 0x5646091b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5301400640 1 -- 172.21.15.71:0/2819485209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091b9570 msgr2=0x5646091b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5301400640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091b9570 0x5646091b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5301400640 1 -- 172.21.15.71:0/2819485209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 msgr2=0x5646091b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5301400640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 0x5646091b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5301400640 1 -- 172.21.15.71:0/2819485209 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564609233000 con 0x5646091aecd0 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5302800640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 0x5646091b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f5301400640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b3bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5646091a9050 tx=0x7f52f00123a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d1c4489213ff4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f52e3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f52e3400640 1 -- 172.21.15.71:0/2819485209 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f0022040 con 0x5646091aecd0 2024-09-05T12:30:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52f00055b0 con 0x5646091aecd0 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f52e3400640 1 -- 172.21.15.71:0/2819485209 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f52f0005070 con 0x5646091aecd0 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f52e3400640 1 -- 172.21.15.71:0/2819485209 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52f0039050 con 0x5646091aecd0 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f530b9da740 10 client.25781 fetch_fsmap learned FSMap version 32 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5646092342f0 con 0x5646091aecd0 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.354+0000 7f52e3400640 1 -- 172.21.15.71:0/2819485209 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52f00055b0 con 0x5646091aecd0 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f52e3400640 1 -- 172.21.15.71:0/2819485209 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f52f0003780 con 0x5646091aecd0 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f530b9da740 10 client.25781 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f530b9da740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:30:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f530b9da740 20 client.25781 populate_metadata read hostname 'smithi071' 2024-09-05T12:30:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f530b9da740 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564609234a40 0x564609236e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:30:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f530b9da740 4 client.25781 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:30:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56460911bb80 con 0x564609234a40 2024-09-05T12:30:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f5302800640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564609234a40 0x564609236e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:30:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f5302800640 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564609234a40 0x564609236e60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80a20508179e673 server_cookie=59699449df681db4 in_seq=0 out_seq=0 2024-09-05T12:30:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.355+0000 7f52e3400640 10 client.25781 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f52e3400640 1 -- 172.21.15.71:0/2819485209 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56460911bb80 con 0x564609234a40 2024-09-05T12:30:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f52e3400640 10 client.25781 handle_command_reply: tid=0 2024-09-05T12:30:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 client.25781 shutdown 2024-09-05T12:30:48.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 msgr2=0x5646091b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b3bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5646091a9050 tx=0x7f52f00123a0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 shutdown_connections 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564609234a40 0x564609236e60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646091b9570 0x5646091b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646091aecd0 0x5646091b3bc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f52e3400640 0 client.25781 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 --2- 172.21.15.71:0/2819485209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646091ae3b0 0x5646091b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 >> 172.21.15.71:0/2819485209 conn(0x5646091aa920 msgr2=0x5646091c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 shutdown_connections 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 1 -- 172.21.15.71:0/2819485209 wait complete. 2024-09-05T12:30:48.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.387+0000 7f530b9da740 20 client.25781 trim_cache size 0 max 16384 2024-09-05T12:30:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:30:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:30:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:30:48.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f52e80a2cf0 msgr2=0x7f52e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f52e80a2cf0 0x7f52e80a51b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f52f40a43a0 tx=0x7f52f40d3590 comp rx=0 tx=0).stop 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 msgr2=0x7f530419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 0x7f530419a420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f52f400ea60 tx=0x7f52f400ef20 comp rx=0 tx=0).stop 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 shutdown_connections 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5304109ba0 0x7f530419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53041044a0 0x7f530419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f52e80a2cf0 0x7f52e80a51b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 --2- 172.21.15.71:0/1121996244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5304103b00 0x7f530419a420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:30:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 >> 172.21.15.71:0/1121996244 conn(0x7f53040ff7c0 msgr2=0x7f5304100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:30:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 shutdown_connections 2024-09-05T12:30:48.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:30:48.388+0000 7f5303200640 1 -- 172.21.15.71:0/1121996244 wait complete. 2024-09-05T12:30:49.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88840 4.74 MB/sec warmup 105 sec latency 1533.523 ms 2024-09-05T12:30:50.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:49 smithi174 ceph-mon[34313]: pgmap v95: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-09-05T12:30:50.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:49 smithi071 ceph-mon[29845]: pgmap v95: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-09-05T12:30:50.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:49 smithi123 ceph-mon[34321]: pgmap v95: 129 pgs: 129 active+clean; 32 MiB data, 583 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 799 KiB/s wr, 73 op/s 2024-09-05T12:30:50.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88840 4.70 MB/sec warmup 106 sec latency 2533.655 ms 2024-09-05T12:30:51.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88881 4.65 MB/sec warmup 107 sec latency 3108.771 ms 2024-09-05T12:30:52.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:51 smithi071 ceph-mon[29845]: pgmap v96: 129 pgs: 129 active+clean; 30 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1011 KiB/s wr, 124 op/s 2024-09-05T12:30:52.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:51 smithi123 ceph-mon[34321]: pgmap v96: 129 pgs: 129 active+clean; 30 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1011 KiB/s wr, 124 op/s 2024-09-05T12:30:52.197 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88881 4.61 MB/sec warmup 108 sec latency 1386.945 ms 2024-09-05T12:30:52.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:51 smithi174 ceph-mon[34313]: pgmap v96: 129 pgs: 129 active+clean; 30 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1011 KiB/s wr, 124 op/s 2024-09-05T12:30:53.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88881 4.57 MB/sec warmup 109 sec latency 2387.088 ms 2024-09-05T12:30:54.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:53 smithi071 ceph-mon[29845]: pgmap v97: 129 pgs: 129 active+clean; 30 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1019 KiB/s wr, 124 op/s 2024-09-05T12:30:54.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:53 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:30:54.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:53 smithi123 ceph-mon[34321]: pgmap v97: 129 pgs: 129 active+clean; 30 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1019 KiB/s wr, 124 op/s 2024-09-05T12:30:54.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:53 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:30:54.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88881 4.53 MB/sec warmup 110 sec latency 3387.153 ms 2024-09-05T12:30:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:53 smithi174 ceph-mon[34313]: pgmap v97: 129 pgs: 129 active+clean; 30 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1019 KiB/s wr, 124 op/s 2024-09-05T12:30:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:53 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:30:55.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:54 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:30:55.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:54 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:30:55.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:54 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:30:55.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:54 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:30:55.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:54 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:30:55.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:54 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:30:55.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 88881 4.48 MB/sec warmup 111 sec latency 4387.284 ms 2024-09-05T12:30:55.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:54 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:30:55.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:54 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:30:55.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:54 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:30:56.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 89194 4.46 MB/sec warmup 112 sec latency 4964.710 ms 2024-09-05T12:30:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:56 smithi071 ceph-mon[29845]: pgmap v98: 129 pgs: 129 active+clean; 30 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1019 KiB/s wr, 122 op/s 2024-09-05T12:30:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:56 smithi123 ceph-mon[34321]: pgmap v98: 129 pgs: 129 active+clean; 30 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1019 KiB/s wr, 122 op/s 2024-09-05T12:30:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:56 smithi174 ceph-mon[34313]: pgmap v98: 129 pgs: 129 active+clean; 30 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1019 KiB/s wr, 122 op/s 2024-09-05T12:30:57.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 91603 4.52 MB/sec warmup 113 sec latency 9.899 ms 2024-09-05T12:30:58.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 92357 4.53 MB/sec warmup 114 sec latency 663.869 ms 2024-09-05T12:30:58.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:30:58 smithi071 ceph-mon[29845]: pgmap v99: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-05T12:30:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:30:58 smithi123 ceph-mon[34321]: pgmap v99: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-05T12:30:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:30:58 smithi174 ceph-mon[34313]: pgmap v99: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.0 MiB/s wr, 131 op/s 2024-09-05T12:30:59.198 INFO:tasks.workunit.client.0.smithi071.stdout: 1 92357 4.49 MB/sec warmup 115 sec latency 1664.011 ms 2024-09-05T12:31:00.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 92357 4.45 MB/sec warmup 116 sec latency 2664.147 ms 2024-09-05T12:31:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:00 smithi071 ceph-mon[29845]: pgmap v100: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 239 KiB/s wr, 65 op/s 2024-09-05T12:31:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:00 smithi123 ceph-mon[34321]: pgmap v100: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 239 KiB/s wr, 65 op/s 2024-09-05T12:31:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:00 smithi174 ceph-mon[34313]: pgmap v100: 129 pgs: 129 active+clean; 30 MiB data, 527 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 239 KiB/s wr, 65 op/s 2024-09-05T12:31:01.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 92459 4.42 MB/sec warmup 117 sec latency 3240.502 ms 2024-09-05T12:31:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.514+0000 7f916f200640 1 -- 172.21.15.71:0/1603411217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9170074ec0 msgr2=0x7f917010dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.514+0000 7f916f200640 1 --2- 172.21.15.71:0/1603411217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9170074ec0 0x7f917010dc30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f915c009900 tx=0x7f915c02a480 comp rx=0 tx=0).stop 2024-09-05T12:31:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 -- 172.21.15.71:0/1603411217 shutdown_connections 2024-09-05T12:31:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 --2- 172.21.15.71:0/1603411217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f917010e170 0x7f9170110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 --2- 172.21.15.71:0/1603411217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9170074ec0 0x7f917010dc30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 --2- 172.21.15.71:0/1603411217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91700745a0 0x7f9170074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 -- 172.21.15.71:0/1603411217 >> 172.21.15.71:0/1603411217 conn(0x7f91700fd5d0 msgr2=0x7f91700ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 -- 172.21.15.71:0/1603411217 shutdown_connections 2024-09-05T12:31:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.515+0000 7f916f200640 1 -- 172.21.15.71:0/1603411217 wait complete. 2024-09-05T12:31:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.516+0000 7f916f200640 1 Processor -- start 2024-09-05T12:31:01.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.516+0000 7f916f200640 1 -- start start 2024-09-05T12:31:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 0x7f917019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9170074ec0 0x7f917019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f917010e170 0x7f917019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9170113ef0 con 0x7f9170074ec0 2024-09-05T12:31:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9170113d70 con 0x7f91700745a0 2024-09-05T12:31:01.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9170114070 con 0x7f917010e170 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 0x7f917019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9170074ec0 0x7f917019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f917010e170 0x7f917019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 0x7f917019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36020/0 (socket says 172.21.15.71:36020) 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.517+0000 7f916de00640 1 -- 172.21.15.71:0/3599960235 learned_addr learned my addr 172.21.15.71:0/3599960235 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916de00640 1 -- 172.21.15.71:0/3599960235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f917010e170 msgr2=0x7f917019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916de00640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f917010e170 0x7f917019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916de00640 1 -- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9170074ec0 msgr2=0x7f917019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916de00640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9170074ec0 0x7f917019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916de00640 1 -- 172.21.15.71:0/3599960235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917019f3f0 con 0x7f91700745a0 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916e800640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f917010e170 0x7f917019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916de00640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 0x7f917019a400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f916000e9f0 tx=0x7f916000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f9166a00640 1 -- 172.21.15.71:0/3599960235 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916000cd80 con 0x7f91700745a0 2024-09-05T12:31:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f9166a00640 1 -- 172.21.15.71:0/3599960235 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9160005500 con 0x7f91700745a0 2024-09-05T12:31:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917019f6e0 con 0x7f91700745a0 2024-09-05T12:31:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f9166a00640 1 -- 172.21.15.71:0/3599960235 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9160010690 con 0x7f91700745a0 2024-09-05T12:31:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.518+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701a6f70 con 0x7f91700745a0 2024-09-05T12:31:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.521+0000 7f9177d22740 1 Processor -- start 2024-09-05T12:31:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.521+0000 7f9166a00640 1 -- 172.21.15.71:0/3599960235 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f91600046c0 con 0x7f91700745a0 2024-09-05T12:31:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.521+0000 7f9177d22740 1 -- start start 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f9177d22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f9177d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f9177d22740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f9177d22740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562d5562a100 con 0x562d556cf570 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f9177d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d555d21b0 con 0x562d556c4cd0 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f9177d22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d556260c0 con 0x562d556c43b0 2024-09-05T12:31:01.524 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556c4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36816/0 (socket says 172.21.15.71:36816) 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 -- 172.21.15.71:0/600514791 learned_addr learned my addr 172.21.15.71:0/600514791 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916de00640 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916d400640 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 -- 172.21.15.71:0/600514791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 msgr2=0x562d556cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 -- 172.21.15.71:0/600514791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 msgr2=0x562d556d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556d1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.522+0000 7f916e800640 1 -- 172.21.15.71:0/600514791 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d5533d210 con 0x562d556c43b0 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f916d400640 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f916e800640 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556c4790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9168011790 tx=0x7f9168011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bd68bba55863836 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f915b400640 1 -- 172.21.15.71:0/600514791 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168003c30 con 0x562d556c43b0 2024-09-05T12:31:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f915b400640 1 -- 172.21.15.71:0/600514791 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916800fe80 con 0x562d556c43b0 2024-09-05T12:31:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f915b400640 1 -- 172.21.15.71:0/600514791 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91680193e0 con 0x562d556c43b0 2024-09-05T12:31:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f9166a00640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91540a2d70 0x7f91540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.523+0000 7f9166a00640 1 -- 172.21.15.71:0/3599960235 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f91600cfb70 con 0x7f91700745a0 2024-09-05T12:31:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f916de00640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91540a2d70 0x7f91540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 -- 172.21.15.71:0/600514791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 msgr2=0x562d556c4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556c4790 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9168011790 tx=0x7f9168011c50 comp rx=0 tx=0).stop 2024-09-05T12:31:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 -- 172.21.15.71:0/600514791 shutdown_connections 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 --2- 172.21.15.71:0/600514791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556c4790 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 -- 172.21.15.71:0/600514791 >> 172.21.15.71:0/600514791 conn(0x562d556c0920 msgr2=0x562d556c0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 -- 172.21.15.71:0/600514791 shutdown_connections 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f9177d22740 1 -- 172.21.15.71:0/600514791 wait complete. 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.524+0000 7f916de00640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91540a2d70 0x7f91540a5230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9160098150 tx=0x7f91600d3620 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 -- start start 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 Processor -- start 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c43b0 0x562d556cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 0x562d556ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562d556d1ff0 con 0x562d556c43b0 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d555b3090 con 0x562d556cf570 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d556d2170 con 0x562d556c4cd0 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f9177d22740 10 client.0 _ll_register_callbacks cb 0x562d55607f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f916d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 0x562d556ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f916d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 0x562d556ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36822/0 (socket says 172.21.15.71:36822) 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.525+0000 7f916d400640 1 -- 172.21.15.71:0/2820867633 learned_addr learned my addr 172.21.15.71:0/2820867633 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916e800640 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c43b0 0x562d556cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916d400640 1 -- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 msgr2=0x562d556cd0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916d400640 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556cd0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916d400640 1 -- 172.21.15.71:0/2820867633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c43b0 msgr2=0x562d556cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916d400640 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c43b0 0x562d556cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916d400640 1 -- 172.21.15.71:0/2820867633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d55409620 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f916d400640 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 0x562d556ccb80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f915c009900 tx=0x7f915c004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3371042db3f6f54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f914fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f914fe00640 1 -- 172.21.15.71:0/2820867633 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c035040 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562d55637830 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.526+0000 7f914fe00640 1 -- 172.21.15.71:0/2820867633 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f915c005230 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f914fe00640 1 -- 172.21.15.71:0/2820867633 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f915c005430 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 10 client.25801 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562d556ddde0 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f914fe00640 1 -- 172.21.15.71:0/2820867633 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562d55637830 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f914fe00640 1 -- 172.21.15.71:0/2820867633 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f915c0039f0 con 0x562d556c4cd0 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 10 client.25801 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 10 client.25801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:01.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 20 client.25801 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d556df2a0 0x562d556e16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 4 client.25801 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.527+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d55631b80 con 0x562d556df2a0 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.528+0000 7f916e800640 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d556df2a0 0x562d556e16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.528+0000 7f916e800640 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d556df2a0 0x562d556e16e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8db40cae1a6b7 server_cookie=80054150b1f361a1 in_seq=0 out_seq=0 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.528+0000 7f914fe00640 10 client.25801 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f914fe00640 1 -- 172.21.15.71:0/2820867633 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d55631b80 con 0x562d556df2a0 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f914fe00640 10 client.25801 handle_command_reply: tid=0 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 client.25801 shutdown 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 msgr2=0x562d556ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 0x562d556ccb80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f915c009900 tx=0x7f915c004270 comp rx=0 tx=0).stop 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 shutdown_connections 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d556df2a0 0x562d556e16e0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f914fe00640 0 client.25801 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c4cd0 0x562d556ccb80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 --2- 172.21.15.71:0/2820867633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c43b0 0x562d556cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.529+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 >> 172.21.15.71:0/2820867633 conn(0x562d556c0920 msgr2=0x562d556c6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.530+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 shutdown_connections 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.530+0000 7f9177d22740 1 -- 172.21.15.71:0/2820867633 wait complete. 2024-09-05T12:31:01.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.530+0000 7f9177d22740 20 client.25801 trim_cache size 0 max 16384 2024-09-05T12:31:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 Processor -- start 2024-09-05T12:31:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 -- start start 2024-09-05T12:31:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c4cd0 0x562d556e7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556bf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91701b0f10 con 0x562d556c4cd0 2024-09-05T12:31:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91701b2b10 con 0x562d556cf570 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f9177d22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91701b15e0 con 0x562d556c43b0 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f916e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f916d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c4cd0 0x562d556e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f916e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556e6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36826/0 (socket says 172.21.15.71:36826) 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.546+0000 7f916e800640 1 -- 172.21.15.71:0/2510137404 learned_addr learned my addr 172.21.15.71:0/2510137404 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916e800640 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556bf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916d400640 1 -- 172.21.15.71:0/2510137404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 msgr2=0x562d556e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916d400640 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916d400640 1 -- 172.21.15.71:0/2510137404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 msgr2=0x562d556bf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916d400640 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556bf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916d400640 1 -- 172.21.15.71:0/2510137404 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d556e1d60 con 0x562d556c4cd0 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916e800640 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:01.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f916d400640 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c4cd0 0x562d556e7530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f915c003fb0 tx=0x7f915c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d122ca39418e0d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.547+0000 7f915aa00640 1 -- 172.21.15.71:0/2510137404 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c057070 con 0x562d556c4cd0 2024-09-05T12:31:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f915aa00640 1 -- 172.21.15.71:0/2510137404 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c040070 con 0x562d556c4cd0 2024-09-05T12:31:01.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f915aa00640 1 -- 172.21.15.71:0/2510137404 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c035030 con 0x562d556c4cd0 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 -- 172.21.15.71:0/2510137404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c4cd0 msgr2=0x562d556e7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c4cd0 0x562d556e7530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f915c003fb0 tx=0x7f915c02a480 comp rx=0 tx=0).stop 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 -- 172.21.15.71:0/2510137404 shutdown_connections 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556cf570 0x562d556bf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556c4cd0 0x562d556e7530 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 --2- 172.21.15.71:0/2510137404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 -- 172.21.15.71:0/2510137404 >> 172.21.15.71:0/2510137404 conn(0x562d556c0920 msgr2=0x562d556cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 -- 172.21.15.71:0/2510137404 shutdown_connections 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.548+0000 7f9177d22740 1 -- 172.21.15.71:0/2510137404 wait complete. 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 -- start start 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 Processor -- start 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556d7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91701b2380 con 0x562d556cf570 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91701b1cb0 con 0x562d556c4cd0 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91701aeef0 con 0x562d556c43b0 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.549+0000 7f9177d22740 10 client.0 _ll_register_callbacks cb 0x7f9160005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36050/0 (socket says 172.21.15.71:36050) 2024-09-05T12:31:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 -- 172.21.15.71:0/745544186 learned_addr learned my addr 172.21.15.71:0/745544186 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916e800640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556d7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916e800640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 -- 172.21.15.71:0/745544186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 msgr2=0x562d556d7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556d7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 -- 172.21.15.71:0/745544186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 msgr2=0x562d556c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 -- 172.21.15.71:0/745544186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d556ca230 con 0x562d556c4cd0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916e800640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f916d400640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556c94f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f915c055040 tx=0x7f915c031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1871a28d3707c021 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.550+0000 7f914fe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f914fe00640 1 -- 172.21.15.71:0/745544186 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f915c057040 con 0x562d556c4cd0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f915c040070 con 0x562d556c4cd0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f914fe00640 1 -- 172.21.15.71:0/745544186 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f915c051680 con 0x562d556c4cd0 2024-09-05T12:31:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f914fe00640 1 -- 172.21.15.71:0/745544186 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f915c035040 con 0x562d556c4cd0 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 10 client.25796 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562d5574a000 con 0x562d556c4cd0 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f914fe00640 1 -- 172.21.15.71:0/745544186 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f915c040070 con 0x562d556c4cd0 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f914fe00640 1 -- 172.21.15.71:0/745544186 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f915c05a020 con 0x562d556c4cd0 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 10 client.25796 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 10 client.25796 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 20 client.25796 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d5574a860 0x562d5574cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 4 client.25796 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.551+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562d555ae7c0 con 0x562d5574a860 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.552+0000 7f916e800640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d5574a860 0x562d5574cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.552+0000 7f916e800640 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d5574a860 0x562d5574cc80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf119b3fdd94bd7 server_cookie=a0207592658e63a4 in_seq=0 out_seq=0 2024-09-05T12:31:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.552+0000 7f914fe00640 10 client.25796 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f914fe00640 1 -- 172.21.15.71:0/745544186 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562d555ae7c0 con 0x562d5574a860 2024-09-05T12:31:01.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f914fe00640 10 client.25796 handle_command_reply: tid=0 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 client.25796 shutdown 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 msgr2=0x562d556c94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556c94f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f915c055040 tx=0x7f915c031400 comp rx=0 tx=0).stop 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 shutdown_connections 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562d5574a860 0x562d5574cc80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562d556cf570 0x562d556c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d556c4cd0 0x562d556c94f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f914fe00640 0 client.25796 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 --2- 172.21.15.71:0/745544186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d556c43b0 0x562d556d7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 >> 172.21.15.71:0/745544186 conn(0x562d556c0920 msgr2=0x562d556bf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 shutdown_connections 2024-09-05T12:31:01.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 1 -- 172.21.15.71:0/745544186 wait complete. 2024-09-05T12:31:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.578+0000 7f9177d22740 20 client.25796 trim_cache size 0 max 16384 2024-09-05T12:31:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:31:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:31:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:31:01.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:31:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91540a2d70 msgr2=0x7f91540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91540a2d70 0x7f91540a5230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9160098150 tx=0x7f91600d3620 comp rx=0 tx=0).stop 2024-09-05T12:31:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 msgr2=0x7f917019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 0x7f917019a400 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f916000e9f0 tx=0x7f916000edf0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 shutdown_connections 2024-09-05T12:31:01.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f917010e170 0x7f917019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91540a2d70 0x7f91540a5230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9170074ec0 0x7f917019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 --2- 172.21.15.71:0/3599960235 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91700745a0 0x7f917019a400 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 >> 172.21.15.71:0/3599960235 conn(0x7f91700fd5d0 msgr2=0x7f91700fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 shutdown_connections 2024-09-05T12:31:01.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:01.579+0000 7f916f200640 1 -- 172.21.15.71:0/3599960235 wait complete. 2024-09-05T12:31:02.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 93009 4.39 MB/sec warmup 118 sec latency 66.351 ms 2024-09-05T12:31:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:02 smithi071 ceph-mon[29845]: pgmap v101: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 444 KiB/s wr, 104 op/s 2024-09-05T12:31:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:02 smithi123 ceph-mon[34321]: pgmap v101: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 444 KiB/s wr, 104 op/s 2024-09-05T12:31:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:02 smithi174 ceph-mon[34313]: pgmap v101: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 444 KiB/s wr, 104 op/s 2024-09-05T12:31:03.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95464 4.47 MB/sec warmup 119 sec latency 9.717 ms 2024-09-05T12:31:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:04 smithi071 ceph-mon[29845]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 236 KiB/s wr, 55 op/s 2024-09-05T12:31:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:04 smithi123 ceph-mon[34321]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 236 KiB/s wr, 55 op/s 2024-09-05T12:31:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:04 smithi174 ceph-mon[34313]: pgmap v102: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 236 KiB/s wr, 55 op/s 2024-09-05T12:31:05.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95822 0.00 MB/sec execute 1 sec latency 1848.893 ms 2024-09-05T12:31:06.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95954 0.16 MB/sec execute 2 sec latency 2425.443 ms 2024-09-05T12:31:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:06 smithi071 ceph-mon[29845]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 219 KiB/s wr, 50 op/s 2024-09-05T12:31:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:06 smithi123 ceph-mon[34321]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 219 KiB/s wr, 50 op/s 2024-09-05T12:31:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:06 smithi174 ceph-mon[34313]: pgmap v103: 129 pgs: 129 active+clean; 32 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 219 KiB/s wr, 50 op/s 2024-09-05T12:31:07.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95954 0.11 MB/sec execute 3 sec latency 1326.271 ms 2024-09-05T12:31:08.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95954 0.08 MB/sec execute 4 sec latency 2326.387 ms 2024-09-05T12:31:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:08 smithi071 ceph-mon[29845]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 566 KiB/s wr, 135 op/s 2024-09-05T12:31:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:08 smithi123 ceph-mon[34321]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 566 KiB/s wr, 135 op/s 2024-09-05T12:31:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:08 smithi174 ceph-mon[34313]: pgmap v104: 129 pgs: 129 active+clean; 34 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 566 KiB/s wr, 135 op/s 2024-09-05T12:31:09.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95954 0.07 MB/sec execute 5 sec latency 3326.503 ms 2024-09-05T12:31:10.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95954 0.05 MB/sec execute 6 sec latency 4326.556 ms 2024-09-05T12:31:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:10 smithi071 ceph-mon[29845]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 555 KiB/s wr, 126 op/s 2024-09-05T12:31:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:10 smithi123 ceph-mon[34321]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 555 KiB/s wr, 126 op/s 2024-09-05T12:31:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:10 smithi174 ceph-mon[34313]: pgmap v105: 129 pgs: 129 active+clean; 34 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 555 KiB/s wr, 126 op/s 2024-09-05T12:31:11.199 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95973 0.05 MB/sec execute 7 sec latency 4900.407 ms 2024-09-05T12:31:12.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95973 0.04 MB/sec execute 8 sec latency 1410.100 ms 2024-09-05T12:31:12.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:12 smithi071 ceph-mon[29845]: pgmap v106: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 571 KiB/s wr, 134 op/s 2024-09-05T12:31:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:12 smithi123 ceph-mon[34321]: pgmap v106: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 571 KiB/s wr, 134 op/s 2024-09-05T12:31:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:12 smithi174 ceph-mon[34313]: pgmap v106: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 571 KiB/s wr, 134 op/s 2024-09-05T12:31:13.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95973 0.04 MB/sec execute 9 sec latency 2410.273 ms 2024-09-05T12:31:14.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 95973 0.04 MB/sec execute 10 sec latency 3410.319 ms 2024-09-05T12:31:14.314 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:14 smithi071 ceph-mon[29845]: pgmap v107: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 367 KiB/s wr, 97 op/s 2024-09-05T12:31:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:14 smithi123 ceph-mon[34321]: pgmap v107: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 367 KiB/s wr, 97 op/s 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- 172.21.15.71:0/3067353362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 msgr2=0x7f0bd0110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/3067353362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd0110540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009ca0 tx=0x7f0bc802a950 comp rx=0 tx=0).stop 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- 172.21.15.71:0/3067353362 shutdown_connections 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/3067353362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd0110540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/3067353362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 0x7f0bd010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/3067353362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 0x7f0bd0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- 172.21.15.71:0/3067353362 >> 172.21.15.71:0/3067353362 conn(0x7f0bd00fd5d0 msgr2=0x7f0bd00ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- 172.21.15.71:0/3067353362 shutdown_connections 2024-09-05T12:31:14.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- 172.21.15.71:0/3067353362 wait complete. 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 Processor -- start 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- start start 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 0x7f0bd019a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 0x7f0bd019a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0113ef0 con 0x7f0bd010e0f0 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0113d70 con 0x7f0bd0074f40 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd0114070 con 0x7f0bd0074620 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd019ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58484/0 (socket says 172.21.15.71:58484) 2024-09-05T12:31:14.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 -- 172.21.15.71:0/2246157426 learned_addr learned my addr 172.21.15.71:0/2246157426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 0x7f0bd019a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 0x7f0bd019a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 -- 172.21.15.71:0/2246157426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 msgr2=0x7f0bd019a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 0x7f0bd019a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 -- 172.21.15.71:0/2246157426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 msgr2=0x7f0bd019a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 0x7f0bd019a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd5400640 1 -- 172.21.15.71:0/2246157426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd019f3a0 con 0x7f0bd010e0f0 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 0x7f0bd019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.693+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 0x7f0bd019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.698+0000 7f0bd5400640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd019ec70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009dc0 tx=0x7f0bc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.698+0000 7f0bcd600640 1 -- 172.21.15.71:0/2246157426 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc8034040 con 0x7f0bd010e0f0 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.698+0000 7f0bcd600640 1 -- 172.21.15.71:0/2246157426 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc802f940 con 0x7f0bd010e0f0 2024-09-05T12:31:14.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.698+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd019f630 con 0x7f0bd010e0f0 2024-09-05T12:31:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.698+0000 7f0bcd600640 1 -- 172.21.15.71:0/2246157426 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc80046a0 con 0x7f0bd010e0f0 2024-09-05T12:31:14.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.698+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd01a6f20 con 0x7f0bd010e0f0 2024-09-05T12:31:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcd600640 1 -- 172.21.15.71:0/2246157426 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0bc8037020 con 0x7f0bd010e0f0 2024-09-05T12:31:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 Processor -- start 2024-09-05T12:31:14.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- start start 2024-09-05T12:31:14.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585e3b0 0x55f62585e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 0x55f625869030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f62586b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f6257c4100 con 0x55f62585ecd0 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f62576c1b0 con 0x55f625869570 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6257c00c0 con 0x55f62585e3b0 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 0x55f625869030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcfe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585e3b0 0x55f62585e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 0x55f625869030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58498/0 (socket says 172.21.15.71:58498) 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 -- 172.21.15.71:0/1597056203 learned_addr learned my addr 172.21.15.71:0/1597056203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:14.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f62586b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 -- 172.21.15.71:0/1597056203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585e3b0 msgr2=0x55f62585e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585e3b0 0x55f62585e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 -- 172.21.15.71:0/1597056203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 msgr2=0x55f62586b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f62586b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 -- 172.21.15.71:0/1597056203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6254d7210 con 0x55f62585ecd0 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f62586b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 0x55f625869030 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40115c0 tx=0x7f0bc4011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92513e2dde7a34f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bc1c00640 1 -- 172.21.15.71:0/1597056203 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400f640 con 0x55f62585ecd0 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bc1c00640 1 -- 172.21.15.71:0/1597056203 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc400fcc0 con 0x55f62585ecd0 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bc1c00640 1 -- 172.21.15.71:0/1597056203 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4019d40 con 0x55f62585ecd0 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcd600640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0bb80a2cf0 0x7f0bb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcd600640 1 -- 172.21.15.71:0/2246157426 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f0bc80f03b0 con 0x7f0bd010e0f0 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0bb80a2cf0 0x7f0bb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1597056203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 msgr2=0x55f625869030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 0x55f625869030 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40115c0 tx=0x7f0bc4011a80 comp rx=0 tx=0).stop 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1597056203 shutdown_connections 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f62586b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585ecd0 0x55f625869030 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1597056203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585e3b0 0x55f62585e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1597056203 >> 172.21.15.71:0/1597056203 conn(0x55f62585a920 msgr2=0x55f62585ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0bb80a2cf0 0x7f0bb80a51b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55f62585bca0 tx=0x7f0bbc013040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1597056203 shutdown_connections 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.699+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1597056203 wait complete. 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- start start 2024-09-05T12:31:14.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 Processor -- start 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625866710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62585ecd0 0x55f625866c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 0x55f625867190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f62586bff0 con 0x55f62585e3b0 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f62574d090 con 0x55f62585ecd0 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f62586c170 con 0x55f625869570 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 10 client.0 _ll_register_callbacks cb 0x55f6257a1f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625866710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bcfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62585ecd0 0x55f625866c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 0x55f625867190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 0x55f625867190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40692/0 (socket says 172.21.15.71:40692) 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bcfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62585ecd0 0x55f625866c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34386/0 (socket says 172.21.15.71:34386) 2024-09-05T12:31:14.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 -- 172.21.15.71:0/1116117804 learned_addr learned my addr 172.21.15.71:0/1116117804 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 -- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62585ecd0 msgr2=0x55f625866c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62585ecd0 0x55f625866c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 -- 172.21.15.71:0/1116117804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 msgr2=0x55f625866710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625866710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 -- 172.21.15.71:0/1116117804 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6255a3620 con 0x55f625869570 2024-09-05T12:31:14.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625866710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd5400640 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 0x55f625867190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80c4d80 tx=0x7f0bc80bc060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ba9f2769638cc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:31:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 1 -- 172.21.15.71:0/1116117804 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc802c040 con 0x55f625869570 2024-09-05T12:31:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 1 -- 172.21.15.71:0/1116117804 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f0bc80b8cb0 con 0x55f625869570 2024-09-05T12:31:14.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc400fcc0 con 0x55f625869570 2024-09-05T12:31:14.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 1 -- 172.21.15.71:0/1116117804 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0bc80bcd40 con 0x55f625869570 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 10 client.25819 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f62586c050 con 0x55f625869570 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 1 -- 172.21.15.71:0/1116117804 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc400fcc0 con 0x55f625869570 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 1 -- 172.21.15.71:0/1116117804 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0bc80bc8a0 con 0x55f625869570 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 10 client.25819 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 10 client.25819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 20 client.25819 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258794d0 0x55f62587b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 4 client.25819 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6257d1830 con 0x55f6258794d0 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258794d0 0x55f62587b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258794d0 0x55f62587b950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85d6da45a204d7a server_cookie=91bc92f61fb8be24 in_seq=0 out_seq=0 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 10 client.25819 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 1 -- 172.21.15.71:0/1116117804 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6257d1830 con 0x55f6258794d0 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 10 client.25819 handle_command_reply: tid=0 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 client.25819 shutdown 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 msgr2=0x55f625867190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 0x55f625867190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0bc80c4d80 tx=0x7f0bc80bc060 comp rx=0 tx=0).stop 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 shutdown_connections 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258794d0 0x55f62587b950 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f625869570 0x55f625867190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f62585ecd0 0x55f625866c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/1116117804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625866710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bb6000640 0 client.25819 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:14.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 >> 172.21.15.71:0/1116117804 conn(0x55f62585a920 msgr2=0x55f625860630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 shutdown_connections 2024-09-05T12:31:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.705+0000 7f0bda1c7740 1 -- 172.21.15.71:0/1116117804 wait complete. 2024-09-05T12:31:14.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.710+0000 7f0bda1c7740 20 client.25819 trim_cache size 0 max 16384 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 Processor -- start 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- start start 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f62585a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625881210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f625881750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01b0e00 con 0x55f62585e3b0 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01b2a00 con 0x55f625869570 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01b14d0 con 0x55f62585ecd0 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f62585a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f62585a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58522/0 (socket says 172.21.15.71:58522) 2024-09-05T12:31:14.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 -- 172.21.15.71:0/688394010 learned_addr learned my addr 172.21.15.71:0/688394010 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625881210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd5400640 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f625881750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 -- 172.21.15.71:0/688394010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 msgr2=0x55f625881210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625881210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 -- 172.21.15.71:0/688394010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 msgr2=0x55f625881750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f625881750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 -- 172.21.15.71:0/688394010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f62585ead0 con 0x55f62585e3b0 2024-09-05T12:31:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd5400640 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f625881750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f62585a3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40058c0 tx=0x7f0bc4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f22bce546b6799e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:14.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bc1200640 1 -- 172.21.15.71:0/688394010 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4030070 con 0x55f62585e3b0 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bc1200640 1 -- 172.21.15.71:0/688394010 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc4019870 con 0x55f62585e3b0 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bc1200640 1 -- 172.21.15.71:0/688394010 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400f930 con 0x55f62585e3b0 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- 172.21.15.71:0/688394010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 msgr2=0x55f62585a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f62585a3c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40058c0 tx=0x7f0bc4009310 comp rx=0 tx=0).stop 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- 172.21.15.71:0/688394010 shutdown_connections 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f625881750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625881210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/688394010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f62585a3c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- 172.21.15.71:0/688394010 >> 172.21.15.71:0/688394010 conn(0x55f62585a920 msgr2=0x55f6258556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- 172.21.15.71:0/688394010 shutdown_connections 2024-09-05T12:31:14.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- 172.21.15.71:0/688394010 wait complete. 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- start start 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 Processor -- start 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625871ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625863370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f6258638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01b2270 con 0x55f62585e3b0 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01b1ba0 con 0x55f625869570 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd01aede0 con 0x55f62585ecd0 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.723+0000 7f0bda1c7740 10 client.0 _ll_register_callbacks cb 0x7f0bc8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625871ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625871ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58524/0 (socket says 172.21.15.71:58524) 2024-09-05T12:31:14.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 -- 172.21.15.71:0/2458238312 learned_addr learned my addr 172.21.15.71:0/2458238312 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625863370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd5400640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f6258638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 -- 172.21.15.71:0/2458238312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 msgr2=0x55f625863370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625863370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 -- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 msgr2=0x55f6258638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f6258638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 -- 172.21.15.71:0/2458238312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6258640e0 con 0x55f62585e3b0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bd4a00640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625871ab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009ce0 tx=0x7f0bc40096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1168ab91180d1e6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 1 -- 172.21.15.71:0/2458238312 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4030070 con 0x55f62585e3b0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4019870 con 0x55f62585e3b0 2024-09-05T12:31:14.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 1 -- 172.21.15.71:0/2458238312 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f0bc4019c30 con 0x55f62585e3b0 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 1 -- 172.21.15.71:0/2458238312 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0bc4039050 con 0x55f62585e3b0 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 10 client.15780 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f6258e4160 con 0x55f62585e3b0 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 1 -- 172.21.15.71:0/2458238312 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bc4019870 con 0x55f62585e3b0 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 1 -- 172.21.15.71:0/2458238312 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f0bc4033020 con 0x55f62585e3b0 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 10 client.15780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:14.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 20 client.15780 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258e46e0 0x55f6258e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 4 client.15780 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:31:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6257cbb80 con 0x55f6258e46e0 2024-09-05T12:31:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258e46e0 0x55f6258e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bcfe00640 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258e46e0 0x55f6258e6b00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960685e6b46a4f35 server_cookie=f5d12681152db79 in_seq=0 out_seq=0 2024-09-05T12:31:14.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.728+0000 7f0bb6000640 10 client.15780 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:14 smithi174 ceph-mon[34313]: pgmap v107: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 367 KiB/s wr, 97 op/s 2024-09-05T12:31:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bb6000640 1 -- 172.21.15.71:0/2458238312 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6257cbb80 con 0x55f6258e46e0 2024-09-05T12:31:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bb6000640 10 client.15780 handle_command_reply: tid=0 2024-09-05T12:31:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 client.15780 shutdown 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 msgr2=0x55f625871ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625871ab0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009ce0 tx=0x7f0bc40096a0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 shutdown_connections 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f6258e46e0 0x55f6258e6b00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f625869570 0x55f6258638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f62585ecd0 0x55f625863370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bb6000640 0 client.15780 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 --2- 172.21.15.71:0/2458238312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f62585e3b0 0x55f625871ab0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 >> 172.21.15.71:0/2458238312 conn(0x55f62585a920 msgr2=0x55f62585fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 shutdown_connections 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 1 -- 172.21.15.71:0/2458238312 wait complete. 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bda1c7740 20 client.15780 trim_cache size 0 max 16384 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:31:14.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:31:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0bb80a2cf0 msgr2=0x7f0bb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0bb80a2cf0 0x7f0bb80a51b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55f62585bca0 tx=0x7f0bbc013040 comp rx=0 tx=0).stop 2024-09-05T12:31:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 msgr2=0x7f0bd019ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd019ec70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009dc0 tx=0x7f0bc8003c10 comp rx=0 tx=0).stop 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 shutdown_connections 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0bb80a2cf0 0x7f0bb80a51b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bd010e0f0 0x7f0bd019ec70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd0074f40 0x7f0bd019a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 --2- 172.21.15.71:0/2246157426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0074620 0x7f0bd019a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 >> 172.21.15.71:0/2246157426 conn(0x7f0bd00fd5d0 msgr2=0x7f0bd00fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 shutdown_connections 2024-09-05T12:31:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:14.758+0000 7f0bd5e00640 1 -- 172.21.15.71:0/2246157426 wait complete. 2024-09-05T12:31:15.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 96059 0.12 MB/sec execute 11 sec latency 4273.028 ms 2024-09-05T12:31:15.205 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.1 2024-09-05T12:31:15.205 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:31:15.657 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:31:16.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 97269 0.56 MB/sec execute 12 sec latency 13.923 ms 2024-09-05T12:31:16.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:16 smithi071 ceph-mon[29845]: pgmap v108: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 363 KiB/s wr, 94 op/s 2024-09-05T12:31:16.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.448+0000 7fcf10400640 1 -- 172.21.15.71:0/1116980918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf08112640 msgr2=0x7fcf08112a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.448+0000 7fcf10400640 1 --2- 172.21.15.71:0/1116980918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf08112640 0x7fcf08112a20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009930 tx=0x7fcefc02a420 comp rx=0 tx=0).stop 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 -- 172.21.15.71:0/1116980918 shutdown_connections 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 --2- 172.21.15.71:0/1116980918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf0810cf30 0x7fcf08119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 --2- 172.21.15.71:0/1116980918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf0810c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 --2- 172.21.15.71:0/1116980918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf08112640 0x7fcf08112a20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 -- 172.21.15.71:0/1116980918 >> 172.21.15.71:0/1116980918 conn(0x7fcf0806cdc0 msgr2=0x7fcf0806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 -- 172.21.15.71:0/1116980918 shutdown_connections 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.449+0000 7fcf10400640 1 -- 172.21.15.71:0/1116980918 wait complete. 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 Processor -- start 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 -- start start 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf081a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:16.592 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf0810cf30 0x7fcf081a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf08112640 0x7fcf081a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0811c9a0 con 0x7fcf0810c590 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0811c820 con 0x7fcf0810cf30 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf10400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0811cb20 con 0x7fcf08112640 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf0da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf081a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf0d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf0810cf30 0x7fcf081a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf0da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf081a2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58534/0 (socket says 172.21.15.71:58534) 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf0da00640 1 -- 172.21.15.71:0/1306152814 learned_addr learned my addr 172.21.15.71:0/1306152814 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.450+0000 7fcf0d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf0810cf30 0x7fcf081a33a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34410/0 (socket says 172.21.15.71:34410) 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0e400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf08112640 0x7fcf081a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0da00640 1 -- 172.21.15.71:0/1306152814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf08112640 msgr2=0x7fcf081a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0da00640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf08112640 0x7fcf081a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0da00640 1 -- 172.21.15.71:0/1306152814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf0810cf30 msgr2=0x7fcf081a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0da00640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf0810cf30 0x7fcf081a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0da00640 1 -- 172.21.15.71:0/1306152814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf081a7e10 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0e400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf08112640 0x7fcf081a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.451+0000 7fcf0da00640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf081a2e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009230 tx=0x7fcefc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.452+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc03f030 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.452+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcefc004440 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.452+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf081a80a0 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.452+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcefc035070 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.452+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf081af9e0 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.453+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0810dcb0 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.455+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcefc042030 con 0x7fcf0810c590 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.457+0000 7fcf06000640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcef00a2cf0 0x7fcef00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:16.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.458+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fcefc0efa50 con 0x7fcf0810c590 2024-09-05T12:31:16.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.459+0000 7fcf0d000640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcef00a2cf0 0x7fcef00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:16.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.459+0000 7fcf0d000640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcef00a2cf0 0x7fcef00a51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf0806d350 tx=0x7fcef4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:16.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.465+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcefc0bbce0 con 0x7fcf0810c590 2024-09-05T12:31:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:16 smithi123 ceph-mon[34321]: pgmap v108: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 363 KiB/s wr, 94 op/s 2024-09-05T12:31:16.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.704+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf0810c9f0 con 0x7fcf0810c590 2024-09-05T12:31:16.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.705+0000 7fcf06000640 1 -- 172.21.15.71:0/1306152814 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11975 (secure 0 0 0) 0x7fcefc0bffe0 con 0x7fcf0810c590 2024-09-05T12:31:16.707 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:31:16.707 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"btime":"2024-09-05T12:29:52:485449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:29:51.941852+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:31:16.708 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcef00a2cf0 msgr2=0x7fcef00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcef00a2cf0 0x7fcef00a51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf0806d350 tx=0x7fcef4005f90 comp rx=0 tx=0).stop 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 msgr2=0x7fcf081a2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf081a2e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcefc009230 tx=0x7fcefc004040 comp rx=0 tx=0).stop 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 shutdown_connections 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf08112640 0x7fcf081a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf0810cf30 0x7fcf081a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcef00a2cf0 0x7fcef00a51b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 --2- 172.21.15.71:0/1306152814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf0810c590 0x7fcf081a2e60 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:16.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 >> 172.21.15.71:0/1306152814 conn(0x7fcf0806cdc0 msgr2=0x7fcf0810aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:16.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 shutdown_connections 2024-09-05T12:31:16.711 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:16.708+0000 7fcf10400640 1 -- 172.21.15.71:0/1306152814 wait complete. 2024-09-05T12:31:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:16 smithi174 ceph-mon[34313]: pgmap v108: 129 pgs: 129 active+clean; 35 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 363 KiB/s wr, 94 op/s 2024-09-05T12:31:17.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 99602 1.58 MB/sec execute 13 sec latency 22.943 ms 2024-09-05T12:31:17.580 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:17 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1306152814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:31:17.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:17 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1306152814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:31:17.661 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:17 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1306152814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:31:17.770 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 154.1 --await-for 45.0' 2024-09-05T12:31:17.771 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 154.1 --await-for 45.0 2024-09-05T12:31:18.163 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:31:18.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 100849 1.85 MB/sec execute 14 sec latency 20.331 ms 2024-09-05T12:31:18.566 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:18 smithi071 ceph-mon[29845]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-09-05T12:31:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:18 smithi123 ceph-mon[34321]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-09-05T12:31:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:18 smithi174 ceph-mon[34313]: pgmap v109: 129 pgs: 129 active+clean; 39 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 159 op/s 2024-09-05T12:31:18.836 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 -- 172.21.15.71:0/2603243268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8109140 msgr2=0x7fb0e8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 --2- 172.21.15.71:0/2603243268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8109140 0x7fb0e8110cc0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009ca0 tx=0x7fb0e002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 -- 172.21.15.71:0/2603243268 shutdown_connections 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 --2- 172.21.15.71:0/2603243268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8109140 0x7fb0e8110cc0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 --2- 172.21.15.71:0/2603243268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 0x7fb0e8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 --2- 172.21.15.71:0/2603243268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8104010 0x7fb0e81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 -- 172.21.15.71:0/2603243268 >> 172.21.15.71:0/2603243268 conn(0x7fb0e80ffa50 msgr2=0x7fb0e8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 -- 172.21.15.71:0/2603243268 shutdown_connections 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.815+0000 7fb0f0600640 1 -- 172.21.15.71:0/2603243268 wait complete. 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.816+0000 7fb0f0600640 1 Processor -- start 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.816+0000 7fb0f0600640 1 -- start start 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0f0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 0x7fb0e819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0f0600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 0x7fb0e819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0f0600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8109140 0x7fb0e819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0f0600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8114150 con 0x7fb0e81049c0 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0f0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e8113fd0 con 0x7fb0e8104010 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0f0600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81142d0 con 0x7fb0e8109140 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 0x7fb0e819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 0x7fb0e819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34426/0 (socket says 172.21.15.71:34426) 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 -- 172.21.15.71:0/16286183 learned_addr learned my addr 172.21.15.71:0/16286183 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0ee600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8109140 0x7fb0e819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0ed200640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 0x7fb0e819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 -- 172.21.15.71:0/16286183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8109140 msgr2=0x7fb0e819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8109140 0x7fb0e819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 -- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 msgr2=0x7fb0e819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 0x7fb0e819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0edc00640 1 -- 172.21.15.71:0/16286183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e819f5c0 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.817+0000 7fb0ed200640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 0x7fb0e819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.818+0000 7fb0edc00640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 0x7fb0e819a610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00e9f0 tx=0x7fb0dc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.818+0000 7fb0e6000640 1 -- 172.21.15.71:0/16286183 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc00cd80 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.818+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0e819f8b0 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.818+0000 7fb0e6000640 1 -- 172.21.15.71:0/16286183 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc010910 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.818+0000 7fb0e6000640 1 -- 172.21.15.71:0/16286183 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0188f0 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.818+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0e81a7190 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.821+0000 7fb0e6000640 1 -- 172.21.15.71:0/16286183 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb0dc010ac0 con 0x7fb0e8104010 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.821+0000 7fb0f2fcb740 1 Processor -- start 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.821+0000 7fb0f2fcb740 1 -- start start 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030ca3b0 0x55f0030ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030cacd0 0x55f0030d5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 0x55f0030d7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f003030530 con 0x55f0030ca3b0 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f002fd7860 con 0x55f0030cacd0 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f00302c4f0 con 0x55f0030d5590 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 0x55f0030d7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 0x55f0030d7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40750/0 (socket says 172.21.15.71:40750) 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 -- 172.21.15.71:0/3217031512 learned_addr learned my addr 172.21.15.71:0/3217031512 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ee600640 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030ca3b0 0x55f0030ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0edc00640 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030cacd0 0x55f0030d5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 -- 172.21.15.71:0/3217031512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030cacd0 msgr2=0x55f0030d5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030cacd0 0x55f0030d5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 -- 172.21.15.71:0/3217031512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030ca3b0 msgr2=0x55f0030ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030ca3b0 0x55f0030ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.822+0000 7fb0ed200640 1 -- 172.21.15.71:0/3217031512 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f002da1b90 con 0x55f0030d5590 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0ed200640 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 0x55f0030d7980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400bcb0 tx=0x7fb0d400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510fae2ef4abc055 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0e6000640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb0d00a2d70 0x7fb0d00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0daa00640 1 -- 172.21.15.71:0/3217031512 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4015070 con 0x55f0030d5590 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0e6000640 1 -- 172.21.15.71:0/16286183 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb0dc0cfb80 con 0x7fb0e8104010 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0daa00640 1 -- 172.21.15.71:0/3217031512 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d4011920 con 0x55f0030d5590 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0daa00640 1 -- 172.21.15.71:0/3217031512 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4010910 con 0x55f0030d5590 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.823+0000 7fb0ee600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb0d00a2d70 0x7fb0d00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/3217031512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 msgr2=0x55f0030d7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0ee600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb0d00a2d70 0x7fb0d00a5230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0004af0 tx=0x7fb0e0004a80 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 0x55f0030d7980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb0d400bcb0 tx=0x7fb0d400f730 comp rx=0 tx=0).stop 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/3217031512 shutdown_connections 2024-09-05T12:31:18.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030d5590 0x55f0030d7980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030cacd0 0x55f0030d5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/3217031512 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030ca3b0 0x55f0030ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/3217031512 >> 172.21.15.71:0/3217031512 conn(0x55f0030c6940 msgr2=0x55f0030c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/3217031512 shutdown_connections 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.824+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/3217031512 wait complete. 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 -- start start 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 Processor -- start 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 0x55f0030d2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030cacd0 0x55f0030d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030d5590 0x55f0030d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f0030d8070 con 0x55f0030d5590 2024-09-05T12:31:18.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f002feae60 con 0x55f0030ca3b0 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f0030d81f0 con 0x55f0030cacd0 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0f2fcb740 10 client.0 _ll_register_callbacks cb 0x55f002fefcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 0x55f0030d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 0x55f0030d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34430/0 (socket says 172.21.15.71:34430) 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0edc00640 1 -- 172.21.15.71:0/1819128346 learned_addr learned my addr 172.21.15.71:0/1819128346 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030cacd0 0x55f0030d2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.825+0000 7fb0ee600640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030d5590 0x55f0030d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0edc00640 1 -- 172.21.15.71:0/1819128346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030cacd0 msgr2=0x55f0030d2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030cacd0 0x55f0030d2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0edc00640 1 -- 172.21.15.71:0/1819128346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030d5590 msgr2=0x55f0030d2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030d5590 0x55f0030d2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0edc00640 1 -- 172.21.15.71:0/1819128346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f002ff6990 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0ed200640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030cacd0 0x55f0030d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 0x55f0030d2570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00e9c0 tx=0x7fb0dc09b6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b38ec1c3f4795c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0cf400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0cf400640 1 -- 172.21.15.71:0/1819128346 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0a0110 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d4011920 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.826+0000 7fb0cf400640 1 -- 172.21.15.71:0/1819128346 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb0dc0983d0 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0cf400640 1 -- 172.21.15.71:0/1819128346 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc0a49d0 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 10 client.25832 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0030e3e50 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0cf400640 1 -- 172.21.15.71:0/1819128346 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d4011920 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0cf400640 1 -- 172.21.15.71:0/1819128346 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb0dc097bd0 con 0x55f0030ca3b0 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 10 client.25832 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 10 client.25832 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:31:18.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 20 client.25832 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f0030e52b0 0x55f0030e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 4 client.25832 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f002fd3310 con 0x55f0030e52b0 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.827+0000 7fb0ed200640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f0030e52b0 0x55f0030e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.828+0000 7fb0ed200640 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f0030e52b0 0x55f0030e7730 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17b95a70b507a84 server_cookie=a59bf5fb2df7429e in_seq=0 out_seq=0 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.828+0000 7fb0cf400640 10 client.25832 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0cf400640 1 -- 172.21.15.71:0/1819128346 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f002fd3310 con 0x55f0030e52b0 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0cf400640 10 client.25832 handle_command_reply: tid=0 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 client.25832 shutdown 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 msgr2=0x55f0030d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 0x55f0030d2570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00e9c0 tx=0x7fb0dc09b6c0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 shutdown_connections 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f0030e52b0 0x55f0030e7730 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030d5590 0x55f0030d2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030cacd0 0x55f0030d2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1819128346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030ca3b0 0x55f0030d2570 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0cf400640 0 client.25832 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 >> 172.21.15.71:0/1819128346 conn(0x55f0030c6940 msgr2=0x55f0030c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 shutdown_connections 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.829+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1819128346 wait complete. 2024-09-05T12:31:18.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.830+0000 7fb0f2fcb740 20 client.25832 trim_cache size 0 max 16384 2024-09-05T12:31:18.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 Processor -- start 2024-09-05T12:31:18.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 -- start start 2024-09-05T12:31:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f0030ed540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81b11e0 con 0x55f0030cacd0 2024-09-05T12:31:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81b2de0 con 0x55f0030d5590 2024-09-05T12:31:18.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81b18b0 con 0x55f0030ca3b0 2024-09-05T12:31:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.852+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40758/0 (socket says 172.21.15.71:40758) 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 -- 172.21.15.71:0/1560430170 learned_addr learned my addr 172.21.15.71:0/1560430170 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030ed000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58584/0 (socket says 172.21.15.71:58584) 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0ee600640 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f0030ed540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 -- 172.21.15.71:0/1560430170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 msgr2=0x55f0030ed540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f0030ed540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 -- 172.21.15.71:0/1560430170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 msgr2=0x55f0030ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 -- 172.21.15.71:0/1560430170 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0030c7400 con 0x55f0030ca3b0 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d0e40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c6a0 tx=0x7fb0dc00c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd32359081efd03d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.853+0000 7fb0da000640 1 -- 172.21.15.71:0/1560430170 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc09c460 con 0x55f0030ca3b0 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0da000640 1 -- 172.21.15.71:0/1560430170 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc09a060 con 0x55f0030ca3b0 2024-09-05T12:31:18.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0da000640 1 -- 172.21.15.71:0/1560430170 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc09b430 con 0x55f0030ca3b0 2024-09-05T12:31:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1560430170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 msgr2=0x55f0030d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d0e40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c6a0 tx=0x7fb0dc00c7d0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1560430170 shutdown_connections 2024-09-05T12:31:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f0030ed540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1560430170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d0e40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.854+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1560430170 >> 172.21.15.71:0/1560430170 conn(0x55f0030c6940 msgr2=0x55f0030c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.855+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1560430170 shutdown_connections 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.855+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1560430170 wait complete. 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.855+0000 7fb0f2fcb740 1 -- start start 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.855+0000 7fb0f2fcb740 1 Processor -- start 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f00314efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81b2650 con 0x55f0030cacd0 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81b1f80 con 0x55f0030d5590 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0e81af1c0 con 0x55f0030ca3b0 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0f2fcb740 10 client.0 _ll_register_callbacks cb 0x7fb0dc00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:18.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40770/0 (socket says 172.21.15.71:40770) 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0ed200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030d9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58598/0 (socket says 172.21.15.71:58598) 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0ed200640 1 -- 172.21.15.71:0/1738704242 learned_addr learned my addr 172.21.15.71:0/1738704242 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0ee600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f00314efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 -- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 msgr2=0x55f00314efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f00314efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 -- 172.21.15.71:0/1738704242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 msgr2=0x55f0030d9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030d9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.856+0000 7fb0edc00640 1 -- 172.21.15.71:0/1738704242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0030d1520 con 0x55f0030ca3b0 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0edc00640 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d9a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0cd590 tx=0x7fb0dc00c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ebbd3ac5f9dccb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:18.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0cf400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:31:18.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0cf400640 1 -- 172.21.15.71:0/1738704242 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc018c90 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0dc09a060 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0cf400640 1 -- 172.21.15.71:0/1738704242 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fb0dc010430 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0cf400640 1 -- 172.21.15.71:0/1738704242 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb0dc010670 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0f2fcb740 10 client.25849 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f0030d1520 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.857+0000 7fb0cf400640 1 -- 172.21.15.71:0/1738704242 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0dc09a060 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0cf400640 1 -- 172.21.15.71:0/1738704242 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fb0dc086a20 con 0x55f0030ca3b0 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0f2fcb740 10 client.25849 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0f2fcb740 10 client.25849 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0f2fcb740 20 client.25849 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f003150a00 0x55f003152e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:18.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0f2fcb740 4 client.25849 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.1, "await_for": 45.0}] 2024-09-05T12:31:18.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.1, "await_for": 45.0}) -- 0x55f00303dc60 con 0x55f003150a00 2024-09-05T12:31:18.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0ed200640 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f003150a00 0x55f003152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:18.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0cf400640 10 client.25849 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:18.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.858+0000 7fb0ed200640 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f003150a00 0x55f003152e20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e0109a4343d5e13 server_cookie=e0e6763c71229a2b in_seq=0 out_seq=0 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.912+0000 7fb0cf400640 1 -- 172.21.15.71:0/1738704242 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f00303dc60 con 0x55f003150a00 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.912+0000 7fb0cf400640 10 client.25849 handle_command_reply: tid=0 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.912+0000 7fb0f2fcb740 1 client.25849 shutdown 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 msgr2=0x55f0030d9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d9a50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0cd590 tx=0x7fb0dc00c840 comp rx=0 tx=0).stop 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 shutdown_connections 2024-09-05T12:31:18.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f003150a00 0x55f003152e20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0030d5590 0x55f00314efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0cf400640 0 client.25849 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f0030cacd0 0x55f0030d9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 --2- 172.21.15.71:0/1738704242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0030ca3b0 0x55f0030d9a50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 >> 172.21.15.71:0/1738704242 conn(0x55f0030c6940 msgr2=0x55f0030cbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 shutdown_connections 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 1 -- 172.21.15.71:0/1738704242 wait complete. 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.913+0000 7fb0f2fcb740 20 client.25849 trim_cache size 0 max 16384 2024-09-05T12:31:18.916 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 32, 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 14, 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "eb2057bd": { 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 14, 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:31:18.917 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 154.1, 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:18.918 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:31:18.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb0d00a2d70 msgr2=0x7fb0d00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb0d00a2d70 0x7fb0d00a5230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0004af0 tx=0x7fb0e0004a80 comp rx=0 tx=0).stop 2024-09-05T12:31:18.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 msgr2=0x7fb0e819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:18.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 0x7fb0e819a610 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00e9f0 tx=0x7fb0dc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 shutdown_connections 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0e8109140 0x7fb0e819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb0d00a2d70 0x7fb0d00a5230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0e81049c0 0x7fb0e819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 --2- 172.21.15.71:0/16286183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0e8104010 0x7fb0e819a610 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 >> 172.21.15.71:0/16286183 conn(0x7fb0e80ffa50 msgr2=0x7fb0e8100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 shutdown_connections 2024-09-05T12:31:18.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:18.914+0000 7fb0f0600640 1 -- 172.21.15.71:0/16286183 wait complete. 2024-09-05T12:31:19.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 2.35 MB/sec execute 15 sec latency 340.129 ms 2024-09-05T12:31:19.959 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eb2057bd', quiesce took 4.8 seconds. Will release after: 29.3 2024-09-05T12:31:20.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 2.20 MB/sec execute 16 sec latency 1340.257 ms 2024-09-05T12:31:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:20 smithi071 ceph-mon[29845]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-09-05T12:31:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:20 smithi123 ceph-mon[34321]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-09-05T12:31:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:20 smithi174 ceph-mon[34313]: pgmap v110: 129 pgs: 129 active+clean; 39 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 1.1 MiB/s wr, 74 op/s 2024-09-05T12:31:21.200 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 2.07 MB/sec execute 17 sec latency 2340.376 ms 2024-09-05T12:31:22.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.96 MB/sec execute 18 sec latency 3340.463 ms 2024-09-05T12:31:22.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:22 smithi071 ceph-mon[29845]: pgmap v111: 129 pgs: 129 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-09-05T12:31:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:22 smithi123 ceph-mon[34321]: pgmap v111: 129 pgs: 129 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-09-05T12:31:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:22 smithi174 ceph-mon[34313]: pgmap v111: 129 pgs: 129 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.5 MiB/s wr, 144 op/s 2024-09-05T12:31:23.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.85 MB/sec execute 19 sec latency 4340.585 ms 2024-09-05T12:31:24.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.76 MB/sec execute 20 sec latency 5340.727 ms 2024-09-05T12:31:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:24 smithi071 ceph-mon[29845]: pgmap v112: 129 pgs: 129 active+clean; 41 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-05T12:31:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:24 smithi123 ceph-mon[34321]: pgmap v112: 129 pgs: 129 active+clean; 41 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-05T12:31:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:24 smithi174 ceph-mon[34313]: pgmap v112: 129 pgs: 129 active+clean; 41 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-05T12:31:25.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.68 MB/sec execute 21 sec latency 6340.798 ms 2024-09-05T12:31:26.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.60 MB/sec execute 22 sec latency 7340.921 ms 2024-09-05T12:31:26.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:26 smithi071 ceph-mon[29845]: pgmap v113: 129 pgs: 129 active+clean; 41 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-05T12:31:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:26 smithi123 ceph-mon[34321]: pgmap v113: 129 pgs: 129 active+clean; 41 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-05T12:31:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:26 smithi174 ceph-mon[34313]: pgmap v113: 129 pgs: 129 active+clean; 41 MiB data, 646 MiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-05T12:31:27.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.53 MB/sec execute 23 sec latency 8341.066 ms 2024-09-05T12:31:27.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 -- 172.21.15.71:0/3154885628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 msgr2=0x7f18b8109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 --2- 172.21.15.71:0/3154885628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b8109f60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009930 tx=0x7f18a802a420 comp rx=0 tx=0).stop 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 -- 172.21.15.71:0/3154885628 shutdown_connections 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 --2- 172.21.15.71:0/3154885628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b8104480 0x7f18b81109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 --2- 172.21.15.71:0/3154885628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b8103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 --2- 172.21.15.71:0/3154885628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b8109f60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.883+0000 7f18bce00640 1 -- 172.21.15.71:0/3154885628 >> 172.21.15.71:0/3154885628 conn(0x7f18b80ff7c0 msgr2=0x7f18b8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.884+0000 7f18bce00640 1 -- 172.21.15.71:0/3154885628 shutdown_connections 2024-09-05T12:31:27.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.884+0000 7f18bce00640 1 -- 172.21.15.71:0/3154885628 wait complete. 2024-09-05T12:31:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.884+0000 7f18bce00640 1 Processor -- start 2024-09-05T12:31:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 -- start start 2024-09-05T12:31:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b8104480 0x7f18b819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8113f30 con 0x7f18b8109b80 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18b8113db0 con 0x7f18b8103ae0 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18bce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81140b0 con 0x7f18b8104480 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35412/0 (socket says 172.21.15.71:35412) 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18b7400640 1 -- 172.21.15.71:0/2158913960 learned_addr learned my addr 172.21.15.71:0/2158913960 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.885+0000 7f18b6a00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b8104480 0x7f18b819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7400640 1 -- 172.21.15.71:0/2158913960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b8104480 msgr2=0x7f18b819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7400640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b8104480 0x7f18b819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7400640 1 -- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 msgr2=0x7f18b819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7400640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7400640 1 -- 172.21.15.71:0/2158913960 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b819f430 con 0x7f18b8103ae0 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7e00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18b7400640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b819a440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009230 tx=0x7f18a8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18afe00640 1 -- 172.21.15.71:0/2158913960 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a803f030 con 0x7f18b8103ae0 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18afe00640 1 -- 172.21.15.71:0/2158913960 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a8003ad0 con 0x7f18b8103ae0 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b819f6c0 con 0x7f18b8103ae0 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.886+0000 7f18afe00640 1 -- 172.21.15.71:0/2158913960 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8035070 con 0x7f18b8103ae0 2024-09-05T12:31:27.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.887+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b81a6fb0 con 0x7f18b8103ae0 2024-09-05T12:31:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.889+0000 7f18c1239740 1 Processor -- start 2024-09-05T12:31:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18afe00640 1 -- 172.21.15.71:0/2158913960 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f18a8042030 con 0x7f18b8103ae0 2024-09-05T12:31:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 -- start start 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 0x55bebf2f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bebf24a100 con 0x55bebf2e4cd0 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bebf1f21b0 con 0x55bebf2ef570 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18c1239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bebf2460c0 con 0x55bebf2e43b0 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18b7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18b6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 0x55bebf2f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18b6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 0x55bebf2f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35422/0 (socket says 172.21.15.71:35422) 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18b6a00640 1 -- 172.21.15.71:0/2729806054 learned_addr learned my addr 172.21.15.71:0/2729806054 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.890+0000 7f18b7e00640 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18b6a00640 1 -- 172.21.15.71:0/2729806054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 msgr2=0x55bebf2e4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18b6a00640 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2e4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18b6a00640 1 -- 172.21.15.71:0/2729806054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 msgr2=0x55bebf2ef030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18b6a00640 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ef030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18b6a00640 1 -- 172.21.15.71:0/2729806054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebef5d210 con 0x55bebf2ef570 2024-09-05T12:31:27.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18b6a00640 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 0x55bebf2f1960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009c90 tx=0x7f18a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37a7f7630a5931c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18afe00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f189c0a2cf0 0x7f189c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18acc00640 1 -- 172.21.15.71:0/2729806054 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a401e070 con 0x55bebf2ef570 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18acc00640 1 -- 172.21.15.71:0/2729806054 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a4019b10 con 0x55bebf2ef570 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18acc00640 1 -- 172.21.15.71:0/2729806054 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a400eac0 con 0x55bebf2ef570 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.891+0000 7f18afe00640 1 -- 172.21.15.71:0/2158913960 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f18a80efd80 con 0x7f18b8103ae0 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18b7e00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f189c0a2cf0 0x7f189c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 -- 172.21.15.71:0/2729806054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 msgr2=0x55bebf2f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 0x55bebf2f1960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f18a4009c90 tx=0x7f18a4011810 comp rx=0 tx=0).stop 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 -- 172.21.15.71:0/2729806054 shutdown_connections 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2ef570 0x55bebf2f1960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ef030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 --2- 172.21.15.71:0/2729806054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2e4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 -- 172.21.15.71:0/2729806054 >> 172.21.15.71:0/2729806054 conn(0x55bebf2e0920 msgr2=0x55bebf2e0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18b7e00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f189c0a2cf0 0x7f189c0a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18b0005fd0 tx=0x7f18b0009600 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 -- 172.21.15.71:0/2729806054 shutdown_connections 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.892+0000 7f18c1239740 1 -- 172.21.15.71:0/2729806054 wait complete. 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 -- start start 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 Processor -- start 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2ec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf2ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bebf2f1ff0 con 0x55bebf2ef570 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bebf1d3090 con 0x55bebf2e43b0 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.893+0000 7f18c1239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bebf2f2170 con 0x55bebf2e4cd0 2024-09-05T12:31:27.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18c1239740 10 client.0 _ll_register_callbacks cb 0x55bebf227f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf2ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ecb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45220/0 (socket says 172.21.15.71:45220) 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf2ed080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43546/0 (socket says 172.21.15.71:43546) 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b6a00640 1 -- 172.21.15.71:0/3545417620 learned_addr learned my addr 172.21.15.71:0/3545417620 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7400640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 -- 172.21.15.71:0/3545417620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 msgr2=0x55bebf2ecb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ecb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 -- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 msgr2=0x55bebf2ec600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2ec600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b7e00640 1 -- 172.21.15.71:0/3545417620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebf029620 con 0x55bebf2ef570 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.894+0000 7f18b6a00640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:27.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18b7e00640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf2ed080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f18b0013830 tx=0x7f18b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830d99dbd7ac1deb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18a1600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18a1600640 1 -- 172.21.15.71:0/3545417620 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b002b060 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a4019b10 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18a1600640 1 -- 172.21.15.71:0/3545417620 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f18b001c670 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18a1600640 1 -- 172.21.15.71:0/3545417620 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18b0032050 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18c1239740 10 client.15816 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bebf2fdf00 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.895+0000 7f18a1600640 1 -- 172.21.15.71:0/3545417620 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a4019b10 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18a1600640 1 -- 172.21.15.71:0/3545417620 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f18b001a040 con 0x55bebf2ef570 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18c1239740 10 client.15816 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18c1239740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18c1239740 20 client.15816 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:27.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18c1239740 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf2ff480 0x55bebf301920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18c1239740 4 client.15816 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bebf257830 con 0x55bebf2ff480 2024-09-05T12:31:27.902 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.896+0000 7f18b7400640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf2ff480 0x55bebf301920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.897+0000 7f18b7400640 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf2ff480 0x55bebf301920 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7fb4c4fa6b47d92 server_cookie=3d905166255fbd94 in_seq=0 out_seq=0 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.897+0000 7f18a1600640 10 client.15816 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.897+0000 7f18a1600640 1 -- 172.21.15.71:0/3545417620 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bebf257830 con 0x55bebf2ff480 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.897+0000 7f18a1600640 10 client.15816 handle_command_reply: tid=0 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.897+0000 7f18c1239740 1 client.15816 shutdown 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 msgr2=0x55bebf2ed080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf2ed080 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f18b0013830 tx=0x7f18b0004a40 comp rx=0 tx=0).stop 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 shutdown_connections 2024-09-05T12:31:27.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf2ff480 0x55bebf301920 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf2ed080 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e4cd0 0x55bebf2ecb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 --2- 172.21.15.71:0/3545417620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18a1600640 0 client.15816 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 >> 172.21.15.71:0/3545417620 conn(0x55bebf2e0920 msgr2=0x55bebf2e6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 shutdown_connections 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 1 -- 172.21.15.71:0/3545417620 wait complete. 2024-09-05T12:31:27.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.898+0000 7f18c1239740 20 client.15816 trim_cache size 0 max 16384 2024-09-05T12:31:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.913+0000 7f18c1239740 1 Processor -- start 2024-09-05T12:31:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.913+0000 7f18c1239740 1 -- start start 2024-09-05T12:31:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.913+0000 7f18c1239740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.913+0000 7f18c1239740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf307230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18c1239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2ef570 0x55bebf307770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18c1239740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81b0eb0 con 0x55bebf2e4cd0 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18c1239740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81b2ab0 con 0x55bebf2e43b0 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18c1239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81b1580 con 0x55bebf2ef570 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf307230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2eb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35448/0 (socket says 172.21.15.71:35448) 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 -- 172.21.15.71:0/18863532 learned_addr learned my addr 172.21.15.71:0/18863532 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7e00640 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2ef570 0x55bebf307770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 -- 172.21.15.71:0/18863532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2ef570 msgr2=0x55bebf307770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2ef570 0x55bebf307770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 -- 172.21.15.71:0/18863532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 msgr2=0x55bebf307230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf307230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 -- 172.21.15.71:0/18863532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bebf2eab30 con 0x55bebf2e43b0 2024-09-05T12:31:27.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.914+0000 7f18b7400640 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2eb5c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18a80b8980 tx=0x7f18a80dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841d6960bd6f2a74 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18a3e00640 1 -- 172.21.15.71:0/18863532 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80f1040 con 0x55bebf2e43b0 2024-09-05T12:31:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18a3e00640 1 -- 172.21.15.71:0/18863532 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18a803b460 con 0x55bebf2e43b0 2024-09-05T12:31:27.918 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18a3e00640 1 -- 172.21.15.71:0/18863532 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a8049040 con 0x55bebf2e43b0 2024-09-05T12:31:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 -- 172.21.15.71:0/18863532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 msgr2=0x55bebf2eb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2eb5c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18a80b8980 tx=0x7f18a80dc040 comp rx=0 tx=0).stop 2024-09-05T12:31:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 -- 172.21.15.71:0/18863532 shutdown_connections 2024-09-05T12:31:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2ef570 0x55bebf307770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2e4cd0 0x55bebf307230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 --2- 172.21.15.71:0/18863532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e43b0 0x55bebf2eb5c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 -- 172.21.15.71:0/18863532 >> 172.21.15.71:0/18863532 conn(0x55bebf2e0920 msgr2=0x55bebf2eaeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 -- 172.21.15.71:0/18863532 shutdown_connections 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.915+0000 7f18c1239740 1 -- 172.21.15.71:0/18863532 wait complete. 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 -- start start 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 Processor -- start 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e4cd0 0x55bebf2f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf3681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81b2320 con 0x55bebf2ef570 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81b1c50 con 0x55bebf2e4cd0 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b81aee90 con 0x55bebf2e43b0 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.916+0000 7f18c1239740 10 client.0 _ll_register_callbacks cb 0x7f18b0011500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:27.922 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf3681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2f2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45238/0 (socket says 172.21.15.71:45238) 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7400640 1 -- 172.21.15.71:0/3523944196 learned_addr learned my addr 172.21.15.71:0/3523944196 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b6a00640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e4cd0 0x55bebf2f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 -- 172.21.15.71:0/3523944196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 msgr2=0x55bebf2f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 -- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e4cd0 msgr2=0x55bebf2f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e4cd0 0x55bebf2f2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 -- 172.21.15.71:0/3523944196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bebf368830 con 0x55bebf2ef570 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b6a00640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e4cd0 0x55bebf2f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7400640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18b7e00640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf3681f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f18b0026ce0 tx=0x7f18b00115b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddc1bde83e4b329 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18a1600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:31:27.923 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.917+0000 7f18a1600640 1 -- 172.21.15.71:0/3523944196 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b00263e0 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bebf251b80 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18a1600640 1 -- 172.21.15.71:0/3523944196 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f18b0026580 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18a1600640 1 -- 172.21.15.71:0/3523944196 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f18b0044050 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 10 client.15822 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bebf369b40 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18a1600640 1 -- 172.21.15.71:0/3523944196 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bebf251b80 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18a1600640 1 -- 172.21.15.71:0/3523944196 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f18b001a040 con 0x55bebf2ef570 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 10 client.15822 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 20 client.15822 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf36a320 0x55bebf36c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:27.924 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 4 client.15822 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:31:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.918+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bebf1ce7c0 con 0x55bebf36a320 2024-09-05T12:31:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.919+0000 7f18b7400640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf36a320 0x55bebf36c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.919+0000 7f18b7400640 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf36a320 0x55bebf36c740 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b86d34cf31adfbd server_cookie=5d7087bb97fedd7 in_seq=0 out_seq=0 2024-09-05T12:31:27.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.919+0000 7f18a1600640 10 client.15822 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18a1600640 1 -- 172.21.15.71:0/3523944196 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bebf1ce7c0 con 0x55bebf36a320 2024-09-05T12:31:27.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18a1600640 10 client.15822 handle_command_reply: tid=0 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 client.15822 shutdown 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 msgr2=0x55bebf3681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf3681f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f18b0026ce0 tx=0x7f18b00115b0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 shutdown_connections 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bebf36a320 0x55bebf36c740 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bebf2ef570 0x55bebf3681f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bebf2e4cd0 0x55bebf2f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 --2- 172.21.15.71:0/3523944196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bebf2e43b0 0x55bebf2f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18a1600640 0 client.15822 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.955+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 >> 172.21.15.71:0/3523944196 conn(0x55bebf2e0920 msgr2=0x55bebf2eb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 shutdown_connections 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18c1239740 1 -- 172.21.15.71:0/3523944196 wait complete. 2024-09-05T12:31:27.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18c1239740 20 client.15822 trim_cache size 0 max 16384 2024-09-05T12:31:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:31:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:31:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:31:27.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f189c0a2cf0 msgr2=0x7f189c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18bce00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f189c0a2cf0 0x7f189c0a51b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f18b0005fd0 tx=0x7f18b0009600 comp rx=0 tx=0).stop 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 msgr2=0x7f18b819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.956+0000 7f18bce00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b819a440 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18a8009230 tx=0x7f18a8004270 comp rx=0 tx=0).stop 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 shutdown_connections 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f189c0a2cf0 0x7f189c0a51b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18b8109b80 0x7f18b819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b8104480 0x7f18b819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 --2- 172.21.15.71:0/2158913960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18b8103ae0 0x7f18b819a440 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 >> 172.21.15.71:0/2158913960 conn(0x7f18b80ff7c0 msgr2=0x7f18b8101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 shutdown_connections 2024-09-05T12:31:27.960 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:27.957+0000 7f18bce00640 1 -- 172.21.15.71:0/2158913960 wait complete. 2024-09-05T12:31:28.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.47 MB/sec execute 24 sec latency 9341.206 ms 2024-09-05T12:31:28.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:28 smithi071 ceph-mon[29845]: pgmap v114: 129 pgs: 129 active+clean; 41 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-05T12:31:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:28 smithi123 ceph-mon[34321]: pgmap v114: 129 pgs: 129 active+clean; 41 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-05T12:31:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:28 smithi174 ceph-mon[34313]: pgmap v114: 129 pgs: 129 active+clean; 41 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.5 MiB/s wr, 148 op/s 2024-09-05T12:31:29.201 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.41 MB/sec execute 25 sec latency 10341.285 ms 2024-09-05T12:31:30.202 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.35 MB/sec execute 26 sec latency 11341.409 ms 2024-09-05T12:31:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:30 smithi071 ceph-mon[29845]: pgmap v115: 129 pgs: 129 active+clean; 41 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 440 KiB/s wr, 83 op/s 2024-09-05T12:31:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:30 smithi123 ceph-mon[34321]: pgmap v115: 129 pgs: 129 active+clean; 41 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 440 KiB/s wr, 83 op/s 2024-09-05T12:31:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:30 smithi174 ceph-mon[34313]: pgmap v115: 129 pgs: 129 active+clean; 41 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 440 KiB/s wr, 83 op/s 2024-09-05T12:31:31.202 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.30 MB/sec execute 27 sec latency 12341.523 ms 2024-09-05T12:31:32.202 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.26 MB/sec execute 28 sec latency 13341.643 ms 2024-09-05T12:31:32.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:32 smithi071 ceph-mon[29845]: pgmap v116: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 465 KiB/s wr, 99 op/s 2024-09-05T12:31:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:32 smithi123 ceph-mon[34321]: pgmap v116: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 465 KiB/s wr, 99 op/s 2024-09-05T12:31:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:32 smithi174 ceph-mon[34313]: pgmap v116: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 465 KiB/s wr, 99 op/s 2024-09-05T12:31:33.202 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.21 MB/sec execute 29 sec latency 14341.790 ms 2024-09-05T12:31:34.202 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.17 MB/sec execute 30 sec latency 15341.931 ms 2024-09-05T12:31:34.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:34 smithi071 ceph-mon[29845]: pgmap v117: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-05T12:31:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:34 smithi123 ceph-mon[34321]: pgmap v117: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-05T12:31:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:34 smithi174 ceph-mon[34313]: pgmap v117: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-09-05T12:31:35.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.14 MB/sec execute 31 sec latency 16342.083 ms 2024-09-05T12:31:36.202 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.10 MB/sec execute 32 sec latency 17342.133 ms 2024-09-05T12:31:36.610 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:36 smithi071 ceph-mon[29845]: pgmap v118: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 28 op/s 2024-09-05T12:31:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:36 smithi123 ceph-mon[34321]: pgmap v118: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 28 op/s 2024-09-05T12:31:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:36 smithi174 ceph-mon[34313]: pgmap v118: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 43 KiB/s wr, 28 op/s 2024-09-05T12:31:37.203 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.07 MB/sec execute 33 sec latency 18342.615 ms 2024-09-05T12:31:38.203 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.04 MB/sec execute 34 sec latency 19342.753 ms 2024-09-05T12:31:38.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:38 smithi071 ceph-mon[29845]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 33 op/s 2024-09-05T12:31:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:38 smithi123 ceph-mon[34321]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 33 op/s 2024-09-05T12:31:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:38 smithi174 ceph-mon[34313]: pgmap v119: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 46 KiB/s wr, 33 op/s 2024-09-05T12:31:39.203 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 1.01 MB/sec execute 35 sec latency 20342.859 ms 2024-09-05T12:31:40.203 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.98 MB/sec execute 36 sec latency 21342.978 ms 2024-09-05T12:31:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:40 smithi071 ceph-mon[29845]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-05T12:31:40.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:40 smithi123 ceph-mon[34321]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-05T12:31:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:40 smithi174 ceph-mon[34313]: pgmap v120: 129 pgs: 129 active+clean; 37 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 25 op/s 2024-09-05T12:31:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 -- 172.21.15.71:0/834069989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c103af0 msgr2=0x7fc82c103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 --2- 172.21.15.71:0/834069989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c103af0 0x7fc82c103f50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802a410 comp rx=0 tx=0).stop 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 -- 172.21.15.71:0/834069989 shutdown_connections 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 --2- 172.21.15.71:0/834069989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c104490 0x7fc82c1109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 --2- 172.21.15.71:0/834069989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c103af0 0x7fc82c103f50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 --2- 172.21.15.71:0/834069989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c109b90 0x7fc82c109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 -- 172.21.15.71:0/834069989 >> 172.21.15.71:0/834069989 conn(0x7fc82c0ff7d0 msgr2=0x7fc82c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 -- 172.21.15.71:0/834069989 shutdown_connections 2024-09-05T12:31:41.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.079+0000 7fc831800640 1 -- 172.21.15.71:0/834069989 wait complete. 2024-09-05T12:31:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.080+0000 7fc831800640 1 Processor -- start 2024-09-05T12:31:41.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.080+0000 7fc831800640 1 -- start start 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc831800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 0x7fc82c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc831800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c104490 0x7fc82c19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc831800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c109b90 0x7fc82c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc831800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c113f40 con 0x7fc82c109b90 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc831800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c113dc0 con 0x7fc82c104490 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc831800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1140c0 con 0x7fc82c103af0 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 0x7fc82c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc830e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c109b90 0x7fc82c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 0x7fc82c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33188/0 (socket says 172.21.15.71:33188) 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 -- 172.21.15.71:0/939466088 learned_addr learned my addr 172.21.15.71:0/939466088 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82b400640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c104490 0x7fc82c19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 -- 172.21.15.71:0/939466088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c104490 msgr2=0x7fc82c19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c104490 0x7fc82c19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 -- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c109b90 msgr2=0x7fc82c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c109b90 0x7fc82c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc82be00640 1 -- 172.21.15.71:0/939466088 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c19f420 con 0x7fc82c103af0 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.081+0000 7fc830e00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c109b90 0x7fc82c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.082+0000 7fc82be00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 0x7fc82c19a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e9f0 tx=0x7fc81c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.082+0000 7fc828c00640 1 -- 172.21.15.71:0/939466088 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00ccc0 con 0x7fc82c103af0 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.082+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c19f710 con 0x7fc82c103af0 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.082+0000 7fc828c00640 1 -- 172.21.15.71:0/939466088 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81c00ce60 con 0x7fc82c103af0 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.082+0000 7fc828c00640 1 -- 172.21.15.71:0/939466088 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c010690 con 0x7fc82c103af0 2024-09-05T12:31:41.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.082+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c1a6fa0 con 0x7fc82c103af0 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc835c58740 1 Processor -- start 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc835c58740 1 -- start start 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc828c00640 1 -- 172.21.15.71:0/939466088 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc81c0046c0 con 0x7fc82c103af0 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc835c58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e733a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc835c58740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7345030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc835c58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7347960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.085+0000 7fc835c58740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c4e72a0100 con 0x55c4e733acd0 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc835c58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c4e72481b0 con 0x55c4e733a3b0 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc835c58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4e729c0c0 con 0x55c4e7345570 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7345030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7347960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc830e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e733a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7345030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42244/0 (socket says 172.21.15.71:42244) 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82be00640 1 -- 172.21.15.71:0/1803472091 learned_addr learned my addr 172.21.15.71:0/1803472091 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82b400640 1 -- 172.21.15.71:0/1803472091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 msgr2=0x55c4e733a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82b400640 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e733a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82b400640 1 -- 172.21.15.71:0/1803472091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 msgr2=0x55c4e7345030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82b400640 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7345030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.086+0000 7fc82b400640 1 -- 172.21.15.71:0/1803472091 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e6fb3210 con 0x55c4e7345570 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc82b400640 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7347960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc818005940 tx=0x7fc818005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de61933636b10e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc828c00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8100a2cf0 0x7fc8100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc821600640 1 -- 172.21.15.71:0/1803472091 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818035030 con 0x55c4e7345570 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc821600640 1 -- 172.21.15.71:0/1803472091 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc818040070 con 0x55c4e7345570 2024-09-05T12:31:41.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc821600640 1 -- 172.21.15.71:0/1803472091 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818004d90 con 0x55c4e7345570 2024-09-05T12:31:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc828c00640 1 -- 172.21.15.71:0/939466088 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc81c0cfb50 con 0x7fc82c103af0 2024-09-05T12:31:41.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.087+0000 7fc830e00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8100a2cf0 0x7fc8100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 -- 172.21.15.71:0/1803472091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 msgr2=0x55c4e7347960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7347960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc818005940 tx=0x7fc818005dd0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc830e00640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8100a2cf0 0x7fc8100a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc824005fd0 tx=0x7fc824009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 -- 172.21.15.71:0/1803472091 shutdown_connections 2024-09-05T12:31:41.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7347960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7345030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 --2- 172.21.15.71:0/1803472091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e733a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 -- 172.21.15.71:0/1803472091 >> 172.21.15.71:0/1803472091 conn(0x55c4e7336920 msgr2=0x55c4e7336d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 -- 172.21.15.71:0/1803472091 shutdown_connections 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 -- 172.21.15.71:0/1803472091 wait complete. 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.088+0000 7fc835c58740 1 -- start start 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 Processor -- start 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 0x55c4e7342560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e7342aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e7345570 0x55c4e7342fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7347ff0 con 0x55c4e7345570 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7229090 con 0x55c4e733acd0 2024-09-05T12:31:41.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c4e7348170 con 0x55c4e733a3b0 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc835c58740 10 client.0 _ll_register_callbacks cb 0x55c4e727df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc830e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e7345570 0x55c4e7342fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc82be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 0x55c4e7342560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc82b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e7342aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc82be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 0x55c4e7342560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33208/0 (socket says 172.21.15.71:33208) 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.089+0000 7fc82be00640 1 -- 172.21.15.71:0/1790468929 learned_addr learned my addr 172.21.15.71:0/1790468929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc82be00640 1 -- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 msgr2=0x55c4e7342aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc82be00640 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e7342aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc82be00640 1 -- 172.21.15.71:0/1790468929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e7345570 msgr2=0x55c4e7342fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc82be00640 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e7345570 0x55c4e7342fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc82be00640 1 -- 172.21.15.71:0/1790468929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e707f620 con 0x55c4e733a3b0 2024-09-05T12:31:41.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc82be00640 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 0x55c4e7342560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc81c09bc60 tx=0x7fc81c097f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e82b5d134d2d60c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc816000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc816000640 1 -- 172.21.15.71:0/1790468929 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c09b630 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc818040070 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc816000640 1 -- 172.21.15.71:0/1790468929 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc81c0a46b0 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc816000640 1 -- 172.21.15.71:0/1790468929 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc81c0cd390 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc835c58740 10 client.25879 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.090+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4e7353e20 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc816000640 1 -- 172.21.15.71:0/1790468929 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc818040070 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc816000640 1 -- 172.21.15.71:0/1790468929 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc81c09b2b0 con 0x55c4e733a3b0 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc835c58740 10 client.25879 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc835c58740 10 client.25879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc835c58740 20 client.25879 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc835c58740 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73552e0 0x55c4e7357730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc835c58740 4 client.25879 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.091+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4e72ad830 con 0x55c4e73552e0 2024-09-05T12:31:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.092+0000 7fc82b400640 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73552e0 0x55c4e7357730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.092+0000 7fc82b400640 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73552e0 0x55c4e7357730 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb195321aadf8d00 server_cookie=420728c7bff12a7e in_seq=0 out_seq=0 2024-09-05T12:31:41.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.092+0000 7fc816000640 10 client.25879 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc816000640 1 -- 172.21.15.71:0/1790468929 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4e72ad830 con 0x55c4e73552e0 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc816000640 10 client.25879 handle_command_reply: tid=0 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 client.25879 shutdown 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 msgr2=0x55c4e7342560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 0x55c4e7342560 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc81c09bc60 tx=0x7fc81c097f70 comp rx=0 tx=0).stop 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 shutdown_connections 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73552e0 0x55c4e7357730 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e7345570 0x55c4e7342fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc816000640 0 client.25879 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e7342aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 --2- 172.21.15.71:0/1790468929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e733a3b0 0x55c4e7342560 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 >> 172.21.15.71:0/1790468929 conn(0x55c4e7336920 msgr2=0x55c4e733c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 shutdown_connections 2024-09-05T12:31:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.093+0000 7fc835c58740 1 -- 172.21.15.71:0/1790468929 wait complete. 2024-09-05T12:31:41.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.094+0000 7fc835c58740 20 client.25879 trim_cache size 0 max 16384 2024-09-05T12:31:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.108+0000 7fc835c58740 1 Processor -- start 2024-09-05T12:31:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.108+0000 7fc835c58740 1 -- start start 2024-09-05T12:31:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc835c58740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 0x55c4e733ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc835c58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e73443c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc835c58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e735d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc835c58740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b0e50 con 0x55c4e733a3b0 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc835c58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b2a50 con 0x55c4e733acd0 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc835c58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b1520 con 0x55c4e7345570 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 0x55c4e733ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 0x55c4e733ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42262/0 (socket says 172.21.15.71:42262) 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 -- 172.21.15.71:0/904494034 learned_addr learned my addr 172.21.15.71:0/904494034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc830e00640 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e735d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82b400640 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e73443c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 -- 172.21.15.71:0/904494034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 msgr2=0x55c4e735d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e735d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 -- 172.21.15.71:0/904494034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 msgr2=0x55c4e73443c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e73443c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.109+0000 7fc82be00640 1 -- 172.21.15.71:0/904494034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4e7345010 con 0x55c4e733a3b0 2024-09-05T12:31:41.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.110+0000 7fc82be00640 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 0x55c4e733ce70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c6a0 tx=0x7fc81c004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d34d0cdd83facc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:41.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.110+0000 7fc820c00640 1 -- 172.21.15.71:0/904494034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c09b630 con 0x55c4e733a3b0 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.110+0000 7fc820c00640 1 -- 172.21.15.71:0/904494034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81c09c3d0 con 0x55c4e733a3b0 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.110+0000 7fc820c00640 1 -- 172.21.15.71:0/904494034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc81c09baa0 con 0x55c4e733a3b0 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.110+0000 7fc835c58740 1 -- 172.21.15.71:0/904494034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 msgr2=0x55c4e733ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.110+0000 7fc835c58740 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 0x55c4e733ce70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00c6a0 tx=0x7fc81c004b00 comp rx=0 tx=0).stop 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 -- 172.21.15.71:0/904494034 shutdown_connections 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e735d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733acd0 0x55c4e73443c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 --2- 172.21.15.71:0/904494034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733a3b0 0x55c4e733ce70 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 -- 172.21.15.71:0/904494034 >> 172.21.15.71:0/904494034 conn(0x55c4e7336920 msgr2=0x55c4e7343cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 -- 172.21.15.71:0/904494034 shutdown_connections 2024-09-05T12:31:41.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 -- 172.21.15.71:0/904494034 wait complete. 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 -- start start 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 Processor -- start 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.111+0000 7fc835c58740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e73491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc835c58740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7349700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc835c58740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7349c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc835c58740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b22c0 con 0x55c4e733acd0 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc835c58740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1b1bf0 con 0x55c4e733a3b0 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc835c58740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c1aee30 con 0x55c4e7345570 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc835c58740 10 client.0 _ll_register_callbacks cb 0x7fc81c0d32f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7349700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7349700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42268/0 (socket says 172.21.15.71:42268) 2024-09-05T12:31:41.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 -- 172.21.15.71:0/1628780671 learned_addr learned my addr 172.21.15.71:0/1628780671 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82be00640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e73491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc830e00640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7349c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 -- 172.21.15.71:0/1628780671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 msgr2=0x55c4e7349c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7349c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 -- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 msgr2=0x55c4e73491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e73491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.112+0000 7fc82b400640 1 -- 172.21.15.71:0/1628780671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4e73bf040 con 0x55c4e733acd0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc82b400640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7349700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc818031470 tx=0x7fc81802ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c22bd2a890ba9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc816000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc816000640 1 -- 172.21.15.71:0/1628780671 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818035040 con 0x55c4e733acd0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc816000640 1 -- 172.21.15.71:0/1628780671 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc81804f460 con 0x55c4e733acd0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc81c09c3d0 con 0x55c4e733acd0 2024-09-05T12:31:41.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc816000640 1 -- 172.21.15.71:0/1628780671 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc818049040 con 0x55c4e733acd0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc835c58740 10 client.15840 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4e73c0370 con 0x55c4e733acd0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.113+0000 7fc816000640 1 -- 172.21.15.71:0/1628780671 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc81c09c3d0 con 0x55c4e733acd0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc816000640 1 -- 172.21.15.71:0/1628780671 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc818003a70 con 0x55c4e733acd0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc835c58740 10 client.15840 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc835c58740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc835c58740 20 client.15840 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc835c58740 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73c06b0 0x55c4e73c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc835c58740 4 client.15840 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4e72a7b80 con 0x55c4e73c06b0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc82be00640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73c06b0 0x55c4e73c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc82be00640 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73c06b0 0x55c4e73c2ad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662081baa7c9a069 server_cookie=316792c9480e40b9 in_seq=0 out_seq=0 2024-09-05T12:31:41.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.114+0000 7fc816000640 10 client.15840 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc816000640 1 -- 172.21.15.71:0/1628780671 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4e72a7b80 con 0x55c4e73c06b0 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc816000640 10 client.15840 handle_command_reply: tid=0 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 client.15840 shutdown 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 msgr2=0x55c4e7349700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7349700 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc818031470 tx=0x7fc81802ab10 comp rx=0 tx=0).stop 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 shutdown_connections 2024-09-05T12:31:41.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c4e73c06b0 0x55c4e73c2ad0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c4e7345570 0x55c4e7349c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c4e733acd0 0x55c4e7349700 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc816000640 0 client.15840 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 --2- 172.21.15.71:0/1628780671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c4e733a3b0 0x55c4e73491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 >> 172.21.15.71:0/1628780671 conn(0x55c4e7336920 msgr2=0x55c4e73359c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 shutdown_connections 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 1 -- 172.21.15.71:0/1628780671 wait complete. 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc835c58740 20 client.15840 trim_cache size 0 max 16384 2024-09-05T12:31:41.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:31:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:31:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:31:41.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8100a2cf0 msgr2=0x7fc8100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8100a2cf0 0x7fc8100a51b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc824005fd0 tx=0x7fc824009500 comp rx=0 tx=0).stop 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 msgr2=0x7fc82c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 0x7fc82c19a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00e9f0 tx=0x7fc81c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 shutdown_connections 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8100a2cf0 0x7fc8100a51b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc82c109b90 0x7fc82c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc82c104490 0x7fc82c19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 --2- 172.21.15.71:0/939466088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc82c103af0 0x7fc82c19a430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 >> 172.21.15.71:0/939466088 conn(0x7fc82c0ff7d0 msgr2=0x7fc82c1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 shutdown_connections 2024-09-05T12:31:41.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:41.137+0000 7fc831800640 1 -- 172.21.15.71:0/939466088 wait complete. 2024-09-05T12:31:41.203 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.95 MB/sec execute 37 sec latency 22343.099 ms 2024-09-05T12:31:42.203 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.93 MB/sec execute 38 sec latency 23343.241 ms 2024-09-05T12:31:42.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:42 smithi071 ceph-mon[29845]: pgmap v121: 129 pgs: 129 active+clean; 38 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 438 KiB/s wr, 44 op/s 2024-09-05T12:31:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:42 smithi123 ceph-mon[34321]: pgmap v121: 129 pgs: 129 active+clean; 38 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 438 KiB/s wr, 44 op/s 2024-09-05T12:31:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:42 smithi174 ceph-mon[34313]: pgmap v121: 129 pgs: 129 active+clean; 38 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 438 KiB/s wr, 44 op/s 2024-09-05T12:31:43.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.90 MB/sec execute 39 sec latency 24343.395 ms 2024-09-05T12:31:44.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.88 MB/sec execute 40 sec latency 25343.541 ms 2024-09-05T12:31:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:44 smithi071 ceph-mon[29845]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 562 KiB/s wr, 35 op/s 2024-09-05T12:31:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:44 smithi123 ceph-mon[34321]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 562 KiB/s wr, 35 op/s 2024-09-05T12:31:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:44 smithi174 ceph-mon[34313]: pgmap v122: 129 pgs: 129 active+clean; 39 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 562 KiB/s wr, 35 op/s 2024-09-05T12:31:45.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.86 MB/sec execute 41 sec latency 26343.688 ms 2024-09-05T12:31:46.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.84 MB/sec execute 42 sec latency 27343.828 ms 2024-09-05T12:31:46.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:46 smithi071 ceph-mon[29845]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 560 KiB/s wr, 30 op/s 2024-09-05T12:31:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:46 smithi123 ceph-mon[34321]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 560 KiB/s wr, 30 op/s 2024-09-05T12:31:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:46 smithi174 ceph-mon[34313]: pgmap v123: 129 pgs: 129 active+clean; 39 MiB data, 624 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 560 KiB/s wr, 30 op/s 2024-09-05T12:31:47.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.82 MB/sec execute 43 sec latency 28343.906 ms 2024-09-05T12:31:48.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.80 MB/sec execute 44 sec latency 29344.048 ms 2024-09-05T12:31:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:48 smithi071 ceph-mon[29845]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 578 KiB/s wr, 41 op/s 2024-09-05T12:31:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:48 smithi123 ceph-mon[34321]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 578 KiB/s wr, 41 op/s 2024-09-05T12:31:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:48 smithi174 ceph-mon[34313]: pgmap v124: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 578 KiB/s wr, 41 op/s 2024-09-05T12:31:49.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.78 MB/sec execute 45 sec latency 30344.121 ms 2024-09-05T12:31:49.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:31:49.643 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:31:50.204 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.77 MB/sec execute 46 sec latency 31344.222 ms 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 -- 172.21.15.71:0/2599464081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674102360 msgr2=0x7fd67410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 --2- 172.21.15.71:0/2599464081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674102360 0x7fd67410e890 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009cd0 tx=0x7fd66c02a8e0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 -- 172.21.15.71:0/2599464081 shutdown_connections 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 --2- 172.21.15.71:0/2599464081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd674102360 0x7fd67410e890 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 --2- 172.21.15.71:0/2599464081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 0x7fd674101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 --2- 172.21.15.71:0/2599464081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6741079c0 0x7fd674107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 -- 172.21.15.71:0/2599464081 >> 172.21.15.71:0/2599464081 conn(0x7fd6740fd740 msgr2=0x7fd6740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:50.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 -- 172.21.15.71:0/2599464081 shutdown_connections 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.283+0000 7fd673200640 1 -- 172.21.15.71:0/2599464081 wait complete. 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.284+0000 7fd673200640 1 Processor -- start 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.284+0000 7fd673200640 1 -- start start 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd673200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 0x7fd67419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd673200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 0x7fd67419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd673200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6741079c0 0x7fd67419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd673200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd674111dc0 con 0x7fd6741079c0 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd673200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd674111c40 con 0x7fd6741019c0 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd673200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd674111f40 con 0x7fd674102360 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 0x7fd67419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:50.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 0x7fd67419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd672800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6741079c0 0x7fd67419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 0x7fd67419aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:51878/0 (socket says 172.21.15.71:51878) 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 -- 172.21.15.71:0/4273341830 learned_addr learned my addr 172.21.15.71:0/4273341830 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 -- 172.21.15.71:0/4273341830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 msgr2=0x7fd67419a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 0x7fd67419a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 -- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6741079c0 msgr2=0x7fd67419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6741079c0 0x7fd67419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671400640 1 -- 172.21.15.71:0/4273341830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67419f480 con 0x7fd674102360 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.285+0000 7fd671e00640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 0x7fd67419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.286+0000 7fd671400640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 0x7fd67419aa10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e9f0 tx=0x7fd66000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.286+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66000cd80 con 0x7fd674102360 2024-09-05T12:31:50.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.286+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67419f770 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.286+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd660010910 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.286+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6600188f0 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.286+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6741a7050 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.287+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd674101e20 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.290+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd660010ac0 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.291+0000 7fd66aa00640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd6580a2cf0 0x7fd6580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.291+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fd6600cff80 con 0x7fd674102360 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.291+0000 7fd671e00640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd6580a2cf0 0x7fd6580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.292+0000 7fd671e00640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd6580a2cf0 0x7fd6580a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd664005d00 tx=0x7fd66400b410 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:50.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.293+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd66009c1b0 con 0x7fd674102360 2024-09-05T12:31:50.422 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:50 smithi071 ceph-mon[29845]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 575 KiB/s wr, 36 op/s 2024-09-05T12:31:50.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.509+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6741a72b0 con 0x7fd674102360 2024-09-05T12:31:50.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.510+0000 7fd66aa00640 1 -- 172.21.15.71:0/4273341830 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11975 (secure 0 0 0) 0x7fd6600a04b0 con 0x7fd674102360 2024-09-05T12:31:50.511 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:31:50.512 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":32,"btime":"2024-09-05T12:29:52:485449+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:29:51.941852+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:31:50.512 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 32 2024-09-05T12:31:50.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.512+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd6580a2cf0 msgr2=0x7fd6580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.512+0000 7fd673200640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd6580a2cf0 0x7fd6580a51b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd664005d00 tx=0x7fd66400b410 comp rx=0 tx=0).stop 2024-09-05T12:31:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 msgr2=0x7fd67419aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 0x7fd67419aa10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd66000e9f0 tx=0x7fd66000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 shutdown_connections 2024-09-05T12:31:50.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd6580a2cf0 0x7fd6580a51b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd6741079c0 0x7fd67419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd674102360 0x7fd67419aa10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 --2- 172.21.15.71:0/4273341830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6741019c0 0x7fd67419a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 >> 172.21.15.71:0/4273341830 conn(0x7fd6740fd740 msgr2=0x7fd6740feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 shutdown_connections 2024-09-05T12:31:50.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:50.513+0000 7fd673200640 1 -- 172.21.15.71:0/4273341830 wait complete. 2024-09-05T12:31:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:50 smithi123 ceph-mon[34321]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 575 KiB/s wr, 36 op/s 2024-09-05T12:31:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:50 smithi174 ceph-mon[34313]: pgmap v125: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 575 KiB/s wr, 36 op/s 2024-09-05T12:31:51.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.75 MB/sec execute 47 sec latency 32344.327 ms 2024-09-05T12:31:51.376 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id eb2057bd --release --await' 2024-09-05T12:31:51.376 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id eb2057bd --release --await 2024-09-05T12:31:51.427 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:51 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4273341830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:31:51.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:51 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4273341830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:31:51.745 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:31:51.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:51 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4273341830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:31:52.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 102700 0.73 MB/sec execute 48 sec latency 33344.462 ms 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 -- 172.21.15.71:0/3289307876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c101de0 msgr2=0x7f5b3c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 --2- 172.21.15.71:0/3289307876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c101de0 0x7f5b3c1021c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009900 tx=0x7f5b3002a410 comp rx=0 tx=0).stop 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 -- 172.21.15.71:0/3289307876 shutdown_connections 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 --2- 172.21.15.71:0/3289307876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c106f10 0x7f5b3c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 --2- 172.21.15.71:0/3289307876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b3c102790 0x7f5b3c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 --2- 172.21.15.71:0/3289307876 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c101de0 0x7f5b3c1021c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 -- 172.21.15.71:0/3289307876 >> 172.21.15.71:0/3289307876 conn(0x7f5b3c0fd8c0 msgr2=0x7f5b3c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:52.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 -- 172.21.15.71:0/3289307876 shutdown_connections 2024-09-05T12:31:52.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.369+0000 7f5b43800640 1 -- 172.21.15.71:0/3289307876 wait complete. 2024-09-05T12:31:52.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.370+0000 7f5b43800640 1 Processor -- start 2024-09-05T12:31:52.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 -- start start 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 0x7f5b3c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c102790 0x7f5b3c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b3c106f10 0x7f5b3c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c111fc0 con 0x7f5b3c102790 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c111e40 con 0x7f5b3c101de0 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b43800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c112140 con 0x7f5b3c106f10 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 0x7f5b3c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 0x7f5b3c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36400/0 (socket says 172.21.15.71:36400) 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b40e00640 1 -- 172.21.15.71:0/2479061891 learned_addr learned my addr 172.21.15.71:0/2479061891 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b3be00640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c102790 0x7f5b3c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.371+0000 7f5b41800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b3c106f10 0x7f5b3c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b40e00640 1 -- 172.21.15.71:0/2479061891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b3c106f10 msgr2=0x7f5b3c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b40e00640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b3c106f10 0x7f5b3c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b40e00640 1 -- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c102790 msgr2=0x7f5b3c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b40e00640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c102790 0x7f5b3c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b40e00640 1 -- 172.21.15.71:0/2479061891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c19f620 con 0x7f5b3c101de0 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b40e00640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 0x7f5b3c19a670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b30004770 tx=0x7f5b30003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b39600640 1 -- 172.21.15.71:0/2479061891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3003f070 con 0x7f5b3c101de0 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b39600640 1 -- 172.21.15.71:0/2479061891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b30031ca0 con 0x7f5b3c101de0 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c19f8b0 con 0x7f5b3c101de0 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.372+0000 7f5b39600640 1 -- 172.21.15.71:0/2479061891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b30035030 con 0x7f5b3c101de0 2024-09-05T12:31:52.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.373+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1a71f0 con 0x7f5b3c101de0 2024-09-05T12:31:52.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.376+0000 7f5b4629b740 1 Processor -- start 2024-09-05T12:31:52.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.376+0000 7f5b4629b740 1 -- start start 2024-09-05T12:31:52.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b39600640 1 -- 172.21.15.71:0/2479061891 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5b30042070 con 0x7f5b3c101de0 2024-09-05T12:31:52.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08a24f0 0x55e3b08a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b4629b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3b07fd650 con 0x55e3b0897c30 2024-09-05T12:31:52.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b4629b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3b07a4f10 con 0x55e3b08a24f0 2024-09-05T12:31:52.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b4629b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3b07f9610 con 0x55e3b0897310 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b41800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b40e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32776/0 (socket says 172.21.15.71:32776) 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b41800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:51902/0 (socket says 172.21.15.71:51902) 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b40e00640 1 -- 172.21.15.71:0/4124714616 learned_addr learned my addr 172.21.15.71:0/4124714616 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.377+0000 7f5b3be00640 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08a24f0 0x55e3b08a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b41800640 1 -- 172.21.15.71:0/4124714616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08a24f0 msgr2=0x55e3b08a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b41800640 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08a24f0 0x55e3b08a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b41800640 1 -- 172.21.15.71:0/4124714616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 msgr2=0x55e3b08a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b41800640 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.380 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b41800640 1 -- 172.21.15.71:0/4124714616 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b050c580 con 0x55e3b0897310 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b41800640 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08976f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009f00 tx=0x7f5b34011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2e71f41fbd093c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b2e000640 1 -- 172.21.15.71:0/4124714616 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34004d20 con 0x55e3b0897310 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b2e000640 1 -- 172.21.15.71:0/4124714616 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34004050 con 0x55e3b0897310 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.378+0000 7f5b2e000640 1 -- 172.21.15.71:0/4124714616 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34005280 con 0x55e3b0897310 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b39600640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b240a2d70 0x7f5b240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b39600640 1 -- 172.21.15.71:0/2479061891 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f5b300efdc0 con 0x7f5b3c101de0 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b40e00640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b240a2d70 0x7f5b240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 -- 172.21.15.71:0/4124714616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 msgr2=0x55e3b08976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08976f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009f00 tx=0x7f5b34011810 comp rx=0 tx=0).stop 2024-09-05T12:31:52.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 -- 172.21.15.71:0/4124714616 shutdown_connections 2024-09-05T12:31:52.383 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08a24f0 0x55e3b08a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 --2- 172.21.15.71:0/4124714616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08976f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 -- 172.21.15.71:0/4124714616 >> 172.21.15.71:0/4124714616 conn(0x55e3b08938a0 msgr2=0x55e3b0893c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.379+0000 7f5b4629b740 1 -- 172.21.15.71:0/4124714616 shutdown_connections 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.380+0000 7f5b40e00640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b240a2d70 0x7f5b240a5230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b30031550 tx=0x7f5b300f3870 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.380+0000 7f5b4629b740 1 -- 172.21.15.71:0/4124714616 wait complete. 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.380+0000 7f5b4629b740 1 -- start start 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.380+0000 7f5b4629b740 1 Processor -- start 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 0x55e3b089f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897c30 0x55e3b089f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b08a24f0 0x55e3b089fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3b08a50c0 con 0x55e3b0897310 2024-09-05T12:31:52.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3b07b8350 con 0x55e3b0897c30 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3b08a5240 con 0x55e3b08a24f0 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b4629b740 10 client.0 _ll_register_callbacks cb 0x55e3b07db290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b41800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 0x55e3b089f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b41800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 0x55e3b089f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32792/0 (socket says 172.21.15.71:32792) 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b41800640 1 -- 172.21.15.71:0/779153166 learned_addr learned my addr 172.21.15.71:0/779153166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b3be00640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897c30 0x55e3b089f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.381+0000 7f5b41800640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b08a24f0 0x55e3b089fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b41800640 1 -- 172.21.15.71:0/779153166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b08a24f0 msgr2=0x55e3b089fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b41800640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b08a24f0 0x55e3b089fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b41800640 1 -- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897c30 msgr2=0x55e3b089f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b41800640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897c30 0x55e3b089f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b41800640 1 -- 172.21.15.71:0/779153166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b05d6960 con 0x55e3b0897310 2024-09-05T12:31:52.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b41800640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 0x55e3b089f380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b34004670 tx=0x7f5b34009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20022e255e3ee9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b22a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b22a00640 1 -- 172.21.15.71:0/779153166 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34005040 con 0x55e3b0897310 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b22a00640 1 -- 172.21.15.71:0/779153166 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f5b340051e0 con 0x55e3b0897310 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b34004050 con 0x55e3b0897310 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b22a00640 1 -- 172.21.15.71:0/779153166 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5b34035050 con 0x55e3b0897310 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.382+0000 7f5b4629b740 10 client.15858 fetch_fsmap learned FSMap version 32 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e3b08b0ca0 con 0x55e3b0897310 2024-09-05T12:31:52.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b22a00640 1 -- 172.21.15.71:0/779153166 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b34004050 con 0x55e3b0897310 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b22a00640 1 -- 172.21.15.71:0/779153166 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f5b34019880 con 0x55e3b0897310 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 10 client.15858 fetch_fsmap finished waiting for FSMap version 32 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 10 client.15858 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 20 client.15858 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08b20a0 0x55e3b08b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 4 client.15858 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.383+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b080abc0 con 0x55e3b08b20a0 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.384+0000 7f5b3be00640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08b20a0 0x55e3b08b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.384+0000 7f5b3be00640 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08b20a0 0x55e3b08b4490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c703bc70713f52 server_cookie=bb818e83ff638bae in_seq=0 out_seq=0 2024-09-05T12:31:52.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.384+0000 7f5b22a00640 10 client.15858 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b22a00640 1 -- 172.21.15.71:0/779153166 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3b080abc0 con 0x55e3b08b20a0 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b22a00640 10 client.15858 handle_command_reply: tid=0 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 client.15858 shutdown 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 msgr2=0x55e3b089f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 0x55e3b089f380 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5b34004670 tx=0x7f5b34009a50 comp rx=0 tx=0).stop 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 shutdown_connections 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08b20a0 0x55e3b08b4490 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b08a24f0 0x55e3b089fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897c30 0x55e3b089f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 --2- 172.21.15.71:0/779153166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897310 0x55e3b089f380 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 >> 172.21.15.71:0/779153166 conn(0x55e3b08938a0 msgr2=0x55e3b08995d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b22a00640 0 client.15858 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 shutdown_connections 2024-09-05T12:31:52.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.385+0000 7f5b4629b740 1 -- 172.21.15.71:0/779153166 wait complete. 2024-09-05T12:31:52.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.386+0000 7f5b4629b740 20 client.15858 trim_cache size 0 max 16384 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 Processor -- start 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 -- start start 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08938a0 0x55e3b089fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08ba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b1130 con 0x55e3b0897c30 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b2d30 con 0x55e3b08938a0 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b4629b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b1800 con 0x55e3b0897310 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b3be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b3be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08b9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:51928/0 (socket says 172.21.15.71:51928) 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b3be00640 1 -- 172.21.15.71:0/599373180 learned_addr learned my addr 172.21.15.71:0/599373180 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.404+0000 7f5b41800640 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.406 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b41800640 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08938a0 0x55e3b089fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b3be00640 1 -- 172.21.15.71:0/599373180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08938a0 msgr2=0x55e3b089fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b3be00640 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08938a0 0x55e3b089fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b3be00640 1 -- 172.21.15.71:0/599373180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 msgr2=0x55e3b08ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b3be00640 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b3be00640 1 -- 172.21.15.71:0/599373180 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b08b4a70 con 0x55e3b0897310 2024-09-05T12:31:52.407 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b3be00640 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08b9d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c10eee0 tx=0x7f5b2800b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efc9708964d4ac58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:52.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b2d600640 1 -- 172.21.15.71:0/599373180 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28003d20 con 0x55e3b0897310 2024-09-05T12:31:52.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b2d600640 1 -- 172.21.15.71:0/599373180 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b28005500 con 0x55e3b0897310 2024-09-05T12:31:52.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.405+0000 7f5b2d600640 1 -- 172.21.15.71:0/599373180 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280266c0 con 0x55e3b0897310 2024-09-05T12:31:52.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 -- 172.21.15.71:0/599373180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 msgr2=0x55e3b08b9d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.409 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08b9d10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c10eee0 tx=0x7f5b2800b390 comp rx=0 tx=0).stop 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 -- 172.21.15.71:0/599373180 shutdown_connections 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b0897c30 0x55e3b08ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897310 0x55e3b08b9d10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 --2- 172.21.15.71:0/599373180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b08938a0 0x55e3b089fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 -- 172.21.15.71:0/599373180 >> 172.21.15.71:0/599373180 conn(0x55e3b088ee60 msgr2=0x55e3b0892aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 -- 172.21.15.71:0/599373180 shutdown_connections 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.406+0000 7f5b4629b740 1 -- 172.21.15.71:0/599373180 wait complete. 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 -- start start 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 Processor -- start 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 0x55e3b08aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897310 0x55e3b08aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897c30 0x55e3b08a7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b25a0 con 0x55e3b08938a0 2024-09-05T12:31:52.410 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1b1ed0 con 0x55e3b0897310 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.407+0000 7f5b4629b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1af110 con 0x55e3b0897c30 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b4629b740 10 client.0 _ll_register_callbacks cb 0x7f5b300f3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 0x55e3b08aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 0x55e3b08aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32800/0 (socket says 172.21.15.71:32800) 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 -- 172.21.15.71:0/2219115816 learned_addr learned my addr 172.21.15.71:0/2219115816 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b3be00640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897310 0x55e3b08aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897c30 0x55e3b08a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 -- 172.21.15.71:0/2219115816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897c30 msgr2=0x55e3b08a7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897c30 0x55e3b08a7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 -- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897310 msgr2=0x55e3b08aaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897310 0x55e3b08aaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.408+0000 7f5b41800640 1 -- 172.21.15.71:0/2219115816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b089d320 con 0x55e3b08938a0 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b41800640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 0x55e3b08aa4c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b34000f80 tx=0x7f5b34004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163c72c25c330c2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:52.411 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b22a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b22a00640 1 -- 172.21.15.71:0/2219115816 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b34019430 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b22a00640 1 -- 172.21.15.71:0/2219115816 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f5b340195d0 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b28005500 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b22a00640 1 -- 172.21.15.71:0/2219115816 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5b340197d0 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b4629b740 10 client.15864 fetch_fsmap learned FSMap version 33 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e3b091d2a0 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.409+0000 7f5b22a00640 1 -- 172.21.15.71:0/2219115816 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b28005500 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b22a00640 1 -- 172.21.15.71:0/2219115816 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7f5b34027410 con 0x55e3b08938a0 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b4629b740 10 client.15864 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b4629b740 10 client.15864 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b4629b740 20 client.15864 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b4629b740 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08a24f0 0x55e3b091fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:52.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b4629b740 4 client.15864 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eb2057bd", "release": true, "await": true}] 2024-09-05T12:31:52.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eb2057bd", "release": true, "await": true}) -- 0x55e3b08050d0 con 0x55e3b08a24f0 2024-09-05T12:31:52.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b3be00640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08a24f0 0x55e3b091fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:52.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b3be00640 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08a24f0 0x55e3b091fa60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b149da1e09fd442 server_cookie=ad7b6c974e41a96f in_seq=0 out_seq=0 2024-09-05T12:31:52.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.410+0000 7f5b22a00640 10 client.15864 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b22a00640 1 -- 172.21.15.71:0/2219115816 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e3b08050d0 con 0x55e3b08a24f0 2024-09-05T12:31:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b22a00640 10 client.15864 handle_command_reply: tid=0 2024-09-05T12:31:52.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 client.15864 shutdown 2024-09-05T12:31:52.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 msgr2=0x55e3b08aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 0x55e3b08aa4c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b34000f80 tx=0x7f5b34004510 comp rx=0 tx=0).stop 2024-09-05T12:31:52.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 shutdown_connections 2024-09-05T12:31:52.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b08a24f0 0x55e3b091fa60 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b0897c30 0x55e3b08a7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b0897310 0x55e3b08aaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 --2- 172.21.15.71:0/2219115816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b08938a0 0x55e3b08aa4c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 >> 172.21.15.71:0/2219115816 conn(0x55e3b088ee60 msgr2=0x55e3b0898d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b22a00640 0 client.15864 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 shutdown_connections 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 1 -- 172.21.15.71:0/2219115816 wait complete. 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b4629b740 20 client.15864 trim_cache size 0 max 16384 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:31:52.420 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 33, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 16, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "eb2057bd": { 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 16, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 154.1, 2024-09-05T12:31:52.421 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:31:52.422 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b240a2d70 msgr2=0x7f5b240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b240a2d70 0x7f5b240a5230 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b30031550 tx=0x7f5b300f3870 comp rx=0 tx=0).stop 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 msgr2=0x7f5b3c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 0x7f5b3c19a670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5b30004770 tx=0x7f5b30003c10 comp rx=0 tx=0).stop 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 shutdown_connections 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b3c106f10 0x7f5b3c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b240a2d70 0x7f5b240a5230 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b3c102790 0x7f5b3c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 --2- 172.21.15.71:0/2479061891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b3c101de0 0x7f5b3c19a670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 >> 172.21.15.71:0/2479061891 conn(0x7f5b3c0fd8c0 msgr2=0x7f5b3c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 shutdown_connections 2024-09-05T12:31:52.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:31:52.416+0000 7f5b43800640 1 -- 172.21.15.71:0/2479061891 wait complete. 2024-09-05T12:31:52.521 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:52 smithi071 ceph-mon[29845]: pgmap v126: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 580 KiB/s wr, 41 op/s 2024-09-05T12:31:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:52 smithi123 ceph-mon[34321]: pgmap v126: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 580 KiB/s wr, 41 op/s 2024-09-05T12:31:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:52 smithi174 ceph-mon[34313]: pgmap v126: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 580 KiB/s wr, 41 op/s 2024-09-05T12:31:53.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 103258 0.81 MB/sec execute 49 sec latency 33851.604 ms 2024-09-05T12:31:53.626 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'eb2057bd', total seconds elapsed: 38.4 2024-09-05T12:31:53.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:53 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:31:53.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:53 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:31:53.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:53 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:31:54.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 105209 0.93 MB/sec execute 50 sec latency 10.880 ms 2024-09-05T12:31:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 -- 172.21.15.71:0/2490218301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 msgr2=0x7fdcc4110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 --2- 172.21.15.71:0/2490218301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc4110bf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ca0 tx=0x7fdcbc02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 -- 172.21.15.71:0/2490218301 shutdown_connections 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 --2- 172.21.15.71:0/2490218301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc4110bf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 --2- 172.21.15.71:0/2490218301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc4104900 0x7fdcc4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 --2- 172.21.15.71:0/2490218301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc4103f50 0x7fdcc4104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 -- 172.21.15.71:0/2490218301 >> 172.21.15.71:0/2490218301 conn(0x7fdcc40ff9e0 msgr2=0x7fdcc4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 -- 172.21.15.71:0/2490218301 shutdown_connections 2024-09-05T12:31:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.261+0000 7fdccba00640 1 -- 172.21.15.71:0/2490218301 wait complete. 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.262+0000 7fdccba00640 1 Processor -- start 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.262+0000 7fdccba00640 1 -- start start 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdccba00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc4103f50 0x7fdcc419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdccba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdccba00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc419ee10 0x7fdcc419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdccba00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4114140 con 0x7fdcc4103f50 2024-09-05T12:31:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdccba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4113fc0 con 0x7fdcc41090a0 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdccba00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41142c0 con 0x7fdcc419ee10 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc4103f50 0x7fdcc419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48032/0 (socket says 172.21.15.71:48032) 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2853991598 learned_addr learned my addr 172.21.15.71:0/2853991598 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc419ee10 0x7fdcc419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2853991598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc419ee10 msgr2=0x7fdcc419f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc419ee10 0x7fdcc419f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc4103f50 msgr2=0x7fdcc419a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc4103f50 0x7fdcc419a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.263+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2853991598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc419f900 con 0x7fdcc41090a0 2024-09-05T12:31:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.264+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc419aa90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9f0 tx=0x7fdcb400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.264+0000 7fdcc1600640 1 -- 172.21.15.71:0/2853991598 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400ccc0 con 0x7fdcc41090a0 2024-09-05T12:31:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.264+0000 7fdcc1600640 1 -- 172.21.15.71:0/2853991598 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb4005500 con 0x7fdcc41090a0 2024-09-05T12:31:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.264+0000 7fdcc1600640 1 -- 172.21.15.71:0/2853991598 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4010690 con 0x7fdcc41090a0 2024-09-05T12:31:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.264+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc419fbf0 con 0x7fdcc41090a0 2024-09-05T12:31:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.264+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a73e0 con 0x7fdcc41090a0 2024-09-05T12:31:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.268+0000 7fdcce84d740 1 Processor -- start 2024-09-05T12:31:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.268+0000 7fdcce84d740 1 -- start start 2024-09-05T12:31:54.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.268+0000 7fdcc1600640 1 -- 172.21.15.71:0/2853991598 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdcb40046c0 con 0x7fdcc41090a0 2024-09-05T12:31:54.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710f63a0 0x55ed710f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcce84d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ed71051100 con 0x55ed710eb1e0 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcce84d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed70ff91b0 con 0x55ed710ebb00 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcce84d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ed7104d0c0 con 0x55ed710f63a0 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710f63a0 0x55ed710f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710f5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48044/0 (socket says 172.21.15.71:48044) 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc9000640 1 -- 172.21.15.71:0/678675877 learned_addr learned my addr 172.21.15.71:0/678675877 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc3e00640 1 -- 172.21.15.71:0/678675877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 msgr2=0x55ed710f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710f5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc3e00640 1 -- 172.21.15.71:0/678675877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 msgr2=0x55ed710eb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710eb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.269+0000 7fdcc3e00640 1 -- 172.21.15.71:0/678675877 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed70d64210 con 0x55ed710f63a0 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710f63a0 0x55ed710f8790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40cce30 tx=0x7fdcb40d3ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5d4492430cea059 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcb2000640 1 -- 172.21.15.71:0/678675877 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40d6070 con 0x55ed710f63a0 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcb2000640 1 -- 172.21.15.71:0/678675877 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb40d8d80 con 0x55ed710f63a0 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcb2000640 1 -- 172.21.15.71:0/678675877 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40dc3b0 con 0x55ed710f63a0 2024-09-05T12:31:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcc1600640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdcac0a2e20 0x7fdcac0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcc1600640 1 -- 172.21.15.71:0/2853991598 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fdcb40cfa20 con 0x7fdcc41090a0 2024-09-05T12:31:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.270+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdcac0a2e20 0x7fdcac0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 -- 172.21.15.71:0/678675877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710f63a0 msgr2=0x55ed710f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710f63a0 0x55ed710f8790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40cce30 tx=0x7fdcb40d3ab0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 -- 172.21.15.71:0/678675877 shutdown_connections 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710f63a0 0x55ed710f8790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 --2- 172.21.15.71:0/678675877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710eb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 -- 172.21.15.71:0/678675877 >> 172.21.15.71:0/678675877 conn(0x55ed710e7750 msgr2=0x55ed710e7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 -- 172.21.15.71:0/678675877 shutdown_connections 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 -- 172.21.15.71:0/678675877 wait complete. 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdcac0a2e20 0x7fdcac0a52e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc006010 tx=0x7fdcbc005d80 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.271+0000 7fdcce84d740 1 -- start start 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 Processor -- start 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed710f3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed710f3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ed710f8fd0 con 0x55ed710eb1e0 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ed70fda090 con 0x55ed710f63a0 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ed710f9150 con 0x55ed710ebb00 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcce84d740 10 client.0 _ll_register_callbacks cb 0x55ed7102ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcc9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed710f3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed710f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58050/0 (socket says 172.21.15.71:58050) 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.272+0000 7fdcc9000640 1 -- 172.21.15.71:0/2042206313 learned_addr learned my addr 172.21.15.71:0/2042206313 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 msgr2=0x55ed710f3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed710f3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2042206313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 msgr2=0x55ed710f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcc3e00640 1 -- 172.21.15.71:0/2042206313 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed70e30620 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed710f3880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9c0 tx=0x7fdcb40ddb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18f686f1de513336 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdca6a00640 1 -- 172.21.15.71:0/2042206313 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40d6030 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcb40d8d80 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdca6a00640 1 -- 172.21.15.71:0/2042206313 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fdcb4099f00 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdca6a00640 1 -- 172.21.15.71:0/2042206313 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcb4097ea0 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcce84d740 10 client.25921 fetch_fsmap learned FSMap version 33 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.273+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed710f8fd0 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdca6a00640 1 -- 172.21.15.71:0/2042206313 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcb40d8d80 con 0x55ed710ebb00 2024-09-05T12:31:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdca6a00640 1 -- 172.21.15.71:0/2042206313 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fdcb4098270 con 0x55ed710ebb00 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcce84d740 10 client.25921 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcce84d740 10 client.25921 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcce84d740 20 client.25921 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcce84d740 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71105db0 0x55ed711081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcce84d740 4 client.25921 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7105e830 con 0x55ed71105db0 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.274+0000 7fdcc9000640 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71105db0 0x55ed711081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.275+0000 7fdcc9000640 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71105db0 0x55ed711081a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3894e9010e479aa0 server_cookie=b0d58bf8e45da38e in_seq=0 out_seq=0 2024-09-05T12:31:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.275+0000 7fdca6a00640 10 client.25921 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.275+0000 7fdca6a00640 1 -- 172.21.15.71:0/2042206313 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed7105e830 con 0x55ed71105db0 2024-09-05T12:31:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.275+0000 7fdca6a00640 10 client.25921 handle_command_reply: tid=0 2024-09-05T12:31:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.275+0000 7fdcce84d740 1 client.25921 shutdown 2024-09-05T12:31:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 msgr2=0x55ed710f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed710f3880 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9c0 tx=0x7fdcb40ddb00 comp rx=0 tx=0).stop 2024-09-05T12:31:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 shutdown_connections 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71105db0 0x55ed711081a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed710f3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed710f3880 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 --2- 172.21.15.71:0/2042206313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdca6a00640 0 client.25921 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 >> 172.21.15.71:0/2042206313 conn(0x55ed710e7750 msgr2=0x55ed710ed4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 shutdown_connections 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 1 -- 172.21.15.71:0/2042206313 wait complete. 2024-09-05T12:31:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.276+0000 7fdcce84d740 20 client.25921 trim_cache size 0 max 16384 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.291+0000 7fdcce84d740 1 Processor -- start 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.291+0000 7fdcce84d740 1 -- start start 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed7110d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed7110de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcce84d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b0ff0 con 0x55ed710eb1e0 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcce84d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b2bc0 con 0x55ed710f63a0 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcce84d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b16c0 con 0x55ed710ebb00 2024-09-05T12:31:54.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed7110de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed7110de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48050/0 (socket says 172.21.15.71:48050) 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 -- 172.21.15.71:0/1265864183 learned_addr learned my addr 172.21.15.71:0/1265864183 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed7110d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 -- 172.21.15.71:0/1265864183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 msgr2=0x55ed7110d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed7110d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 -- 172.21.15.71:0/1265864183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 msgr2=0x55ed710ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.292+0000 7fdcc9a00640 1 -- 172.21.15.71:0/1265864183 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed710e1c20 con 0x55ed710f63a0 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.293+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed7110d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.293+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed7110de20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ca0 tx=0x7fdcbc041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a162c650b20b3780 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.293+0000 7fdcb1600640 1 -- 172.21.15.71:0/1265864183 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc003d70 con 0x55ed710f63a0 2024-09-05T12:31:54.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.293+0000 7fdcb1600640 1 -- 172.21.15.71:0/1265864183 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc036050 con 0x55ed710f63a0 2024-09-05T12:31:54.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.293+0000 7fdcb1600640 1 -- 172.21.15.71:0/1265864183 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc004700 con 0x55ed710f63a0 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 -- 172.21.15.71:0/1265864183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 msgr2=0x55ed7110de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed7110de20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009ca0 tx=0x7fdcbc041000 comp rx=0 tx=0).stop 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 -- 172.21.15.71:0/1265864183 shutdown_connections 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710f63a0 0x55ed7110de20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710ebb00 0x55ed7110d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 --2- 172.21.15.71:0/1265864183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710eb1e0 0x55ed710ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 -- 172.21.15.71:0/1265864183 >> 172.21.15.71:0/1265864183 conn(0x55ed710e7750 msgr2=0x55ed710ecc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 -- 172.21.15.71:0/1265864183 shutdown_connections 2024-09-05T12:31:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 -- 172.21.15.71:0/1265864183 wait complete. 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 -- start start 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.294+0000 7fdcce84d740 1 Processor -- start 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 0x55ed710fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710faa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710f63a0 0x55ed7116f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b2460 con 0x55ed710f63a0 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41b1d90 con 0x55ed710ebb00 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc41aefd0 con 0x55ed710eb1e0 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcce84d740 10 client.0 _ll_register_callbacks cb 0x7fdcb4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 0x55ed710fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710faa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 0x55ed710fa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:51896/0 (socket says 172.21.15.71:51896) 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 -- 172.21.15.71:0/633108849 learned_addr learned my addr 172.21.15.71:0/633108849 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710f63a0 0x55ed7116f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 -- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 msgr2=0x55ed710faa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710faa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 -- 172.21.15.71:0/633108849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710f63a0 msgr2=0x55ed7116f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710f63a0 0x55ed7116f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9000640 1 -- 172.21.15.71:0/633108849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7116fdf0 con 0x55ed710eb1e0 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.295+0000 7fdcc9a00640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710f63a0 0x55ed7116f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdcc9000640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 0x55ed710fa520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8005bc0 tx=0x7fdcb8012a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a515e5a4ecda8739 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdca6a00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:31:54.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdca6a00640 1 -- 172.21.15.71:0/633108849 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8004250 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcbc036050 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdca6a00640 1 -- 172.21.15.71:0/633108849 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fdcb80043f0 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdca6a00640 1 -- 172.21.15.71:0/633108849 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fdcb8005020 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdcce84d740 10 client.25927 fetch_fsmap learned FSMap version 33 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.296+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ed71170f90 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdca6a00640 1 -- 172.21.15.71:0/633108849 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcbc036050 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdca6a00640 1 -- 172.21.15.71:0/633108849 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 33) ==== 8382+0+0 (secure 0 0 0) 0x7fdcb80035f0 con 0x55ed710eb1e0 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcce84d740 10 client.25927 fetch_fsmap finished waiting for FSMap version 33 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcce84d740 10 client.25927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcce84d740 20 client.25927 populate_metadata read hostname 'smithi071' 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcce84d740 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71171770 0x55ed71173b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcce84d740 4 client.25927 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:31:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed71058b80 con 0x55ed71171770 2024-09-05T12:31:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.297+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71171770 0x55ed71173b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:31:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.298+0000 7fdcc3e00640 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71171770 0x55ed71173b90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf69e3c0842a6a53 server_cookie=168a8e8beaff313e in_seq=0 out_seq=0 2024-09-05T12:31:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.298+0000 7fdca6a00640 10 client.25927 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdca6a00640 1 -- 172.21.15.71:0/633108849 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed71058b80 con 0x55ed71171770 2024-09-05T12:31:54.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdca6a00640 10 client.25927 handle_command_reply: tid=0 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 client.25927 shutdown 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 msgr2=0x55ed710fa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 0x55ed710fa520 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8005bc0 tx=0x7fdcb8012a40 comp rx=0 tx=0).stop 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 shutdown_connections 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ed71171770 0x55ed71173b90 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ed710f63a0 0x55ed7116f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ed710ebb00 0x55ed710faa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdca6a00640 0 client.25927 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 --2- 172.21.15.71:0/633108849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ed710eb1e0 0x55ed710fa520 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 >> 172.21.15.71:0/633108849 conn(0x55ed710e7750 msgr2=0x55ed710f17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 shutdown_connections 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.323+0000 7fdcce84d740 1 -- 172.21.15.71:0/633108849 wait complete. 2024-09-05T12:31:54.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdcce84d740 20 client.25927 trim_cache size 0 max 16384 2024-09-05T12:31:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:31:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:31:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:31:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdcac0a2e20 msgr2=0x7fdcac0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdcac0a2e20 0x7fdcac0a52e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc006010 tx=0x7fdcbc005d80 comp rx=0 tx=0).stop 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 msgr2=0x7fdcc419aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc419aa90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400e9f0 tx=0x7fdcb400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 shutdown_connections 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcc419ee10 0x7fdcc419f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcc41090a0 0x7fdcc419aa90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdcac0a2e20 0x7fdcac0a52e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 --2- 172.21.15.71:0/2853991598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdcc4103f50 0x7fdcc419a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:31:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.324+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 >> 172.21.15.71:0/2853991598 conn(0x7fdcc40ff9e0 msgr2=0x7fdcc4100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:31:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.325+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 shutdown_connections 2024-09-05T12:31:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:31:54.325+0000 7fdccba00640 1 -- 172.21.15.71:0/2853991598 wait complete. 2024-09-05T12:31:54.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:54 smithi071 ceph-mon[29845]: pgmap v127: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 174 KiB/s wr, 22 op/s 2024-09-05T12:31:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:54 smithi123 ceph-mon[34321]: pgmap v127: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 174 KiB/s wr, 22 op/s 2024-09-05T12:31:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:54 smithi174 ceph-mon[34313]: pgmap v127: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 174 KiB/s wr, 22 op/s 2024-09-05T12:31:55.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 106822 1.16 MB/sec execute 51 sec latency 17.510 ms 2024-09-05T12:31:55.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:55 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:31:55.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:55 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:31:55.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:55 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:31:55.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:55 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:31:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:55 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:31:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:55 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:31:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:55 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:31:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:55 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:31:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:55 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:31:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:55 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:31:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:55 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:31:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:55 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:31:56.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 108840 1.27 MB/sec execute 52 sec latency 10.801 ms 2024-09-05T12:31:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:56 smithi123 ceph-mon[34321]: pgmap v128: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-09-05T12:31:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:56 smithi174 ceph-mon[34313]: pgmap v128: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-09-05T12:31:56.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:56 smithi071 ceph-mon[29845]: pgmap v128: 129 pgs: 129 active+clean; 39 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 16 op/s 2024-09-05T12:31:57.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 110292 1.47 MB/sec execute 53 sec latency 166.029 ms 2024-09-05T12:31:58.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 112201 1.55 MB/sec execute 54 sec latency 14.658 ms 2024-09-05T12:31:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:31:58 smithi123 ceph-mon[34321]: pgmap v129: 129 pgs: 129 active+clean; 42 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 331 KiB/s wr, 82 op/s 2024-09-05T12:31:58.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:31:58 smithi174 ceph-mon[34313]: pgmap v129: 129 pgs: 129 active+clean; 42 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 331 KiB/s wr, 82 op/s 2024-09-05T12:31:58.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:31:58 smithi071 ceph-mon[29845]: pgmap v129: 129 pgs: 129 active+clean; 42 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 331 KiB/s wr, 82 op/s 2024-09-05T12:31:59.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 113868 1.76 MB/sec execute 55 sec latency 54.130 ms 2024-09-05T12:32:00.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 115933 1.86 MB/sec execute 56 sec latency 10.523 ms 2024-09-05T12:32:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:00 smithi123 ceph-mon[34321]: pgmap v130: 129 pgs: 129 active+clean; 42 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 312 KiB/s wr, 71 op/s 2024-09-05T12:32:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:00 smithi174 ceph-mon[34313]: pgmap v130: 129 pgs: 129 active+clean; 42 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 312 KiB/s wr, 71 op/s 2024-09-05T12:32:00.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:00 smithi071 ceph-mon[29845]: pgmap v130: 129 pgs: 129 active+clean; 42 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 312 KiB/s wr, 71 op/s 2024-09-05T12:32:01.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 116775 1.94 MB/sec execute 57 sec latency 554.684 ms 2024-09-05T12:32:02.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 117332 2.00 MB/sec execute 58 sec latency 57.060 ms 2024-09-05T12:32:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:02 smithi123 ceph-mon[34321]: pgmap v131: 129 pgs: 129 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 841 KiB/s wr, 192 op/s 2024-09-05T12:32:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:02 smithi174 ceph-mon[34313]: pgmap v131: 129 pgs: 129 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 841 KiB/s wr, 192 op/s 2024-09-05T12:32:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:02 smithi071 ceph-mon[29845]: pgmap v131: 129 pgs: 129 active+clean; 45 MiB data, 675 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 841 KiB/s wr, 192 op/s 2024-09-05T12:32:03.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 118728 2.06 MB/sec execute 59 sec latency 10.545 ms 2024-09-05T12:32:04.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 120850 2.25 MB/sec execute 60 sec latency 48.900 ms 2024-09-05T12:32:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:04 smithi123 ceph-mon[34321]: pgmap v132: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 885 KiB/s wr, 193 op/s 2024-09-05T12:32:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:04 smithi174 ceph-mon[34313]: pgmap v132: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 885 KiB/s wr, 193 op/s 2024-09-05T12:32:04.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:04 smithi071 ceph-mon[29845]: pgmap v132: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 885 KiB/s wr, 193 op/s 2024-09-05T12:32:05.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 122252 2.31 MB/sec execute 61 sec latency 10.875 ms 2024-09-05T12:32:06.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 123619 2.39 MB/sec execute 62 sec latency 439.427 ms 2024-09-05T12:32:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:06 smithi174 ceph-mon[34313]: pgmap v133: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 885 KiB/s wr, 192 op/s 2024-09-05T12:32:06.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:06 smithi071 ceph-mon[29845]: pgmap v133: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 885 KiB/s wr, 192 op/s 2024-09-05T12:32:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:06 smithi123 ceph-mon[34321]: pgmap v133: 129 pgs: 129 active+clean; 45 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 885 KiB/s wr, 192 op/s 2024-09-05T12:32:07.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 124973 2.46 MB/sec execute 63 sec latency 10.620 ms 2024-09-05T12:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3027454990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc101ad0 msgr2=0x7f8fcc101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3027454990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc101f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009860 tx=0x7f8fb802a340 comp rx=0 tx=0).stop 2024-09-05T12:32:07.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3027454990 shutdown_connections 2024-09-05T12:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3027454990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc102470 0x7f8fcc10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3027454990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc101f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3027454990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc107b70 0x7f8fcc107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3027454990 >> 172.21.15.71:0/3027454990 conn(0x7f8fcc0fd7d0 msgr2=0x7f8fcc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:07.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3027454990 shutdown_connections 2024-09-05T12:32:07.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.450+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3027454990 wait complete. 2024-09-05T12:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.451+0000 7f8fd0a00640 1 Processor -- start 2024-09-05T12:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.451+0000 7f8fd0a00640 1 -- start start 2024-09-05T12:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fd0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fd0a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 0x7f8fcc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fd0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 0x7f8fcc19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fd0a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0748d0 con 0x7f8fcc101ad0 2024-09-05T12:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fd0a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc074750 con 0x7f8fcc107b70 2024-09-05T12:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fd0a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc074a50 con 0x7f8fcc102470 2024-09-05T12:32:07.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 0x7f8fcc19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fc9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 0x7f8fcc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fca800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 0x7f8fcc19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57668/0 (socket says 172.21.15.71:57668) 2024-09-05T12:32:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fc9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 0x7f8fcc19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52682/0 (socket says 172.21.15.71:52682) 2024-09-05T12:32:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.452+0000 7f8fca800640 1 -- 172.21.15.71:0/3995766663 learned_addr learned my addr 172.21.15.71:0/3995766663 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:07.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fca800640 1 -- 172.21.15.71:0/3995766663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 msgr2=0x7f8fcc19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fca800640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 0x7f8fcc19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fca800640 1 -- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc101ad0 msgr2=0x7f8fcc19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fca800640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fca800640 1 -- 172.21.15.71:0/3995766663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc19f560 con 0x7f8fcc107b70 2024-09-05T12:32:07.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fc9400640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 0x7f8fcc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fca800640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 0x7f8fcc19ee30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009fd0 tx=0x7f8fc400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fc2a00640 1 -- 172.21.15.71:0/3995766663 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004b20 con 0x7f8fcc107b70 2024-09-05T12:32:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fc2a00640 1 -- 172.21.15.71:0/3995766663 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fc4003e50 con 0x7f8fcc107b70 2024-09-05T12:32:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc19f850 con 0x7f8fcc107b70 2024-09-05T12:32:07.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.453+0000 7f8fc2a00640 1 -- 172.21.15.71:0/3995766663 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400b630 con 0x7f8fcc107b70 2024-09-05T12:32:07.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.454+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc1a70e0 con 0x7f8fcc107b70 2024-09-05T12:32:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.454+0000 7f8fd39dd740 1 Processor -- start 2024-09-05T12:32:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.454+0000 7f8fc2a00640 1 -- 172.21.15.71:0/3995766663 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8fc401d020 con 0x7f8fcc107b70 2024-09-05T12:32:07.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 -- start start 2024-09-05T12:32:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 0x557ab147b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1485e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab14863a0 0x557ab1488790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557ab13e1100 con 0x557ab147bb00 2024-09-05T12:32:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ab13891b0 con 0x557ab14863a0 2024-09-05T12:32:07.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ab13dd0c0 con 0x557ab147b1e0 2024-09-05T12:32:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 0x557ab147b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 0x557ab147b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52694/0 (socket says 172.21.15.71:52694) 2024-09-05T12:32:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 -- 172.21.15.71:0/161347352 learned_addr learned my addr 172.21.15.71:0/161347352 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1485e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab14863a0 0x557ab1488790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 -- 172.21.15.71:0/161347352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab14863a0 msgr2=0x557ab1488790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab14863a0 0x557ab1488790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 -- 172.21.15.71:0/161347352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 msgr2=0x557ab1485e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1485e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 -- 172.21.15.71:0/161347352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab10f4210 con 0x557ab147b1e0 2024-09-05T12:32:07.467 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9400640 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 0x557ab147b5c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80095f0 tx=0x7f8fb8009490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e93d236db4e4019 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc2a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8fb00a2da0 0x7f8fb00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc2a00640 1 -- 172.21.15.71:0/3995766663 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f8fc40cf740 con 0x7f8fcc107b70 2024-09-05T12:32:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fb7000640 1 -- 172.21.15.71:0/161347352 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8037040 con 0x557ab147b1e0 2024-09-05T12:32:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fb7000640 1 -- 172.21.15.71:0/161347352 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb8030820 con 0x557ab147b1e0 2024-09-05T12:32:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fb7000640 1 -- 172.21.15.71:0/161347352 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8003d40 con 0x557ab147b1e0 2024-09-05T12:32:07.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.458+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8fb00a2da0 0x7f8fb00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 -- 172.21.15.71:0/161347352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 msgr2=0x557ab147b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 0x557ab147b5c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80095f0 tx=0x7f8fb8009490 comp rx=0 tx=0).stop 2024-09-05T12:32:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8fb00a2da0 0x7f8fb00a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009f40 tx=0x7f8fbc005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 -- 172.21.15.71:0/161347352 shutdown_connections 2024-09-05T12:32:07.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab14863a0 0x557ab1488790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1485e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/161347352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147b1e0 0x557ab147b5c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.463+0000 7f8fd39dd740 1 -- 172.21.15.71:0/161347352 >> 172.21.15.71:0/161347352 conn(0x557ab1477750 msgr2=0x557ab1477b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.465+0000 7f8fd39dd740 1 -- 172.21.15.71:0/161347352 shutdown_connections 2024-09-05T12:32:07.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.465+0000 7f8fd39dd740 1 -- 172.21.15.71:0/161347352 wait complete. 2024-09-05T12:32:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.466+0000 7f8fd39dd740 1 -- start start 2024-09-05T12:32:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.466+0000 7f8fd39dd740 1 Processor -- start 2024-09-05T12:32:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1483380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147bb00 0x557ab14838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab14863a0 0x557ab1483e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557ab1491780 con 0x557ab14863a0 2024-09-05T12:32:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ab136a090 con 0x557ab147b1e0 2024-09-05T12:32:07.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ab1491900 con 0x557ab147bb00 2024-09-05T12:32:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.467+0000 7f8fd39dd740 10 client.0 _ll_register_callbacks cb 0x557ab13bef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.468+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1483380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.468+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1483380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57684/0 (socket says 172.21.15.71:57684) 2024-09-05T12:32:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.468+0000 7f8fc9e00640 1 -- 172.21.15.71:0/820477908 learned_addr learned my addr 172.21.15.71:0/820477908 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:07.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.468+0000 7f8fc9400640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147bb00 0x557ab14838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.469+0000 7f8fca800640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab14863a0 0x557ab1483e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.469+0000 7f8fc9400640 1 -- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 msgr2=0x557ab1483380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.469+0000 7f8fc9400640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1483380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.469+0000 7f8fc9400640 1 -- 172.21.15.71:0/820477908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab14863a0 msgr2=0x557ab1483e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.469+0000 7f8fc9400640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab14863a0 0x557ab1483e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.469+0000 7f8fc9400640 1 -- 172.21.15.71:0/820477908 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab11c0620 con 0x557ab147bb00 2024-09-05T12:32:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.470+0000 7f8fc9400640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147bb00 0x557ab14838c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80057c0 tx=0x7f8fb802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0b51bb041b63db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.470+0000 7f8fab400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:32:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.470+0000 7f8fab400640 1 -- 172.21.15.71:0/820477908 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8037070 con 0x557ab147bb00 2024-09-05T12:32:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.471+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb8030820 con 0x557ab147bb00 2024-09-05T12:32:07.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.470+0000 7f8fab400640 1 -- 172.21.15.71:0/820477908 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f8fb8005040 con 0x557ab147bb00 2024-09-05T12:32:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.472+0000 7f8fab400640 1 -- 172.21.15.71:0/820477908 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8fb8005240 con 0x557ab147bb00 2024-09-05T12:32:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.472+0000 7f8fd39dd740 10 client.25939 fetch_fsmap learned FSMap version 34 2024-09-05T12:32:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.472+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ab1491780 con 0x557ab147bb00 2024-09-05T12:32:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.472+0000 7f8fab400640 1 -- 172.21.15.71:0/820477908 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb8030820 con 0x557ab147bb00 2024-09-05T12:32:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.472+0000 7f8fab400640 1 -- 172.21.15.71:0/820477908 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f8fb803d040 con 0x557ab147bb00 2024-09-05T12:32:07.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fd39dd740 10 client.25939 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T12:32:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fd39dd740 10 client.25939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fd39dd740 20 client.25939 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab14af670 0x557ab14b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fd39dd740 4 client.25939 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:32:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ab13ee830 con 0x557ab14af670 2024-09-05T12:32:07.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.473+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab14af670 0x557ab14b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.474+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab14af670 0x557ab14b1a60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980271f5310b91 server_cookie=5ab0fe87496a4d4b in_seq=0 out_seq=0 2024-09-05T12:32:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.475+0000 7f8fab400640 10 client.25939 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fab400640 1 -- 172.21.15.71:0/820477908 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ab13ee830 con 0x557ab14af670 2024-09-05T12:32:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fab400640 10 client.25939 handle_command_reply: tid=0 2024-09-05T12:32:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 client.25939 shutdown 2024-09-05T12:32:07.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147bb00 msgr2=0x557ab14838c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147bb00 0x557ab14838c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80057c0 tx=0x7f8fb802a340 comp rx=0 tx=0).stop 2024-09-05T12:32:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 shutdown_connections 2024-09-05T12:32:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab14af670 0x557ab14b1a60 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab14863a0 0x557ab1483e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab147bb00 0x557ab14838c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/820477908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1483380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fab400640 0 client.25939 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.476+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 >> 172.21.15.71:0/820477908 conn(0x557ab1477750 msgr2=0x557ab147d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.477+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 shutdown_connections 2024-09-05T12:32:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.477+0000 7f8fd39dd740 1 -- 172.21.15.71:0/820477908 wait complete. 2024-09-05T12:32:07.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.478+0000 7f8fd39dd740 20 client.25939 trim_cache size 0 max 16384 2024-09-05T12:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 Processor -- start 2024-09-05T12:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 -- start start 2024-09-05T12:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147b1e0 0x557ab147dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 0x557ab14b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab14b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0714c0 con 0x557ab147b1e0 2024-09-05T12:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc0730c0 con 0x557ab147bb00 2024-09-05T12:32:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc071b90 con 0x557ab14863a0 2024-09-05T12:32:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fc9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 0x557ab14b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147b1e0 0x557ab147dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fc9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 0x557ab14b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57694/0 (socket says 172.21.15.71:57694) 2024-09-05T12:32:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fc9400640 1 -- 172.21.15.71:0/3161511711 learned_addr learned my addr 172.21.15.71:0/3161511711 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:07.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147b1e0 0x557ab147dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40346/0 (socket says 172.21.15.71:40346) 2024-09-05T12:32:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.501+0000 7f8fca800640 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab14b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fc9400640 1 -- 172.21.15.71:0/3161511711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 msgr2=0x557ab14b76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fc9400640 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab14b76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fc9400640 1 -- 172.21.15.71:0/3161511711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147b1e0 msgr2=0x557ab147dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fc9400640 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147b1e0 0x557ab147dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fc9400640 1 -- 172.21.15.71:0/3161511711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ab1472d60 con 0x557ab147bb00 2024-09-05T12:32:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fc9400640 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 0x557ab14b7180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8005d70 tx=0x7f8fb8009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac6a58e842475de server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fb6600640 1 -- 172.21.15.71:0/3161511711 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8037070 con 0x557ab147bb00 2024-09-05T12:32:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fb6600640 1 -- 172.21.15.71:0/3161511711 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb8003800 con 0x557ab147bb00 2024-09-05T12:32:07.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.502+0000 7f8fb6600640 1 -- 172.21.15.71:0/3161511711 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80042d0 con 0x557ab147bb00 2024-09-05T12:32:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 -- 172.21.15.71:0/3161511711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 msgr2=0x557ab14b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 0x557ab14b7180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8005d70 tx=0x7f8fb8009450 comp rx=0 tx=0).stop 2024-09-05T12:32:07.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 -- 172.21.15.71:0/3161511711 shutdown_connections 2024-09-05T12:32:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab14b76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147bb00 0x557ab14b7180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/3161511711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147b1e0 0x557ab147dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 -- 172.21.15.71:0/3161511711 >> 172.21.15.71:0/3161511711 conn(0x557ab1477750 msgr2=0x557ab1482450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:07.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 -- 172.21.15.71:0/3161511711 shutdown_connections 2024-09-05T12:32:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.503+0000 7f8fd39dd740 1 -- 172.21.15.71:0/3161511711 wait complete. 2024-09-05T12:32:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 -- start start 2024-09-05T12:32:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 Processor -- start 2024-09-05T12:32:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1491f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1492490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab1518140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc072960 con 0x557ab147bb00 2024-09-05T12:32:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc072290 con 0x557ab147b1e0 2024-09-05T12:32:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc06f520 con 0x557ab14863a0 2024-09-05T12:32:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.504+0000 7f8fd39dd740 10 client.0 _ll_register_callbacks cb 0x7f8fbc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:07.517 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1491f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1491f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57698/0 (socket says 172.21.15.71:57698) 2024-09-05T12:32:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 -- 172.21.15.71:0/4128501841 learned_addr learned my addr 172.21.15.71:0/4128501841 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fca800640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab1518140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9400640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1492490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 -- 172.21.15.71:0/4128501841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 msgr2=0x557ab1518140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab1518140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 -- 172.21.15.71:0/4128501841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 msgr2=0x557ab1492490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1492490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 -- 172.21.15.71:0/4128501841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ab1518680 con 0x557ab147b1e0 2024-09-05T12:32:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9400640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1492490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:32:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fc9e00640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1491f50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x557ab14b20c0 tx=0x7f8fbc016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3bb6f703e373b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fab400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:32:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fab400640 1 -- 172.21.15.71:0/4128501841 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011a30 con 0x557ab147b1e0 2024-09-05T12:32:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557ab13e8b80 con 0x557ab147b1e0 2024-09-05T12:32:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.505+0000 7f8fab400640 1 -- 172.21.15.71:0/4128501841 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f8fbc011bd0 con 0x557ab147b1e0 2024-09-05T12:32:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fab400640 1 -- 172.21.15.71:0/4128501841 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8fbc03a050 con 0x557ab147b1e0 2024-09-05T12:32:07.520 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 10 client.25934 fetch_fsmap learned FSMap version 34 2024-09-05T12:32:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557ab1519920 con 0x557ab147b1e0 2024-09-05T12:32:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fab400640 1 -- 172.21.15.71:0/4128501841 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557ab13e8b80 con 0x557ab147b1e0 2024-09-05T12:32:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fab400640 1 -- 172.21.15.71:0/4128501841 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 34) ==== 8386+0+0 (secure 0 0 0) 0x7f8fbc031410 con 0x557ab147b1e0 2024-09-05T12:32:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 10 client.25934 fetch_fsmap finished waiting for FSMap version 34 2024-09-05T12:32:07.521 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 20 client.25934 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab151a080 0x557ab151c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 4 client.25934 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.506+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ab13657c0 con 0x557ab151a080 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.507+0000 7f8fc9400640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab151a080 0x557ab151c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.507+0000 7f8fc9400640 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab151a080 0x557ab151c4a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3859e8d64c554cd server_cookie=93dd86f41a1a41bf in_seq=0 out_seq=0 2024-09-05T12:32:07.522 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.507+0000 7f8fab400640 10 client.25934 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fab400640 1 -- 172.21.15.71:0/4128501841 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ab13657c0 con 0x557ab151a080 2024-09-05T12:32:07.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fab400640 10 client.25934 handle_command_reply: tid=0 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 client.25934 shutdown 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 msgr2=0x557ab1491f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1491f50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x557ab14b20c0 tx=0x7f8fbc016000 comp rx=0 tx=0).stop 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 shutdown_connections 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557ab151a080 0x557ab151c4a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ab14863a0 0x557ab1518140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ab147bb00 0x557ab1492490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 --2- 172.21.15.71:0/4128501841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ab147b1e0 0x557ab1491f50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fab400640 0 client.25934 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 >> 172.21.15.71:0/4128501841 conn(0x557ab1477750 msgr2=0x557ab1483320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 shutdown_connections 2024-09-05T12:32:07.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 1 -- 172.21.15.71:0/4128501841 wait complete. 2024-09-05T12:32:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.568+0000 7f8fd39dd740 20 client.25934 trim_cache size 0 max 16384 2024-09-05T12:32:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:32:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:32:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:32:07.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:32:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8fb00a2da0 msgr2=0x7f8fb00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8fb00a2da0 0x7f8fb00a5260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009f40 tx=0x7f8fbc005f90 comp rx=0 tx=0).stop 2024-09-05T12:32:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 msgr2=0x7f8fcc19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:07.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 0x7f8fcc19ee30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4009fd0 tx=0x7f8fc400c800 comp rx=0 tx=0).stop 2024-09-05T12:32:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 shutdown_connections 2024-09-05T12:32:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8fcc107b70 0x7f8fcc19ee30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8fcc102470 0x7f8fcc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8fb00a2da0 0x7f8fb00a5260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 --2- 172.21.15.71:0/3995766663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fcc101ad0 0x7f8fcc19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 >> 172.21.15.71:0/3995766663 conn(0x7f8fcc0fd7d0 msgr2=0x7f8fcc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 shutdown_connections 2024-09-05T12:32:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:07.569+0000 7f8fd0a00640 1 -- 172.21.15.71:0/3995766663 wait complete. 2024-09-05T12:32:08.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 126948 2.59 MB/sec execute 64 sec latency 123.290 ms 2024-09-05T12:32:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:08 smithi174 ceph-mon[34313]: pgmap v134: 129 pgs: 129 active+clean; 50 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2024-09-05T12:32:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:08 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:08.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:08 smithi071 ceph-mon[29845]: pgmap v134: 129 pgs: 129 active+clean; 50 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2024-09-05T12:32:08.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:08 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:08.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:08 smithi123 ceph-mon[34321]: pgmap v134: 129 pgs: 129 active+clean; 50 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.4 MiB/s wr, 337 op/s 2024-09-05T12:32:08.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:08 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:09.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 128329 2.67 MB/sec execute 65 sec latency 12.604 ms 2024-09-05T12:32:10.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 130600 2.80 MB/sec execute 66 sec latency 51.047 ms 2024-09-05T12:32:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:10 smithi174 ceph-mon[34313]: pgmap v135: 129 pgs: 129 active+clean; 50 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-09-05T12:32:10.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:10 smithi071 ceph-mon[29845]: pgmap v135: 129 pgs: 129 active+clean; 50 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-09-05T12:32:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:10 smithi123 ceph-mon[34321]: pgmap v135: 129 pgs: 129 active+clean; 50 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.1 MiB/s wr, 271 op/s 2024-09-05T12:32:11.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 132016 2.88 MB/sec execute 67 sec latency 9.911 ms 2024-09-05T12:32:12.205 INFO:tasks.workunit.client.0.smithi071.stdout: 1 134286 3.02 MB/sec execute 68 sec latency 9.709 ms 2024-09-05T12:32:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:12 smithi174 ceph-mon[34313]: pgmap v136: 129 pgs: 129 active+clean; 50 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-05T12:32:12.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:12 smithi071 ceph-mon[29845]: pgmap v136: 129 pgs: 129 active+clean; 50 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-05T12:32:12.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:12 smithi123 ceph-mon[34321]: pgmap v136: 129 pgs: 129 active+clean; 50 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 1.6 MiB/s wr, 400 op/s 2024-09-05T12:32:13.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 135623 3.08 MB/sec execute 69 sec latency 10.910 ms 2024-09-05T12:32:13.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:13 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:13.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:13 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:13.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:13 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:14.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 137402 3.14 MB/sec execute 70 sec latency 212.371 ms 2024-09-05T12:32:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:14 smithi174 ceph-mon[34313]: pgmap v137: 129 pgs: 129 active+clean; 51 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-09-05T12:32:14.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:14 smithi071 ceph-mon[29845]: pgmap v137: 129 pgs: 129 active+clean; 51 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-09-05T12:32:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:14 smithi123 ceph-mon[34321]: pgmap v137: 129 pgs: 129 active+clean; 51 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.2 MiB/s wr, 306 op/s 2024-09-05T12:32:15.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 138956 3.26 MB/sec execute 71 sec latency 12.577 ms 2024-09-05T12:32:16.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 141262 3.37 MB/sec execute 72 sec latency 10.948 ms 2024-09-05T12:32:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:16 smithi174 ceph-mon[34313]: pgmap v138: 129 pgs: 129 active+clean; 51 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.1 MiB/s wr, 300 op/s 2024-09-05T12:32:16.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:16 smithi071 ceph-mon[29845]: pgmap v138: 129 pgs: 129 active+clean; 51 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.1 MiB/s wr, 300 op/s 2024-09-05T12:32:16.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:16 smithi123 ceph-mon[34321]: pgmap v138: 129 pgs: 129 active+clean; 51 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.1 MiB/s wr, 300 op/s 2024-09-05T12:32:17.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 142578 3.43 MB/sec execute 73 sec latency 14.312 ms 2024-09-05T12:32:18.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 144439 3.49 MB/sec execute 74 sec latency 186.659 ms 2024-09-05T12:32:18.713 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:18 smithi071 ceph-mon[29845]: pgmap v139: 129 pgs: 129 active+clean; 56 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 412 op/s 2024-09-05T12:32:18.731 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:18 smithi123 ceph-mon[34321]: pgmap v139: 129 pgs: 129 active+clean; 56 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 412 op/s 2024-09-05T12:32:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:18 smithi174 ceph-mon[34313]: pgmap v139: 129 pgs: 129 active+clean; 56 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 412 op/s 2024-09-05T12:32:19.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 145982 3.59 MB/sec execute 75 sec latency 10.149 ms 2024-09-05T12:32:20.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 147965 3.65 MB/sec execute 76 sec latency 64.429 ms 2024-09-05T12:32:20.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:20 smithi071 ceph-mon[29845]: pgmap v140: 129 pgs: 129 active+clean; 56 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.701+0000 7fe49e800640 1 -- 172.21.15.71:0/164002321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe498102b40 msgr2=0x7fe498102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.701+0000 7fe49e800640 1 --2- 172.21.15.71:0/164002321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe498102b40 0x7fe498102f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009900 tx=0x7fe48c02a480 comp rx=0 tx=0).stop 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.701+0000 7fe49e800640 1 -- 172.21.15.71:0/164002321 shutdown_connections 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.701+0000 7fe49e800640 1 --2- 172.21.15.71:0/164002321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49810bf10 0x7fe49810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.701+0000 7fe49e800640 1 --2- 172.21.15.71:0/164002321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe498103460 0x7fe49810b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.701+0000 7fe49e800640 1 --2- 172.21.15.71:0/164002321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe498102b40 0x7fe498102f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.702+0000 7fe49e800640 1 -- 172.21.15.71:0/164002321 >> 172.21.15.71:0/164002321 conn(0x7fe4980777c0 msgr2=0x7fe4980feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.702+0000 7fe49e800640 1 -- 172.21.15.71:0/164002321 shutdown_connections 2024-09-05T12:32:20.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.702+0000 7fe49e800640 1 -- 172.21.15.71:0/164002321 wait complete. 2024-09-05T12:32:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.702+0000 7fe49e800640 1 Processor -- start 2024-09-05T12:32:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 -- start start 2024-09-05T12:32:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 0x7fe49819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe498103460 0x7fe49819a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe49810bf10 0x7fe49819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe498111ce0 con 0x7fe498102b40 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe498111b60 con 0x7fe498103460 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49e800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe498111e60 con 0x7fe49810bf10 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 0x7fe49819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 0x7fe49819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50530/0 (socket says 172.21.15.71:50530) 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49d400640 1 -- 172.21.15.71:0/2338024769 learned_addr learned my addr 172.21.15.71:0/2338024769 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49de00640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe49810bf10 0x7fe49819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.703+0000 7fe49ca00640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe498103460 0x7fe49819a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49d400640 1 -- 172.21.15.71:0/2338024769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe49810bf10 msgr2=0x7fe49819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49d400640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe49810bf10 0x7fe49819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49d400640 1 -- 172.21.15.71:0/2338024769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe498103460 msgr2=0x7fe49819a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49d400640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe498103460 0x7fe49819a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49d400640 1 -- 172.21.15.71:0/2338024769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49819f410 con 0x7fe498102b40 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49de00640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe49810bf10 0x7fe49819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49d400640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 0x7fe49819a420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe48c006010 tx=0x7fe48c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe496000640 1 -- 172.21.15.71:0/2338024769 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c03f030 con 0x7fe498102b40 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe496000640 1 -- 172.21.15.71:0/2338024769 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe48c003ad0 con 0x7fe498102b40 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49819f6a0 con 0x7fe498102b40 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe496000640 1 -- 172.21.15.71:0/2338024769 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c035070 con 0x7fe498102b40 2024-09-05T12:32:20.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.704+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981a6f90 con 0x7fe498102b40 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.708+0000 7fe496000640 1 -- 172.21.15.71:0/2338024769 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe48c042030 con 0x7fe498102b40 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.708+0000 7fe4a2caf740 1 Processor -- start 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.708+0000 7fe4a2caf740 1 -- start start 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.708+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d246790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d246cd0 0x563c5d251030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d253960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563c5d1ac100 con 0x563c5d251570 2024-09-05T12:32:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c5d1541b0 con 0x563c5d2463b0 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c5d1a80c0 con 0x563c5d246cd0 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d246790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d246790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44086/0 (socket says 172.21.15.71:44086) 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 -- 172.21.15.71:0/3608930274 learned_addr learned my addr 172.21.15.71:0/3608930274 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49d400640 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d246cd0 0x563c5d251030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49ca00640 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d253960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 -- 172.21.15.71:0/3608930274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d246cd0 msgr2=0x563c5d251030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d246cd0 0x563c5d251030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 -- 172.21.15.71:0/3608930274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 msgr2=0x563c5d253960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d253960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.709+0000 7fe49de00640 1 -- 172.21.15.71:0/3608930274 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5cebf210 con 0x563c5d2463b0 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe496000640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4800a2d70 0x7fe4800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe49de00640 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d246790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe490011790 tx=0x7fe490011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aeae6e91c10a9ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe496000640 1 -- 172.21.15.71:0/2338024769 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe48c0ef8e0 con 0x7fe498102b40 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe49d400640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4800a2d70 0x7fe4800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe48a600640 1 -- 172.21.15.71:0/3608930274 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490004d20 con 0x563c5d2463b0 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe48a600640 1 -- 172.21.15.71:0/3608930274 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe490003970 con 0x563c5d2463b0 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.710+0000 7fe48a600640 1 -- 172.21.15.71:0/3608930274 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490004500 con 0x563c5d2463b0 2024-09-05T12:32:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3608930274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 msgr2=0x563c5d246790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe49d400640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4800a2d70 0x7fe4800a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0ecda0 tx=0x7fe48c0f5550 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d246790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe490011790 tx=0x7fe490011c50 comp rx=0 tx=0).stop 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3608930274 shutdown_connections 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d253960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d246cd0 0x563c5d251030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3608930274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d246790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3608930274 >> 172.21.15.71:0/3608930274 conn(0x563c5d242920 msgr2=0x563c5d242d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3608930274 shutdown_connections 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.711+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3608930274 wait complete. 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 -- start start 2024-09-05T12:32:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 Processor -- start 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d24e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d24eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d24f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563c5d253ff0 con 0x563c5d246cd0 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c5d135090 con 0x563c5d2463b0 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c5d254170 con 0x563c5d251570 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.712+0000 7fe4a2caf740 10 client.0 _ll_register_callbacks cb 0x563c5d189f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d24eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d24e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d24eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50546/0 (socket says 172.21.15.71:50546) 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 -- 172.21.15.71:0/1219442413 learned_addr learned my addr 172.21.15.71:0/1219442413 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49de00640 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d24f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 -- 172.21.15.71:0/1219442413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 msgr2=0x563c5d24f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d24f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 -- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 msgr2=0x563c5d24e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d24e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 -- 172.21.15.71:0/1219442413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5cf8b620 con 0x563c5d246cd0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe49ca00640 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d24eb30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe48400bf00 tx=0x7fe484011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c52543eb0e8a3da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe47ea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe47ea00640 1 -- 172.21.15.71:0/1219442413 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484004020 con 0x563c5d246cd0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe490003970 con 0x563c5d246cd0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.713+0000 7fe47ea00640 1 -- 172.21.15.71:0/1219442413 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe4840041c0 con 0x563c5d246cd0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe47ea00640 1 -- 172.21.15.71:0/1219442413 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe484024050 con 0x563c5d246cd0 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 10 client.15909 fetch_fsmap learned FSMap version 35 2024-09-05T12:32:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563c5d25fe50 con 0x563c5d246cd0 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe47ea00640 1 -- 172.21.15.71:0/1219442413 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe490003970 con 0x563c5d246cd0 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe47ea00640 1 -- 172.21.15.71:0/1219442413 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fe4840038f0 con 0x563c5d246cd0 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 10 client.15909 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 20 client.15909 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d261330 0x563c5d263760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 4 client.15909 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.714+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c5d1b9830 con 0x563c5d261330 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.715+0000 7fe49de00640 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d261330 0x563c5d263760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.715+0000 7fe49de00640 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d261330 0x563c5d263760 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0959aa19999bf4 server_cookie=281c856830837481 in_seq=0 out_seq=0 2024-09-05T12:32:20.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.715+0000 7fe47ea00640 10 client.15909 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:20.722 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe47ea00640 1 -- 172.21.15.71:0/1219442413 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c5d1b9830 con 0x563c5d261330 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe47ea00640 10 client.15909 handle_command_reply: tid=0 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 client.15909 shutdown 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 msgr2=0x563c5d24eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d24eb30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe48400bf00 tx=0x7fe484011810 comp rx=0 tx=0).stop 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 shutdown_connections 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d261330 0x563c5d263760 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d24f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d24eb30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe47ea00640 0 client.15909 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:20.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1219442413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d24e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 >> 172.21.15.71:0/1219442413 conn(0x563c5d242920 msgr2=0x563c5d248630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 shutdown_connections 2024-09-05T12:32:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1219442413 wait complete. 2024-09-05T12:32:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.716+0000 7fe4a2caf740 20 client.15909 trim_cache size 0 max 16384 2024-09-05T12:32:20.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 Processor -- start 2024-09-05T12:32:20.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 -- start start 2024-09-05T12:32:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d2690c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d269600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d247e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981b0ea0 con 0x563c5d246cd0 2024-09-05T12:32:20.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981b2aa0 con 0x563c5d2463b0 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981b1570 con 0x563c5d251570 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe49de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d2690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d269600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe49de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d2690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44108/0 (socket says 172.21.15.71:44108) 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.733+0000 7fe49de00640 1 -- 172.21.15.71:0/1706560739 learned_addr learned my addr 172.21.15.71:0/1706560739 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49de00640 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d247e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49ca00640 1 -- 172.21.15.71:0/1706560739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 msgr2=0x563c5d247e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49ca00640 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d247e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49ca00640 1 -- 172.21.15.71:0/1706560739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 msgr2=0x563c5d2690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49ca00640 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d2690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49ca00640 1 -- 172.21.15.71:0/1706560739 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5d24a1a0 con 0x563c5d246cd0 2024-09-05T12:32:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49de00640 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d247e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:32:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe49ca00640 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d269600 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe484003d10 tx=0x7fe484011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c2290cf124de0c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe489c00640 1 -- 172.21.15.71:0/1706560739 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484000e60 con 0x563c5d246cd0 2024-09-05T12:32:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe489c00640 1 -- 172.21.15.71:0/1706560739 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe484004460 con 0x563c5d246cd0 2024-09-05T12:32:20.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.734+0000 7fe489c00640 1 -- 172.21.15.71:0/1706560739 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840051c0 con 0x563c5d246cd0 2024-09-05T12:32:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1706560739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 msgr2=0x563c5d269600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d269600 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe484003d10 tx=0x7fe484011d10 comp rx=0 tx=0).stop 2024-09-05T12:32:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1706560739 shutdown_connections 2024-09-05T12:32:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d251570 0x563c5d247e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d246cd0 0x563c5d269600 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/1706560739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d2463b0 0x563c5d2690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1706560739 >> 172.21.15.71:0/1706560739 conn(0x563c5d242920 msgr2=0x563c5d247550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1706560739 shutdown_connections 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.735+0000 7fe4a2caf740 1 -- 172.21.15.71:0/1706560739 wait complete. 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 -- start start 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 Processor -- start 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d2463b0 0x563c5d259b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 0x563c5d254ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d2553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981b2370 con 0x563c5d251570 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981b1ca0 con 0x563c5d246cd0 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.736+0000 7fe4a2caf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981aee80 con 0x563c5d2463b0 2024-09-05T12:32:20.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe4a2caf740 10 client.0 _ll_register_callbacks cb 0x7fe48400bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 0x563c5d254ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 0x563c5d254ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44112/0 (socket says 172.21.15.71:44112) 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 -- 172.21.15.71:0/3133209785 learned_addr learned my addr 172.21.15.71:0/3133209785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49de00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d2463b0 0x563c5d259b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49de00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d2553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 -- 172.21.15.71:0/3133209785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d2463b0 msgr2=0x563c5d259b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d2463b0 0x563c5d259b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 -- 172.21.15.71:0/3133209785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 msgr2=0x563c5d2553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d2553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 -- 172.21.15.71:0/3133209785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c5d255c10 con 0x563c5d246cd0 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49de00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d2553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:32:20.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe49ca00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 0x563c5d254ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe484003e30 tx=0x7fe484004110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff25ebdc28aff58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.737+0000 7fe47ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe47ea00640 1 -- 172.21.15.71:0/3133209785 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe484036020 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe484004460 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe47ea00640 1 -- 172.21.15.71:0/3133209785 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe48402d040 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe47ea00640 1 -- 172.21.15.71:0/3133209785 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe484041050 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 10 client.25952 fetch_fsmap learned FSMap version 35 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x563c5d2cc030 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe47ea00640 1 -- 172.21.15.71:0/3133209785 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe484004460 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe47ea00640 1 -- 172.21.15.71:0/3133209785 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7fe4840314d0 con 0x563c5d246cd0 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 10 client.25952 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T12:32:20.745 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 10 client.25952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 20 client.25952 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d2cc770 0x563c5d2ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 4 client.25952 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.738+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c5d1b3b80 con 0x563c5d2cc770 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.739+0000 7fe49de00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d2cc770 0x563c5d2ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.739+0000 7fe47ea00640 10 client.25952 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:20.746 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.739+0000 7fe49de00640 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d2cc770 0x563c5d2ceb90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1c04a8687bbc605 server_cookie=462d94fe9c6c9ba6 in_seq=0 out_seq=0 2024-09-05T12:32:20.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:20 smithi174 ceph-mon[34313]: pgmap v140: 129 pgs: 129 active+clean; 56 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-09-05T12:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.802+0000 7fe47ea00640 1 -- 172.21.15.71:0/3133209785 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c5d1b3b80 con 0x563c5d2cc770 2024-09-05T12:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.802+0000 7fe47ea00640 10 client.25952 handle_command_reply: tid=0 2024-09-05T12:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.802+0000 7fe4a2caf740 1 client.25952 shutdown 2024-09-05T12:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 msgr2=0x563c5d254ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 0x563c5d254ea0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe484003e30 tx=0x7fe484004110 comp rx=0 tx=0).stop 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 shutdown_connections 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563c5d2cc770 0x563c5d2ceb90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563c5d251570 0x563c5d2553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c5d246cd0 0x563c5d254ea0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe484003e30 tx=0x7fe484004110 comp rx=0 tx=0).stop 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 --2- 172.21.15.71:0/3133209785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c5d2463b0 0x563c5d259b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 >> 172.21.15.71:0/3133209785 conn(0x563c5d242920 msgr2=0x563c5d24e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:20.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.803+0000 7fe47ea00640 0 client.25952 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.804+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 shutdown_connections 2024-09-05T12:32:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.804+0000 7fe4a2caf740 1 -- 172.21.15.71:0/3133209785 wait complete. 2024-09-05T12:32:20.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.804+0000 7fe4a2caf740 20 client.25952 trim_cache size 0 max 16384 2024-09-05T12:32:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:32:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:32:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:32:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:32:20.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4800a2d70 msgr2=0x7fe4800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4800a2d70 0x7fe4800a5230 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0ecda0 tx=0x7fe48c0f5550 comp rx=0 tx=0).stop 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 msgr2=0x7fe49819a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 0x7fe49819a420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe48c006010 tx=0x7fe48c004270 comp rx=0 tx=0).stop 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 shutdown_connections 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe49810bf10 0x7fe49819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe498103460 0x7fe49819a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4800a2d70 0x7fe4800a5230 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 --2- 172.21.15.71:0/2338024769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe498102b40 0x7fe49819a420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 >> 172.21.15.71:0/2338024769 conn(0x7fe4980777c0 msgr2=0x7fe498100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 shutdown_connections 2024-09-05T12:32:20.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:20.805+0000 7fe49e800640 1 -- 172.21.15.71:0/2338024769 wait complete. 2024-09-05T12:32:20.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:20 smithi123 ceph-mon[34321]: pgmap v140: 129 pgs: 129 active+clean; 56 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 268 op/s 2024-09-05T12:32:21.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 149237 3.74 MB/sec execute 77 sec latency 389.756 ms 2024-09-05T12:32:22.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 150688 3.77 MB/sec execute 78 sec latency 11.491 ms 2024-09-05T12:32:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:22 smithi174 ceph-mon[34313]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-09-05T12:32:22.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:22 smithi071 ceph-mon[29845]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-09-05T12:32:22.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:22 smithi123 ceph-mon[34321]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.7 MiB/s wr, 380 op/s 2024-09-05T12:32:23.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 152819 3.89 MB/sec execute 79 sec latency 10.906 ms 2024-09-05T12:32:24.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 153950 3.91 MB/sec execute 80 sec latency 110.456 ms 2024-09-05T12:32:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:24 smithi174 ceph-mon[34313]: pgmap v142: 129 pgs: 129 active+clean; 62 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 266 op/s 2024-09-05T12:32:24.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:24 smithi071 ceph-mon[29845]: pgmap v142: 129 pgs: 129 active+clean; 62 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 266 op/s 2024-09-05T12:32:24.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:24 smithi123 ceph-mon[34321]: pgmap v142: 129 pgs: 129 active+clean; 62 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 266 op/s 2024-09-05T12:32:25.206 INFO:tasks.workunit.client.0.smithi071.stdout: 1 155906 3.98 MB/sec execute 81 sec latency 218.293 ms 2024-09-05T12:32:26.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 157130 4.01 MB/sec execute 82 sec latency 10.682 ms 2024-09-05T12:32:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:26 smithi174 ceph-mon[34313]: pgmap v143: 129 pgs: 129 active+clean; 62 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2024-09-05T12:32:26.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:26 smithi071 ceph-mon[29845]: pgmap v143: 129 pgs: 129 active+clean; 62 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2024-09-05T12:32:26.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:26 smithi123 ceph-mon[34321]: pgmap v143: 129 pgs: 129 active+clean; 62 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.2 MiB/s wr, 239 op/s 2024-09-05T12:32:27.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 158608 4.03 MB/sec execute 83 sec latency 415.465 ms 2024-09-05T12:32:28.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 160165 4.12 MB/sec execute 84 sec latency 9.390 ms 2024-09-05T12:32:28.741 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:28 smithi071 ceph-mon[29845]: pgmap v144: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 390 op/s 2024-09-05T12:32:28.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:28 smithi174 ceph-mon[34313]: pgmap v144: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 390 op/s 2024-09-05T12:32:28.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:28 smithi123 ceph-mon[34321]: pgmap v144: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 390 op/s 2024-09-05T12:32:29.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 160677 4.09 MB/sec execute 85 sec latency 488.749 ms 2024-09-05T12:32:30.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 160677 4.04 MB/sec execute 86 sec latency 1488.794 ms 2024-09-05T12:32:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:30 smithi174 ceph-mon[34313]: pgmap v145: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-05T12:32:30.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:30 smithi071 ceph-mon[29845]: pgmap v145: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-05T12:32:30.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:30 smithi123 ceph-mon[34321]: pgmap v145: 129 pgs: 129 active+clean; 60 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-05T12:32:31.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 162401 4.10 MB/sec execute 87 sec latency 1723.222 ms 2024-09-05T12:32:32.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 163789 4.15 MB/sec execute 88 sec latency 14.585 ms 2024-09-05T12:32:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:32 smithi174 ceph-mon[34313]: pgmap v146: 129 pgs: 129 active+clean; 66 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.0 MiB/s wr, 374 op/s 2024-09-05T12:32:32.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:32 smithi071 ceph-mon[29845]: pgmap v146: 129 pgs: 129 active+clean; 66 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.0 MiB/s wr, 374 op/s 2024-09-05T12:32:32.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:32 smithi123 ceph-mon[34321]: pgmap v146: 129 pgs: 129 active+clean; 66 MiB data, 891 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.0 MiB/s wr, 374 op/s 2024-09-05T12:32:33.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 165714 4.19 MB/sec execute 89 sec latency 147.260 ms 2024-09-05T12:32:33.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 -- 172.21.15.71:0/3701560113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 msgr2=0x7f06f0110600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 --2- 172.21.15.71:0/3701560113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f0110600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009ca0 tx=0x7f06e802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 -- 172.21.15.71:0/3701560113 shutdown_connections 2024-09-05T12:32:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 --2- 172.21.15.71:0/3701560113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f0110600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 --2- 172.21.15.71:0/3701560113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 0x7f06f010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 --2- 172.21.15.71:0/3701560113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 -- 172.21.15.71:0/3701560113 >> 172.21.15.71:0/3701560113 conn(0x7f06f00777c0 msgr2=0x7f06f00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 -- 172.21.15.71:0/3701560113 shutdown_connections 2024-09-05T12:32:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.934+0000 7f06f5e00640 1 -- 172.21.15.71:0/3701560113 wait complete. 2024-09-05T12:32:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.935+0000 7f06f5e00640 1 Processor -- start 2024-09-05T12:32:33.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.935+0000 7f06f5e00640 1 -- start start 2024-09-05T12:32:33.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 0x7f06f019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0113f30 con 0x7f06f0100660 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06f0113db0 con 0x7f06f0101010 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01140b0 con 0x7f06f010e210 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47336/0 (socket says 172.21.15.71:47336) 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 -- 172.21.15.71:0/1161429250 learned_addr learned my addr 172.21.15.71:0/1161429250 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5400640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06efe00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 0x7f06f019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 -- 172.21.15.71:0/1161429250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 msgr2=0x7f06f019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 -- 172.21.15.71:0/1161429250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 msgr2=0x7f06f019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 0x7f06f019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 -- 172.21.15.71:0/1161429250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f019f3f0 con 0x7f06f0100660 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f5400640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06efe00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 0x7f06f019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.936+0000 7f06f4a00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f019a400 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f06e400e9f0 tx=0x7f06e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06ed600640 1 -- 172.21.15.71:0/1161429250 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e400cd80 con 0x7f06f0100660 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f019f6e0 con 0x7f06f0100660 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06ed600640 1 -- 172.21.15.71:0/1161429250 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e4010910 con 0x7f06f0100660 2024-09-05T12:32:33.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06ed600640 1 -- 172.21.15.71:0/1161429250 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e40188f0 con 0x7f06f0100660 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01a6f70 con 0x7f06f0100660 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 Processor -- start 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06ed600640 1 -- 172.21.15.71:0/1161429250 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f06e4010ac0 con 0x7f06f0100660 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 -- start start 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68aa7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ab2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68ab4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560c68a0d100 con 0x560c68aa7cd0 2024-09-05T12:32:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560c689b51b0 con 0x560c68ab2570 2024-09-05T12:32:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06fa45c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560c68a090c0 con 0x560c68aa73b0 2024-09-05T12:32:33.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68aa7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68aa7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58750/0 (socket says 172.21.15.71:58750) 2024-09-05T12:32:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 -- 172.21.15.71:0/984378579 learned_addr learned my addr 172.21.15.71:0/984378579 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f4a00640 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ab2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06efe00640 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68ab4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 -- 172.21.15.71:0/984378579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 msgr2=0x560c68ab4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68ab4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 -- 172.21.15.71:0/984378579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 msgr2=0x560c68ab2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ab2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 -- 172.21.15.71:0/984378579 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c68720210 con 0x560c68aa73b0 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06efe00640 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68ab4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06f5400640 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68aa7790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06f0074af0 tx=0x7f06e8033000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ecf87e0ba86e24d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06e1c00640 1 -- 172.21.15.71:0/984378579 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e8003b70 con 0x560c68aa73b0 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06e1c00640 1 -- 172.21.15.71:0/984378579 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8036040 con 0x560c68aa73b0 2024-09-05T12:32:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.938+0000 7f06e1c00640 1 -- 172.21.15.71:0/984378579 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e80045c0 con 0x560c68aa73b0 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06ed600640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06d80a2d70 0x7f06d80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06ed600640 1 -- 172.21.15.71:0/1161429250 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f06e40cfb20 con 0x7f06f0100660 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/984378579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 msgr2=0x560c68aa7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68aa7790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06f0074af0 tx=0x7f06e8033000 comp rx=0 tx=0).stop 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f4a00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06d80a2d70 0x7f06d80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/984378579 shutdown_connections 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68ab4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ab2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- 172.21.15.71:0/984378579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68aa7790 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/984378579 >> 172.21.15.71:0/984378579 conn(0x560c68aa3920 msgr2=0x560c68aa3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/984378579 shutdown_connections 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/984378579 wait complete. 2024-09-05T12:32:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f4a00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06d80a2d70 0x7f06d80a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06e409bce0 tx=0x7f06e40d35d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- start start 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 Processor -- start 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 0x560c68aaf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa7cd0 0x560c68aafab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68aafff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560c68ab4fc0 con 0x560c68aa73b0 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560c68996090 con 0x560c68aa7cd0 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560c68ab5140 con 0x560c68ab2570 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 10 client.0 _ll_register_callbacks cb 0x560c689eaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 0x560c68aaf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa7cd0 0x560c68aafab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06efe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa7cd0 0x560c68aafab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35528/0 (socket says 172.21.15.71:35528) 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 0x560c68aaf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47354/0 (socket says 172.21.15.71:47354) 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06efe00640 1 -- 172.21.15.71:0/1747089707 learned_addr learned my addr 172.21.15.71:0/1747089707 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68aafff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 -- 172.21.15.71:0/1747089707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 msgr2=0x560c68aafff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68aafff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 -- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa7cd0 msgr2=0x560c68aafab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa7cd0 0x560c68aafab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 -- 172.21.15.71:0/1747089707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c687ec620 con 0x560c68aa73b0 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06f5400640 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 0x560c68aaf570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009bf0 tx=0x7f06e8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64023399090be40e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06d6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06d6000640 1 -- 172.21.15.71:0/1747089707 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e805a060 con 0x560c68aa73b0 2024-09-05T12:32:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e8036040 con 0x560c68aa73b0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06d6000640 1 -- 172.21.15.71:0/1747089707 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f06e8031650 con 0x560c68aa73b0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06d6000640 1 -- 172.21.15.71:0/1747089707 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f06e8004850 con 0x560c68aa73b0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 10 client.15924 fetch_fsmap learned FSMap version 35 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c68ab4fc0 con 0x560c68aa73b0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.943+0000 7f06d6000640 1 -- 172.21.15.71:0/1747089707 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e8036040 con 0x560c68aa73b0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06d6000640 1 -- 172.21.15.71:0/1747089707 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f06e8044070 con 0x560c68aa73b0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 10 client.15924 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 20 client.15924 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68aca5c0 0x560c68acc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 4 client.15924 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c68a1a830 con 0x560c68aca5c0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06efe00640 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68aca5c0 0x560c68acc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06efe00640 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68aca5c0 0x560c68acc9b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d3df779a98e5d82 server_cookie=522071c6579a902 in_seq=0 out_seq=0 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06d6000640 10 client.15924 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06d6000640 1 -- 172.21.15.71:0/1747089707 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c68a1a830 con 0x560c68aca5c0 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06d6000640 10 client.15924 handle_command_reply: tid=0 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 client.15924 shutdown 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 msgr2=0x560c68aaf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 0x560c68aaf570 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f06e8009bf0 tx=0x7f06e8009650 comp rx=0 tx=0).stop 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 shutdown_connections 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68aca5c0 0x560c68acc9b0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68aafff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa7cd0 0x560c68aafab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06d6000640 0 client.15924 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 --2- 172.21.15.71:0/1747089707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa73b0 0x560c68aaf570 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 >> 172.21.15.71:0/1747089707 conn(0x560c68aa3920 msgr2=0x560c68aa9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 shutdown_connections 2024-09-05T12:32:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 1 -- 172.21.15.71:0/1747089707 wait complete. 2024-09-05T12:32:33.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.947+0000 7f06fa45c740 20 client.15924 trim_cache size 0 max 16384 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 Processor -- start 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- start start 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68ad2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ad27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68aa8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aca30 con 0x560c68aa7cd0 2024-09-05T12:32:33.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ae630 con 0x560c68ab2570 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ad100 con 0x560c68aa73b0 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ad27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ad27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47362/0 (socket says 172.21.15.71:47362) 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/2686254749 learned_addr learned my addr 172.21.15.71:0/2686254749 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/2686254749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 msgr2=0x560c68ad2260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68ad2260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/2686254749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 msgr2=0x560c68aa8e10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68aa8e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/2686254749 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c68aa82e0 con 0x560c68aa7cd0 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ad27a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x560c68ab0bb0 tx=0x7f06dc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a67637049c715d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06e1200640 1 -- 172.21.15.71:0/2686254749 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc004220 con 0x560c68aa7cd0 2024-09-05T12:32:33.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06e1200640 1 -- 172.21.15.71:0/2686254749 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06dc010e40 con 0x560c68aa7cd0 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06e1200640 1 -- 172.21.15.71:0/2686254749 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc0253f0 con 0x560c68aa7cd0 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/2686254749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 msgr2=0x560c68ad27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ad27a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x560c68ab0bb0 tx=0x7f06dc00b390 comp rx=0 tx=0).stop 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/2686254749 shutdown_connections 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68ab2570 0x560c68aa8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68ad27a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- 172.21.15.71:0/2686254749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68aa73b0 0x560c68ad2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/2686254749 >> 172.21.15.71:0/2686254749 conn(0x560c68aa3920 msgr2=0x560c68aa8550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/2686254749 shutdown_connections 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/2686254749 wait complete. 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- start start 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 Processor -- start 2024-09-05T12:32:33.977 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa73b0 0x560c68ac33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68abe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68abeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01adf00 con 0x560c68aa7cd0 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01ad830 con 0x560c68aa73b0 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06f01aaa30 con 0x560c68ab2570 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 10 client.0 _ll_register_callbacks cb 0x7f06e40d34a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68abe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68abe750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47374/0 (socket says 172.21.15.71:47374) 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/3826822913 learned_addr learned my addr 172.21.15.71:0/3826822913 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:33.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/3826822913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 msgr2=0x560c68abeee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68abeee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa73b0 msgr2=0x560c68ac33e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa73b0 0x560c68ac33e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 -- 172.21.15.71:0/3826822913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c68b34230 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06efe00640 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68abe750 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06dc011010 tx=0x7f06dc003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef4cba1d6fa8a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06d6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06d6000640 1 -- 172.21.15.71:0/3826822913 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc010b60 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06d6000640 1 -- 172.21.15.71:0/3826822913 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f06dc004020 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06dc010e40 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06d6000640 1 -- 172.21.15.71:0/3826822913 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f06dc03b050 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 10 client.15930 fetch_fsmap learned FSMap version 35 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.964+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560c68b353d0 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06d6000640 1 -- 172.21.15.71:0/3826822913 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06dc010e40 con 0x560c68aa7cd0 2024-09-05T12:32:33.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06d6000640 1 -- 172.21.15.71:0/3826822913 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) ==== 8386+0+0 (secure 0 0 0) 0x7f06dc036030 con 0x560c68aa7cd0 2024-09-05T12:32:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06fa45c740 10 client.15930 fetch_fsmap finished waiting for FSMap version 35 2024-09-05T12:32:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06fa45c740 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06fa45c740 20 client.15930 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06fa45c740 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68b35b70 0x560c68b37f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06fa45c740 4 client.15930 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:32:33.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.969+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c68a14b80 con 0x560c68b35b70 2024-09-05T12:32:33.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:33.983+0000 7f06f5400640 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68b35b70 0x560c68b37f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.042+0000 7f06f5400640 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68b35b70 0x560c68b37f90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4043e1b4d01fe431 server_cookie=3fdf9010dd591cbb in_seq=0 out_seq=0 2024-09-05T12:32:34.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.042+0000 7f06d6000640 10 client.15930 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:34.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06d6000640 1 -- 172.21.15.71:0/3826822913 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c68a14b80 con 0x560c68b35b70 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06d6000640 10 client.15930 handle_command_reply: tid=0 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 client.15930 shutdown 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 msgr2=0x560c68abe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68abe750 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f06dc011010 tx=0x7f06dc003cd0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 shutdown_connections 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560c68b35b70 0x560c68b37f90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560c68ab2570 0x560c68abeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c68aa7cd0 0x560c68abe750 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 --2- 172.21.15.71:0/3826822913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560c68aa73b0 0x560c68ac33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 >> 172.21.15.71:0/3826822913 conn(0x560c68aa3920 msgr2=0x560c68aaf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06d6000640 0 client.15930 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:34.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 shutdown_connections 2024-09-05T12:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 1 -- 172.21.15.71:0/3826822913 wait complete. 2024-09-05T12:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06fa45c740 20 client.15930 trim_cache size 0 max 16384 2024-09-05T12:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:32:34.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06d80a2d70 msgr2=0x7f06d80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06d80a2d70 0x7f06d80a5230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f06e409bce0 tx=0x7f06e40d35d0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 msgr2=0x7f06f019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f019a400 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f06e400e9f0 tx=0x7f06e400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 shutdown_connections 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06f010e210 0x7f06f019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0101010 0x7f06f019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06d80a2d70 0x7f06d80a5230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 --2- 172.21.15.71:0/1161429250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06f0100660 0x7f06f019a400 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 >> 172.21.15.71:0/1161429250 conn(0x7f06f00777c0 msgr2=0x7f06f00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 shutdown_connections 2024-09-05T12:32:34.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:34.106+0000 7f06f5e00640 1 -- 172.21.15.71:0/1161429250 wait complete. 2024-09-05T12:32:34.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 166963 4.26 MB/sec execute 90 sec latency 279.850 ms 2024-09-05T12:32:34.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:34 smithi071 ceph-mon[29845]: pgmap v147: 129 pgs: 129 active+clean; 67 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 279 op/s 2024-09-05T12:32:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:34 smithi174 ceph-mon[34313]: pgmap v147: 129 pgs: 129 active+clean; 67 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 279 op/s 2024-09-05T12:32:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:34 smithi123 ceph-mon[34321]: pgmap v147: 129 pgs: 129 active+clean; 67 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 279 op/s 2024-09-05T12:32:35.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 167805 4.24 MB/sec execute 91 sec latency 152.165 ms 2024-09-05T12:32:36.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 169632 4.30 MB/sec execute 92 sec latency 292.251 ms 2024-09-05T12:32:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:36 smithi174 ceph-mon[34313]: pgmap v148: 129 pgs: 129 active+clean; 67 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-05T12:32:36.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:36 smithi071 ceph-mon[29845]: pgmap v148: 129 pgs: 129 active+clean; 67 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-05T12:32:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:36 smithi123 ceph-mon[34321]: pgmap v148: 129 pgs: 129 active+clean; 67 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-05T12:32:37.207 INFO:tasks.workunit.client.0.smithi071.stdout: 1 170949 4.34 MB/sec execute 93 sec latency 12.912 ms 2024-09-05T12:32:38.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 172869 4.37 MB/sec execute 94 sec latency 149.564 ms 2024-09-05T12:32:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:38 smithi174 ceph-mon[34313]: pgmap v149: 129 pgs: 129 active+clean; 63 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 413 op/s 2024-09-05T12:32:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:38 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:38.850 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:38 smithi071 ceph-mon[29845]: pgmap v149: 129 pgs: 129 active+clean; 63 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 413 op/s 2024-09-05T12:32:38.850 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:38 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:38 smithi123 ceph-mon[34321]: pgmap v149: 129 pgs: 129 active+clean; 63 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 413 op/s 2024-09-05T12:32:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:38 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:32:39.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 174095 4.44 MB/sec execute 95 sec latency 296.696 ms 2024-09-05T12:32:40.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 174191 4.40 MB/sec execute 96 sec latency 1078.320 ms 2024-09-05T12:32:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:40 smithi174 ceph-mon[34313]: pgmap v150: 129 pgs: 129 active+clean; 63 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-09-05T12:32:40.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:40 smithi071 ceph-mon[29845]: pgmap v150: 129 pgs: 129 active+clean; 63 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-09-05T12:32:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:40 smithi123 ceph-mon[34321]: pgmap v150: 129 pgs: 129 active+clean; 63 MiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.9 MiB/s wr, 262 op/s 2024-09-05T12:32:41.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 176168 4.43 MB/sec execute 97 sec latency 19.205 ms 2024-09-05T12:32:42.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 177659 4.50 MB/sec execute 98 sec latency 138.902 ms 2024-09-05T12:32:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:42 smithi174 ceph-mon[34313]: pgmap v151: 129 pgs: 129 active+clean; 66 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-05T12:32:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:42 smithi071 ceph-mon[29845]: pgmap v151: 129 pgs: 129 active+clean; 66 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-05T12:32:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:42 smithi123 ceph-mon[34321]: pgmap v151: 129 pgs: 129 active+clean; 66 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 364 op/s 2024-09-05T12:32:43.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 179570 4.52 MB/sec execute 99 sec latency 11.156 ms 2024-09-05T12:32:44.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 180994 4.59 MB/sec execute 100 sec latency 205.679 ms 2024-09-05T12:32:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:44 smithi174 ceph-mon[34313]: pgmap v152: 129 pgs: 129 active+clean; 66 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-09-05T12:32:44.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:44 smithi071 ceph-mon[29845]: pgmap v152: 129 pgs: 129 active+clean; 66 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-09-05T12:32:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:44 smithi123 ceph-mon[34321]: pgmap v152: 129 pgs: 129 active+clean; 66 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.1 MiB/s wr, 281 op/s 2024-09-05T12:32:45.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 181127 4.56 MB/sec execute 101 sec latency 966.663 ms 2024-09-05T12:32:46.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 181826 4.52 MB/sec execute 102 sec latency 69.875 ms 2024-09-05T12:32:46.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:46 smithi071 ceph-mon[29845]: pgmap v153: 129 pgs: 129 active+clean; 66 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 265 op/s 2024-09-05T12:32:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:46 smithi123 ceph-mon[34321]: pgmap v153: 129 pgs: 129 active+clean; 66 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 265 op/s 2024-09-05T12:32:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:46 smithi174 ceph-mon[34313]: pgmap v153: 129 pgs: 129 active+clean; 66 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 265 op/s 2024-09-05T12:32:47.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 184269 4.61 MB/sec execute 103 sec latency 13.128 ms 2024-09-05T12:32:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 -- 172.21.15.71:0/3457687526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 msgr2=0x7fbe98107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.238 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/3457687526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe98107ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009860 tx=0x7fbe8c02a340 comp rx=0 tx=0).stop 2024-09-05T12:32:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 -- 172.21.15.71:0/3457687526 shutdown_connections 2024-09-05T12:32:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/3457687526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98102250 0x7fbe9810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/3457687526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe98101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/3457687526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe98107ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.239 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 -- 172.21.15.71:0/3457687526 >> 172.21.15.71:0/3457687526 conn(0x7fbe980fd5d0 msgr2=0x7fbe980ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 -- 172.21.15.71:0/3457687526 shutdown_connections 2024-09-05T12:32:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.232+0000 7fbe9ec00640 1 -- 172.21.15.71:0/3457687526 wait complete. 2024-09-05T12:32:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.233+0000 7fbe9ec00640 1 Processor -- start 2024-09-05T12:32:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.233+0000 7fbe9ec00640 1 -- start start 2024-09-05T12:32:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe9819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.240 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98102250 0x7fbe9819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe9819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111d00 con 0x7fbe98102250 2024-09-05T12:32:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111b80 con 0x7fbe981078c0 2024-09-05T12:32:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe98111e80 con 0x7fbe981018b0 2024-09-05T12:32:47.241 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe9819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe9819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe9819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48054/0 (socket says 172.21.15.71:48054) 2024-09-05T12:32:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 -- 172.21.15.71:0/1811918020 learned_addr learned my addr 172.21.15.71:0/1811918020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98102250 0x7fbe9819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.242 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 -- 172.21.15.71:0/1811918020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 msgr2=0x7fbe9819ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe9819ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 -- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98102250 msgr2=0x7fbe9819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98102250 0x7fbe9819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9d800640 1 -- 172.21.15.71:0/1811918020 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9819f3c0 con 0x7fbe981018b0 2024-09-05T12:32:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.234+0000 7fbe9e200640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe9819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:47.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.235+0000 7fbe9d800640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe9819a3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c035040 tx=0x7fbe8c033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.235+0000 7fbe96000640 1 -- 172.21.15.71:0/1811918020 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c037040 con 0x7fbe981018b0 2024-09-05T12:32:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.235+0000 7fbe96000640 1 -- 172.21.15.71:0/1811918020 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe8c040070 con 0x7fbe981018b0 2024-09-05T12:32:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.235+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9819f650 con 0x7fbe981018b0 2024-09-05T12:32:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.235+0000 7fbe96000640 1 -- 172.21.15.71:0/1811918020 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c003950 con 0x7fbe981018b0 2024-09-05T12:32:47.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.235+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe981a6f40 con 0x7fbe981018b0 2024-09-05T12:32:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.238+0000 7fbea300e740 1 Processor -- start 2024-09-05T12:32:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.238+0000 7fbe96000640 1 -- 172.21.15.71:0/1811918020 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbe8c0040e0 con 0x7fbe981018b0 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.238+0000 7fbea300e740 1 -- start start 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbea300e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244122790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbea300e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbea300e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a24412f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbea300e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a244088100 con 0x55a24412d570 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbea300e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a2440301b0 con 0x55a2441223b0 2024-09-05T12:32:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbea300e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a2440840c0 con 0x55a244122cd0 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244122790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37170/0 (socket says 172.21.15.71:37170) 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 -- 172.21.15.71:0/754541285 learned_addr learned my addr 172.21.15.71:0/754541285 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a24412f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 -- 172.21.15.71:0/754541285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 msgr2=0x55a24412d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 -- 172.21.15.71:0/754541285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 msgr2=0x55a24412f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a24412f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 -- 172.21.15.71:0/754541285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a243d9b210 con 0x55a2441223b0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.239+0000 7fbe9e200640 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244122790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe90011790 tx=0x7fbe90011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57609aca3123be43 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbe8aa00640 1 -- 172.21.15.71:0/754541285 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90003c30 con 0x55a2441223b0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbe96000640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe800a2cf0 0x7fbe800a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbe8aa00640 1 -- 172.21.15.71:0/754541285 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe9000fe80 con 0x55a2441223b0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbe8aa00640 1 -- 172.21.15.71:0/754541285 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe900193e0 con 0x55a2441223b0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbe96000640 1 -- 172.21.15.71:0/1811918020 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbe8c0ef650 con 0x7fbe981018b0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbe9d800640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe800a2cf0 0x7fbe800a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbea300e740 1 -- 172.21.15.71:0/754541285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 msgr2=0x55a244122790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.240+0000 7fbea300e740 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244122790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbe90011790 tx=0x7fbe90011c50 comp rx=0 tx=0).stop 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 -- 172.21.15.71:0/754541285 shutdown_connections 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a24412f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbe9d800640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe800a2cf0 0x7fbe800a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0ecc20 tx=0x7fbe8c0f5550 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 --2- 172.21.15.71:0/754541285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244122790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 -- 172.21.15.71:0/754541285 >> 172.21.15.71:0/754541285 conn(0x55a24411e920 msgr2=0x55a24411ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 -- 172.21.15.71:0/754541285 shutdown_connections 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 -- 172.21.15.71:0/754541285 wait complete. 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 -- start start 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.241+0000 7fbea300e740 1 Processor -- start 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a24412d570 0x55a24412b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a24412fff0 con 0x55a2441223b0 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a244011090 con 0x55a24412d570 2024-09-05T12:32:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a244130170 con 0x55a244122cd0 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbea300e740 10 client.0 _ll_register_callbacks cb 0x55a244065f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48070/0 (socket says 172.21.15.71:48070) 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2541342644 learned_addr learned my addr 172.21.15.71:0/2541342644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a24412d570 0x55a24412b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a24412d570 msgr2=0x55a24412b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a24412d570 0x55a24412b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2541342644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 msgr2=0x55a24412a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2541342644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a243e67620 con 0x55a244122cd0 2024-09-05T12:32:47.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.242+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412ac30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe840116f0 tx=0x7fbe84011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b44e47723002d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbe7f400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbe7f400640 1 -- 172.21.15.71:0/2541342644 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8401e020 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a244095830 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbe7f400640 1 -- 172.21.15.71:0/2541342644 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbe84017050 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbe7f400640 1 -- 172.21.15.71:0/2541342644 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbe84026050 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbea300e740 10 client.25987 fetch_fsmap learned FSMap version 36 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a24413be50 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.243+0000 7fbe7f400640 1 -- 172.21.15.71:0/2541342644 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a244095830 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbe7f400640 1 -- 172.21.15.71:0/2541342644 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fbe84019070 con 0x55a244122cd0 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbea300e740 10 client.25987 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbea300e740 10 client.25987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbea300e740 20 client.25987 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:47.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbea300e740 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a24413d380 0x55a24413f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbea300e740 4 client.25987 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a24408fb80 con 0x55a24413d380 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a24413d380 0x55a24413f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbe7f400640 10 client.25987 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.244+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a24413d380 0x55a24413f770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9319f388cd21e309 server_cookie=866e8b2bed2e1428 in_seq=0 out_seq=0 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbe7f400640 1 -- 172.21.15.71:0/2541342644 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a24408fb80 con 0x55a24413d380 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbe7f400640 10 client.25987 handle_command_reply: tid=0 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 client.25987 shutdown 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 msgr2=0x55a24412ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412ac30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbe840116f0 tx=0x7fbe84011bb0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 shutdown_connections 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a24413d380 0x55a24413f770 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a24412d570 0x55a24412b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a24412ac30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbe7f400640 0 client.25987 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 --2- 172.21.15.71:0/2541342644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.245+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 >> 172.21.15.71:0/2541342644 conn(0x55a24411e920 msgr2=0x55a244124630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.246+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 shutdown_connections 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.246+0000 7fbea300e740 1 -- 172.21.15.71:0/2541342644 wait complete. 2024-09-05T12:32:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.246+0000 7fbea300e740 20 client.25987 trim_cache size 0 max 16384 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 Processor -- start 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 -- start start 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 0x55a244145030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24412d570 0x55a244145570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981b0ed0 con 0x55a2441223b0 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981b2a70 con 0x55a244122cd0 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbea300e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981b15a0 con 0x55a24412d570 2024-09-05T12:32:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbe9ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 0x55a244145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbe9e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbe9ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 0x55a244145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37186/0 (socket says 172.21.15.71:37186) 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.262+0000 7fbe9ce00640 1 -- 172.21.15.71:0/3756213289 learned_addr learned my addr 172.21.15.71:0/3756213289 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9e200640 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24412d570 0x55a244145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9ce00640 1 -- 172.21.15.71:0/3756213289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24412d570 msgr2=0x55a244145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24412d570 0x55a244145570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9ce00640 1 -- 172.21.15.71:0/3756213289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 msgr2=0x55a24412c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9ce00640 1 -- 172.21.15.71:0/3756213289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a24413fae0 con 0x55a244122cd0 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9e200640 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 0x55a244145030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe84003bc0 tx=0x7fbe8400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ff7d8f491ac8574 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe8a000640 1 -- 172.21.15.71:0/3756213289 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8402f070 con 0x55a244122cd0 2024-09-05T12:32:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe8a000640 1 -- 172.21.15.71:0/3756213289 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe840035f0 con 0x55a244122cd0 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.263+0000 7fbe8a000640 1 -- 172.21.15.71:0/3756213289 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe8401e070 con 0x55a244122cd0 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 -- 172.21.15.71:0/3756213289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 msgr2=0x55a244145030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 0x55a244145030 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbe84003bc0 tx=0x7fbe8400f790 comp rx=0 tx=0).stop 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 -- 172.21.15.71:0/3756213289 shutdown_connections 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a24412d570 0x55a244145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a244122cd0 0x55a244145030 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 --2- 172.21.15.71:0/3756213289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2441223b0 0x55a24412c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 -- 172.21.15.71:0/3756213289 >> 172.21.15.71:0/3756213289 conn(0x55a24411e920 msgr2=0x55a24412be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:47.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 -- 172.21.15.71:0/3756213289 shutdown_connections 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.264+0000 7fbea300e740 1 -- 172.21.15.71:0/3756213289 wait complete. 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 -- start start 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 Processor -- start 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244135b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a244130e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a244131420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981b23a0 con 0x55a24412d570 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981b1cd0 con 0x55a2441223b0 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe981aeeb0 con 0x55a244122cd0 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbea300e740 10 client.0 _ll_register_callbacks cb 0x7fbe84009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.265+0000 7fbe9ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a244130e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244135b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a244130e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48098/0 (socket says 172.21.15.71:48098) 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2268245656 learned_addr learned my addr 172.21.15.71:0/2268245656 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a244131420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 msgr2=0x55a244135b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244135b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2268245656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 msgr2=0x55a244131420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a244131420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 -- 172.21.15.71:0/2268245656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a244131c50 con 0x55a244122cd0 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a244131420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe9ce00640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a244130e80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe84003c10 tx=0x7fbe840052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c6d46bcc2c998db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe7f400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:32:47.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe7f400640 1 -- 172.21.15.71:0/2268245656 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84038060 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.266+0000 7fbe7f400640 1 -- 172.21.15.71:0/2268245656 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbe840054b0 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe840035f0 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbe7f400640 1 -- 172.21.15.71:0/2268245656 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbe8403f050 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 10 client.25993 fetch_fsmap learned FSMap version 36 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a2441a80e0 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbe7f400640 1 -- 172.21.15.71:0/2268245656 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe840035f0 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbe7f400640 1 -- 172.21.15.71:0/2268245656 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fbe84019040 con 0x55a244122cd0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 10 client.25993 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 10 client.25993 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 20 client.25993 populate_metadata read hostname 'smithi071' 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2441a86e0 0x55a2441aab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 4 client.25993 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.267+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a24400c7c0 con 0x55a2441a86e0 2024-09-05T12:32:47.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.268+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2441a86e0 0x55a2441aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.268+0000 7fbe9e200640 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2441a86e0 0x55a2441aab00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00ee7b284e2d7e server_cookie=93581398297ed49e in_seq=0 out_seq=0 2024-09-05T12:32:47.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.268+0000 7fbe7f400640 10 client.25993 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbe7f400640 1 -- 172.21.15.71:0/2268245656 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a24400c7c0 con 0x55a2441a86e0 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbe7f400640 10 client.25993 handle_command_reply: tid=0 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 client.25993 shutdown 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 msgr2=0x55a244130e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a244130e80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbe84003c10 tx=0x7fbe840052c0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 shutdown_connections 2024-09-05T12:32:47.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2441a86e0 0x55a2441aab00 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a24412d570 0x55a244131420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a244122cd0 0x55a244130e80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbe7f400640 0 client.25993 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 --2- 172.21.15.71:0/2268245656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2441223b0 0x55a244135b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 >> 172.21.15.71:0/2268245656 conn(0x55a24411e920 msgr2=0x55a24412aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 shutdown_connections 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.321+0000 7fbea300e740 1 -- 172.21.15.71:0/2268245656 wait complete. 2024-09-05T12:32:47.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbea300e740 20 client.25993 trim_cache size 0 max 16384 2024-09-05T12:32:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:32:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:32:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:32:47.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe800a2cf0 msgr2=0x7fbe800a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe800a2cf0 0x7fbe800a51b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0ecc20 tx=0x7fbe8c0f5550 comp rx=0 tx=0).stop 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 msgr2=0x7fbe9819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe9819a3d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c035040 tx=0x7fbe8c033b60 comp rx=0 tx=0).stop 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 shutdown_connections 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe981078c0 0x7fbe9819ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbe800a2cf0 0x7fbe800a51b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98102250 0x7fbe9819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.322+0000 7fbe9ec00640 1 --2- 172.21.15.71:0/1811918020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe981018b0 0x7fbe9819a3d0 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c035040 tx=0x7fbe8c033b60 comp rx=0 tx=0).stop 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.323+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 >> 172.21.15.71:0/1811918020 conn(0x7fbe980fd5d0 msgr2=0x7fbe9810ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.323+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 shutdown_connections 2024-09-05T12:32:47.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:32:47.323+0000 7fbe9ec00640 1 -- 172.21.15.71:0/1811918020 wait complete. 2024-09-05T12:32:48.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 185493 4.63 MB/sec execute 104 sec latency 44.086 ms 2024-09-05T12:32:48.812 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:48 smithi174 ceph-mon[34313]: pgmap v154: 129 pgs: 129 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.9 MiB/s wr, 374 op/s 2024-09-05T12:32:48.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:48 smithi071 ceph-mon[29845]: pgmap v154: 129 pgs: 129 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.9 MiB/s wr, 374 op/s 2024-09-05T12:32:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:48 smithi123 ceph-mon[34321]: pgmap v154: 129 pgs: 129 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.9 MiB/s wr, 374 op/s 2024-09-05T12:32:49.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 187650 4.70 MB/sec execute 105 sec latency 138.282 ms 2024-09-05T12:32:50.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 187650 4.65 MB/sec execute 106 sec latency 1138.325 ms 2024-09-05T12:32:50.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:50 smithi071 ceph-mon[29845]: pgmap v155: 129 pgs: 129 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-09-05T12:32:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:50 smithi123 ceph-mon[34321]: pgmap v155: 129 pgs: 129 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-09-05T12:32:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:50 smithi174 ceph-mon[34313]: pgmap v155: 129 pgs: 129 active+clean; 76 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 224 op/s 2024-09-05T12:32:51.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 188727 4.66 MB/sec execute 107 sec latency 1145.284 ms 2024-09-05T12:32:52.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 190615 4.69 MB/sec execute 108 sec latency 173.103 ms 2024-09-05T12:32:52.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:52 smithi071 ceph-mon[29845]: pgmap v156: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-09-05T12:32:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:52 smithi123 ceph-mon[34321]: pgmap v156: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-09-05T12:32:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:52 smithi174 ceph-mon[34313]: pgmap v156: 129 pgs: 129 active+clean; 74 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-09-05T12:32:53.208 INFO:tasks.workunit.client.0.smithi071.stdout: 1 192225 4.75 MB/sec execute 109 sec latency 16.530 ms 2024-09-05T12:32:54.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 194467 4.82 MB/sec execute 110 sec latency 32.770 ms 2024-09-05T12:32:54.708 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:54 smithi071 ceph-mon[29845]: pgmap v157: 129 pgs: 129 active+clean; 73 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-05T12:32:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:54 smithi123 ceph-mon[34321]: pgmap v157: 129 pgs: 129 active+clean; 73 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-05T12:32:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:54 smithi174 ceph-mon[34313]: pgmap v157: 129 pgs: 129 active+clean; 73 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 262 op/s 2024-09-05T12:32:55.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 194467 4.78 MB/sec execute 111 sec latency 1032.822 ms 2024-09-05T12:32:55.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:55 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:32:55.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:55 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:32:56.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:55 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:32:56.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 195504 4.79 MB/sec execute 112 sec latency 1298.970 ms 2024-09-05T12:32:56.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:56 smithi071 ceph-mon[29845]: pgmap v158: 129 pgs: 129 active+clean; 73 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 248 op/s 2024-09-05T12:32:56.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:56 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:32:56.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:56 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:32:56.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:56 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:32:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:56 smithi123 ceph-mon[34321]: pgmap v158: 129 pgs: 129 active+clean; 73 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 248 op/s 2024-09-05T12:32:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:56 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:32:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:56 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:32:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:56 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:32:56.956 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:32:56.956 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:32:56.957 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:32:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:56 smithi174 ceph-mon[34313]: pgmap v158: 129 pgs: 129 active+clean; 73 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 248 op/s 2024-09-05T12:32:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:56 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:32:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:56 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:32:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:56 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:32:57.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 197537 4.82 MB/sec execute 113 sec latency 13.780 ms 2024-09-05T12:32:57.360 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:32:57.841 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:57 smithi071 ceph-mon[29845]: pgmap v159: 129 pgs: 129 active+clean; 74 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-09-05T12:32:57.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:57 smithi123 ceph-mon[34321]: pgmap v159: 129 pgs: 129 active+clean; 74 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 -- 172.21.15.71:0/1729108243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4104010 msgr2=0x7f87d41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1729108243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4104010 0x7f87d41043f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009900 tx=0x7f87c402a410 comp rx=0 tx=0).stop 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 -- 172.21.15.71:0/1729108243 shutdown_connections 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1729108243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d4109140 0x7f87d4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1729108243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d41049c0 0x7f87d4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1729108243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4104010 0x7f87d41043f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:57.972 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 -- 172.21.15.71:0/1729108243 >> 172.21.15.71:0/1729108243 conn(0x7f87d40ffa50 msgr2=0x7f87d4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 -- 172.21.15.71:0/1729108243 shutdown_connections 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.965+0000 7f87d9a00640 1 -- 172.21.15.71:0/1729108243 wait complete. 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.966+0000 7f87d9a00640 1 Processor -- start 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 -- start start 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d4104010 0x7f87d419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d41049c0 0x7f87d419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4109140 0x7f87d419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f87d41141b0 con 0x7f87d41049c0 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4114030 con 0x7f87d4109140 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d9a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4114330 con 0x7f87d4104010 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d41049c0 0x7f87d419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:57.973 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d41049c0 0x7f87d419ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35202/0 (socket says 172.21.15.71:35202) 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d2000640 1 -- 172.21.15.71:0/1563417434 learned_addr learned my addr 172.21.15.71:0/1563417434 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.967+0000 7f87d3400640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4109140 0x7f87d419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d2a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d4104010 0x7f87d419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d3400640 1 -- 172.21.15.71:0/1563417434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d4104010 msgr2=0x7f87d419a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d3400640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d4104010 0x7f87d419a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d3400640 1 -- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d41049c0 msgr2=0x7f87d419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d3400640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d41049c0 0x7f87d419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d3400640 1 -- 172.21.15.71:0/1563417434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d419f6e0 con 0x7f87d4109140 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.968+0000 7f87d3400640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4109140 0x7f87d419f000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009fd0 tx=0x7f87cc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.969+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc00ce20 con 0x7f87d4109140 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.969+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d419f9d0 con 0x7f87d4109140 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.969+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87cc005020 con 0x7f87d4109140 2024-09-05T12:32:57.974 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.969+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87cc010720 con 0x7f87d4109140 2024-09-05T12:32:57.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.969+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d41a72b0 con 0x7f87d4109140 2024-09-05T12:32:57.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.970+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d41043f0 con 0x7f87d4109140 2024-09-05T12:32:57.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.973+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f87cc005540 con 0x7f87d4109140 2024-09-05T12:32:57.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.974+0000 7f87cb400640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f87b80a2d70 0x7f87b80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:32:57.975 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.974+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f87cc0cfdb0 con 0x7f87d4109140 2024-09-05T12:32:57.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.974+0000 7f87d2a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f87b80a2d70 0x7f87b80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:32:57.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.975+0000 7f87d2a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f87b80a2d70 0x7f87b80a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000c00 tx=0x7f87c4005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:32:57.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:57.978+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87cc0d4070 con 0x7f87d4109140 2024-09-05T12:32:58.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:57 smithi174 ceph-mon[34313]: pgmap v159: 129 pgs: 129 active+clean; 74 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.1 MiB/s wr, 362 op/s 2024-09-05T12:32:58.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 199064 4.88 MB/sec execute 114 sec latency 13.261 ms 2024-09-05T12:32:58.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.249+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87d410c320 con 0x7f87d4109140 2024-09-05T12:32:58.251 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:32:58.251 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"btime":"2024-09-05T12:32:37:493040+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:32:36.945626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:32:58.251 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.249+0000 7f87cb400640 1 -- 172.21.15.71:0/1563417434 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11971 (secure 0 0 0) 0x7f87cc09c630 con 0x7f87d4109140 2024-09-05T12:32:58.251 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-09-05T12:32:58.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f87b80a2d70 msgr2=0x7f87b80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:58.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f87b80a2d70 0x7f87b80a5230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f87c4000c00 tx=0x7f87c4005b50 comp rx=0 tx=0).stop 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4109140 msgr2=0x7f87d419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4109140 0x7f87d419f000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009fd0 tx=0x7f87cc00ef10 comp rx=0 tx=0).stop 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 shutdown_connections 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87d4109140 0x7f87d419f000 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f87b80a2d70 0x7f87b80a5230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f87d41049c0 0x7f87d419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 --2- 172.21.15.71:0/1563417434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d4104010 0x7f87d419a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.252+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 >> 172.21.15.71:0/1563417434 conn(0x7f87d40ffa50 msgr2=0x7f87d4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.253+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 shutdown_connections 2024-09-05T12:32:58.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:32:58.253+0000 7f87d9a00640 1 -- 172.21.15.71:0/1563417434 wait complete. 2024-09-05T12:32:58.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:58 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1563417434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:32:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:58 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1563417434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:32:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:58 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1563417434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:32:59.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 201030 4.90 MB/sec execute 115 sec latency 25.670 ms 2024-09-05T12:32:59.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:32:59 smithi071 ceph-mon[29845]: pgmap v160: 129 pgs: 129 active+clean; 74 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-09-05T12:32:59.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:32:59 smithi123 ceph-mon[34321]: pgmap v160: 129 pgs: 129 active+clean; 74 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-09-05T12:33:00.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:32:59 smithi174 ceph-mon[34313]: pgmap v160: 129 pgs: 129 active+clean; 74 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 253 op/s 2024-09-05T12:33:00.209 INFO:tasks.workunit.client.0.smithi071.stdout: 1 201130 4.86 MB/sec execute 116 sec latency 728.882 ms 2024-09-05T12:33:00.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 -- 172.21.15.71:0/1504446844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c1078c0 msgr2=0x7fe59c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 --2- 172.21.15.71:0/1504446844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c1078c0 0x7fe59c107ca0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009930 tx=0x7fe58c02a420 comp rx=0 tx=0).stop 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 -- 172.21.15.71:0/1504446844 shutdown_connections 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 --2- 172.21.15.71:0/1504446844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c102250 0x7fe59c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 --2- 172.21.15.71:0/1504446844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1018b0 0x7fe59c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 --2- 172.21.15.71:0/1504446844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c1078c0 0x7fe59c107ca0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.441+0000 7fe5a1800640 1 -- 172.21.15.71:0/1504446844 >> 172.21.15.71:0/1504446844 conn(0x7fe59c0fd5d0 msgr2=0x7fe59c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.442+0000 7fe5a1800640 1 -- 172.21.15.71:0/1504446844 shutdown_connections 2024-09-05T12:33:00.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.442+0000 7fe5a1800640 1 -- 172.21.15.71:0/1504446844 wait complete. 2024-09-05T12:33:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 Processor -- start 2024-09-05T12:33:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 -- start start 2024-09-05T12:33:00.445 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1018b0 0x7fe59c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 0x7fe59c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1078c0 0x7fe59c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111d50 con 0x7fe59c1018b0 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111bd0 con 0x7fe59c1078c0 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe5a1800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c111ed0 con 0x7fe59c102250 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.443+0000 7fe59be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1018b0 0x7fe59c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe59b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 0x7fe59c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1078c0 0x7fe59c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe59b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 0x7fe59c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:43814/0 (socket says 172.21.15.71:43814) 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe59b400640 1 -- 172.21.15.71:0/4287845467 learned_addr learned my addr 172.21.15.71:0/4287845467 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 -- 172.21.15.71:0/4287845467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 msgr2=0x7fe59c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 0x7fe59c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 -- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1018b0 msgr2=0x7fe59c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1018b0 0x7fe59c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 -- 172.21.15.71:0/4287845467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c19f470 con 0x7fe59c1078c0 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe59b400640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 0x7fe59c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a0e00640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1078c0 0x7fe59c19ed40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe59400ec50 tx=0x7fe59400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe598c00640 1 -- 172.21.15.71:0/4287845467 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594003b70 con 0x7fe59c1078c0 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe598c00640 1 -- 172.21.15.71:0/4287845467 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe594003d10 con 0x7fe59c1078c0 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.444+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59c19f760 con 0x7fe59c1078c0 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.445+0000 7fe598c00640 1 -- 172.21.15.71:0/4287845467 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594010890 con 0x7fe59c1078c0 2024-09-05T12:33:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.445+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59c1a6ff0 con 0x7fe59c1078c0 2024-09-05T12:33:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.448+0000 7fe5a5c7b740 1 Processor -- start 2024-09-05T12:33:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe598c00640 1 -- 172.21.15.71:0/4287845467 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe594003ec0 con 0x7fe59c1078c0 2024-09-05T12:33:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 -- start start 2024-09-05T12:33:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68ef3b0 0x55ced68ef790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 0x55ced68fa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68fa570 0x55ced68fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ced6855100 con 0x55ced68fa570 2024-09-05T12:33:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ced67fd1b0 con 0x55ced68efcd0 2024-09-05T12:33:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.449+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ced68510c0 con 0x55ced68ef3b0 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 0x55ced68fa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 0x55ced68fa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51226/0 (socket says 172.21.15.71:51226) 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 -- 172.21.15.71:0/320221570 learned_addr learned my addr 172.21.15.71:0/320221570 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68ef3b0 0x55ced68ef790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59b400640 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68fa570 0x55ced68fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 -- 172.21.15.71:0/320221570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68ef3b0 msgr2=0x55ced68ef790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68ef3b0 0x55ced68ef790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 -- 172.21.15.71:0/320221570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68fa570 msgr2=0x55ced68fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68fa570 0x55ced68fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 -- 172.21.15.71:0/320221570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced6568210 con 0x55ced68efcd0 2024-09-05T12:33:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59b400640 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68fa570 0x55ced68fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.450+0000 7fe59be00640 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 0x55ced68fa030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009930 tx=0x7fe58c005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abd766ed65f2069c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe591600640 1 -- 172.21.15.71:0/320221570 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c035040 con 0x55ced68efcd0 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe598c00640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe5800a2d70 0x7fe5800a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe591600640 1 -- 172.21.15.71:0/320221570 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58c005520 con 0x55ced68efcd0 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe591600640 1 -- 172.21.15.71:0/320221570 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0035f0 con 0x55ced68efcd0 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe598c00640 1 -- 172.21.15.71:0/4287845467 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe5940cfc60 con 0x7fe59c1078c0 2024-09-05T12:33:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe59b400640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe5800a2d70 0x7fe5800a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/320221570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 msgr2=0x55ced68fa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.451+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 0x55ced68fa030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009930 tx=0x7fe58c005de0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe59b400640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe5800a2d70 0x7fe5800a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55ced68ecca0 tx=0x7fe588005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/320221570 shutdown_connections 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68fa570 0x55ced68fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68efcd0 0x55ced68fa030 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/320221570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68ef3b0 0x55ced68ef790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/320221570 >> 172.21.15.71:0/320221570 conn(0x55ced68eb920 msgr2=0x55ced68ebd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/320221570 shutdown_connections 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/320221570 wait complete. 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 -- start start 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.452+0000 7fe5a5c7b740 1 Processor -- start 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced68f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced68f8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ced68fcff0 con 0x55ced68ef3b0 2024-09-05T12:33:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ced67de090 con 0x55ced68fa570 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ced68fd170 con 0x55ced68efcd0 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a5c7b740 10 client.0 _ll_register_callbacks cb 0x55ced6832f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe59be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced68f8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe59b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced68f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a0e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced68f8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51240/0 (socket says 172.21.15.71:51240) 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.453+0000 7fe5a0e00640 1 -- 172.21.15.71:0/4284605434 learned_addr learned my addr 172.21.15.71:0/4284605434 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe59b400640 1 -- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 msgr2=0x55ced68f8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe59b400640 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced68f8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe59b400640 1 -- 172.21.15.71:0/4284605434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 msgr2=0x55ced68f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe59b400640 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe59b400640 1 -- 172.21.15.71:0/4284605434 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced6634620 con 0x55ced68efcd0 2024-09-05T12:33:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe59b400640 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced68f7b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe59c10ec10 tx=0x7fe5880097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca183c1bd1c6d23c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe586000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe586000640 1 -- 172.21.15.71:0/4284605434 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58801a050 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe58c005520 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.454+0000 7fe586000640 1 -- 172.21.15.71:0/4284605434 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe588005220 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe586000640 1 -- 172.21.15.71:0/4284605434 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe588005420 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 10 client.26011 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ced6908df0 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe586000640 1 -- 172.21.15.71:0/4284605434 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58c005520 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe586000640 1 -- 172.21.15.71:0/4284605434 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fe58801e720 con 0x55ced68efcd0 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 10 client.26011 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 10 client.26011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 20 client.26011 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced690a260 0x55ced690c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 4 client.26011 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.455+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ced6862830 con 0x55ced690a260 2024-09-05T12:33:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.456+0000 7fe59be00640 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced690a260 0x55ced690c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.456+0000 7fe59be00640 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced690a260 0x55ced690c6f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe065708ac629a4e server_cookie=96531664787c4c4a in_seq=0 out_seq=0 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.456+0000 7fe586000640 10 client.26011 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe586000640 1 -- 172.21.15.71:0/4284605434 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ced6862830 con 0x55ced690a260 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe586000640 10 client.26011 handle_command_reply: tid=0 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 client.26011 shutdown 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 msgr2=0x55ced68f7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced68f7b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe59c10ec10 tx=0x7fe5880097a0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 shutdown_connections 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced690a260 0x55ced690c6f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced68f8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced68f7b20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/4284605434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe586000640 0 client.26011 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 >> 172.21.15.71:0/4284605434 conn(0x55ced68eb920 msgr2=0x55ced68f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 shutdown_connections 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/4284605434 wait complete. 2024-09-05T12:33:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.457+0000 7fe5a5c7b740 20 client.26011 trim_cache size 0 max 16384 2024-09-05T12:33:00.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.472+0000 7fe5a5c7b740 1 Processor -- start 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 -- start start 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced6912000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced6912540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1b0f70 con 0x55ced68ef3b0 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1b2b10 con 0x55ced68fa570 2024-09-05T12:33:00.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1b1640 con 0x55ced68efcd0 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced6912000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced6912000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:43832/0 (socket says 172.21.15.71:43832) 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 -- 172.21.15.71:0/744049441 learned_addr learned my addr 172.21.15.71:0/744049441 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe5a0e00640 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced6912540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 -- 172.21.15.71:0/744049441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 msgr2=0x55ced6912540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced6912540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 -- 172.21.15.71:0/744049441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 msgr2=0x55ced68f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.473+0000 7fe59b400640 1 -- 172.21.15.71:0/744049441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ced68f6570 con 0x55ced68efcd0 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.474+0000 7fe59b400640 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced6912000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe58801eb90 tx=0x7fe58800f400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2925c61a394b3f05 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.474+0000 7fe590c00640 1 -- 172.21.15.71:0/744049441 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58803c020 con 0x55ced68efcd0 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.474+0000 7fe590c00640 1 -- 172.21.15.71:0/744049441 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5880104e0 con 0x55ced68efcd0 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.474+0000 7fe590c00640 1 -- 172.21.15.71:0/744049441 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588038ac0 con 0x55ced68efcd0 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/744049441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 msgr2=0x55ced6912000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced6912000 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe58801eb90 tx=0x7fe58800f400 comp rx=0 tx=0).stop 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/744049441 shutdown_connections 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68fa570 0x55ced6912540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68efcd0 0x55ced6912000 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/744049441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68ef3b0 0x55ced68f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/744049441 >> 172.21.15.71:0/744049441 conn(0x55ced68eb920 msgr2=0x55ced68f0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/744049441 shutdown_connections 2024-09-05T12:33:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/744049441 wait complete. 2024-09-05T12:33:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 -- start start 2024-09-05T12:33:00.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.475+0000 7fe5a5c7b740 1 Processor -- start 2024-09-05T12:33:00.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68ef3b0 0x55ced68f4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 0x55ced68f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68fa570 0x55ced68f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1b2440 con 0x55ced68efcd0 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1b1d70 con 0x55ced68ef3b0 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c1aef50 con 0x55ced68fa570 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a5c7b740 10 client.0 _ll_register_callbacks cb 0x7fe588010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 0x55ced68f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe5a0e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68fa570 0x55ced68f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 0x55ced68f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:35256/0 (socket says 172.21.15.71:35256) 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 -- 172.21.15.71:0/133589450 learned_addr learned my addr 172.21.15.71:0/133589450 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59be00640 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68ef3b0 0x55ced68f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 -- 172.21.15.71:0/133589450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68fa570 msgr2=0x55ced68f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68fa570 0x55ced68f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 -- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68ef3b0 msgr2=0x55ced68f4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:00.481 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68ef3b0 0x55ced68f4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.476+0000 7fe59b400640 1 -- 172.21.15.71:0/133589450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ced6974100 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe59b400640 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 0x55ced68f4c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe58801ef30 tx=0x7fe5880097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71da3a84950874d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe586000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe586000640 1 -- 172.21.15.71:0/133589450 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58803c030 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe586000640 1 -- 172.21.15.71:0/133589450 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fe5880055c0 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5880104e0 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe586000640 1 -- 172.21.15.71:0/133589450 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe588050050 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe5a5c7b740 10 client.15972 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ced69753b0 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe586000640 1 -- 172.21.15.71:0/133589450 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5880104e0 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.477+0000 7fe586000640 1 -- 172.21.15.71:0/133589450 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fe5880387d0 con 0x55ced68efcd0 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe5a5c7b740 10 client.15972 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe5a5c7b740 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:00.482 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe5a5c7b740 20 client.15972 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced6975a70 0x55ced6977e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe5a5c7b740 4 client.15972 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:33:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ced685cb80 con 0x55ced6975a70 2024-09-05T12:33:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe59be00640 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced6975a70 0x55ced6977e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe59be00640 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced6975a70 0x55ced6977e90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f6d2989320537e server_cookie=b569e4ba20aa130a in_seq=0 out_seq=0 2024-09-05T12:33:00.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.478+0000 7fe586000640 10 client.15972 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.538+0000 7fe586000640 1 -- 172.21.15.71:0/133589450 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ced685cb80 con 0x55ced6975a70 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.538+0000 7fe586000640 10 client.15972 handle_command_reply: tid=0 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.538+0000 7fe5a5c7b740 1 client.15972 shutdown 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 msgr2=0x55ced68f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 0x55ced68f4c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe58801ef30 tx=0x7fe5880097a0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 shutdown_connections 2024-09-05T12:33:00.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ced6975a70 0x55ced6977e90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ced68fa570 0x55ced68f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe586000640 0 client.15972 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ced68efcd0 0x55ced68f4c40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 --2- 172.21.15.71:0/133589450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ced68ef3b0 0x55ced68f4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 >> 172.21.15.71:0/133589450 conn(0x55ced68eb920 msgr2=0x55ced68f1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 shutdown_connections 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 1 -- 172.21.15.71:0/133589450 wait complete. 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.539+0000 7fe5a5c7b740 20 client.15972 trim_cache size 0 max 16384 2024-09-05T12:33:00.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:33:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:33:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:33:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:33:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe5800a2d70 msgr2=0x7fe5800a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe5800a2d70 0x7fe5800a5230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55ced68ecca0 tx=0x7fe588005f90 comp rx=0 tx=0).stop 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1078c0 msgr2=0x7fe59c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1078c0 0x7fe59c19ed40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe59400ec50 tx=0x7fe59400c6a0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 shutdown_connections 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe59c1078c0 0x7fe59c19ed40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c102250 0x7fe59c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe5800a2d70 0x7fe5800a5230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 --2- 172.21.15.71:0/4287845467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe59c1018b0 0x7fe59c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 >> 172.21.15.71:0/4287845467 conn(0x7fe59c0fd5d0 msgr2=0x7fe59c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 shutdown_connections 2024-09-05T12:33:00.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:00.540+0000 7fe5a1800640 1 -- 172.21.15.71:0/4287845467 wait complete. 2024-09-05T12:33:00.682 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:33:00.682 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:33:01.121 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:33:01.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202361 4.91 MB/sec execute 117 sec latency 950.581 ms 2024-09-05T12:33:01.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 -- 172.21.15.71:0/117896094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1019d0 msgr2=0x7fc40c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 --2- 172.21.15.71:0/117896094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1019d0 0x7fc40c101e30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009900 tx=0x7fc3f802a410 comp rx=0 tx=0).stop 2024-09-05T12:33:01.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 -- 172.21.15.71:0/117896094 shutdown_connections 2024-09-05T12:33:01.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 --2- 172.21.15.71:0/117896094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c102370 0x7fc40c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 --2- 172.21.15.71:0/117896094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1019d0 0x7fc40c101e30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 --2- 172.21.15.71:0/117896094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c1079d0 0x7fc40c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 -- 172.21.15.71:0/117896094 >> 172.21.15.71:0/117896094 conn(0x7fc40c0fd710 msgr2=0x7fc40c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 -- 172.21.15.71:0/117896094 shutdown_connections 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.761+0000 7fc412a00640 1 -- 172.21.15.71:0/117896094 wait complete. 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 Processor -- start 2024-09-05T12:33:01.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 -- start start 2024-09-05T12:33:01.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c1019d0 0x7fc40c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c102370 0x7fc40c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 0x7fc40c19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c111de0 con 0x7fc40c102370 2024-09-05T12:33:01.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c111c60 con 0x7fc40c1079d0 2024-09-05T12:33:01.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c111f60 con 0x7fc40c1019d0 2024-09-05T12:33:01.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 0x7fc40c19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc410c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c102370 0x7fc40c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.763+0000 7fc412000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 0x7fc40c19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51258/0 (socket says 172.21.15.71:51258) 2024-09-05T12:33:01.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 -- 172.21.15.71:0/3236602070 learned_addr learned my addr 172.21.15.71:0/3236602070 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:01.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc411600640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c1019d0 0x7fc40c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 -- 172.21.15.71:0/3236602070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c1019d0 msgr2=0x7fc40c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c1019d0 0x7fc40c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 -- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c102370 msgr2=0x7fc40c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c102370 0x7fc40c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 -- 172.21.15.71:0/3236602070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c19d230 con 0x7fc40c1079d0 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc412000640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 0x7fc40c19cb50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc40400eb10 tx=0x7fc40400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.764+0000 7fc40a000640 1 -- 172.21.15.71:0/3236602070 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404004b20 con 0x7fc40c1079d0 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.765+0000 7fc40a000640 1 -- 172.21.15.71:0/3236602070 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc404004cc0 con 0x7fc40c1079d0 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.765+0000 7fc40a000640 1 -- 172.21.15.71:0/3236602070 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404010760 con 0x7fc40c1079d0 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.765+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c19d520 con 0x7fc40c1079d0 2024-09-05T12:33:01.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.765+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c074420 con 0x7fc40c1079d0 2024-09-05T12:33:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.768+0000 7fc416a59740 1 Processor -- start 2024-09-05T12:33:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.768+0000 7fc40a000640 1 -- 172.21.15.71:0/3236602070 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc404010900 con 0x7fc40c1079d0 2024-09-05T12:33:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 -- start start 2024-09-05T12:33:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695e560 0x55797695e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976969200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.770 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x55797696bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579768c4530 con 0x557976969740 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55797686b860 con 0x55797695ee80 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc416a59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579768c04f0 con 0x55797695e560 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc411600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976969200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc410c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695e560 0x55797695e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc411600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976969200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51270/0 (socket says 172.21.15.71:51270) 2024-09-05T12:33:01.771 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.769+0000 7fc411600640 1 -- 172.21.15.71:0/1625368027 learned_addr learned my addr 172.21.15.71:0/1625368027 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc410c00640 1 -- 172.21.15.71:0/1625368027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 msgr2=0x557976969200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc410c00640 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976969200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc410c00640 1 -- 172.21.15.71:0/1625368027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 msgr2=0x55797696bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc410c00640 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x55797696bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc410c00640 1 -- 172.21.15.71:0/1625368027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557976635b90 con 0x55797695e560 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc411600640 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976969200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc410c00640 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695e560 0x55797695e940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3f802aa00 tx=0x7fc3f8005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8c4a9a60bd5d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.770+0000 7fc40a000640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc3f40a2d70 0x7fc3f40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc3fe600640 1 -- 172.21.15.71:0/1625368027 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f804a070 con 0x55797695e560 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc40a000640 1 -- 172.21.15.71:0/3236602070 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc4040cfd30 con 0x7fc40c1079d0 2024-09-05T12:33:01.772 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc411600640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc3f40a2d70 0x7fc3f40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc3fe600640 1 -- 172.21.15.71:0/1625368027 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8005510 con 0x55797695e560 2024-09-05T12:33:01.773 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc3fe600640 1 -- 172.21.15.71:0/1625368027 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8035040 con 0x55797695e560 2024-09-05T12:33:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc416a59740 1 -- 172.21.15.71:0/1625368027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695e560 msgr2=0x55797695e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc416a59740 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695e560 0x55797695e940 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc3f802aa00 tx=0x7fc3f8005e40 comp rx=0 tx=0).stop 2024-09-05T12:33:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.771+0000 7fc411600640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc3f40a2d70 0x7fc3f40a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc400005bc0 tx=0x7fc400005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:01.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 -- 172.21.15.71:0/1625368027 shutdown_connections 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x55797696bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976969200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 --2- 172.21.15.71:0/1625368027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695e560 0x55797695e940 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 -- 172.21.15.71:0/1625368027 >> 172.21.15.71:0/1625368027 conn(0x55797695aaf0 msgr2=0x55797695aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 -- 172.21.15.71:0/1625368027 shutdown_connections 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 -- 172.21.15.71:0/1625368027 wait complete. 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.772+0000 7fc416a59740 1 -- start start 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 Processor -- start 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976966740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695ee80 0x557976966c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x5579769671c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55797696c2b0 con 0x55797695ee80 2024-09-05T12:33:01.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55797687ee60 con 0x55797695e560 2024-09-05T12:33:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55797696c430 con 0x557976969740 2024-09-05T12:33:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.773+0000 7fc416a59740 10 client.0 _ll_register_callbacks cb 0x557976883cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976966740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc410c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695ee80 0x557976966c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976966740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51280/0 (socket says 172.21.15.71:51280) 2024-09-05T12:33:01.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 -- 172.21.15.71:0/2514246474 learned_addr learned my addr 172.21.15.71:0/2514246474 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:01.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc412000640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x5579769671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 -- 172.21.15.71:0/2514246474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 msgr2=0x5579769671c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x5579769671c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 -- 172.21.15.71:0/2514246474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695ee80 msgr2=0x557976966c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695ee80 0x557976966c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc411600640 1 -- 172.21.15.71:0/2514246474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55797688a990 con 0x55797695e560 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.774+0000 7fc412000640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x5579769671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc411600640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976966740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc400004620 tx=0x7fc400004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f662abbcb66b2a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc3f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2514246474 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400009c80 con 0x55797695e560 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f8005510 con 0x55797695e560 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2514246474 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc40000f690 con 0x55797695e560 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2514246474 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc400032050 con 0x55797695e560 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc416a59740 10 client.26015 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:01.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x557976978130 con 0x55797695e560 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.775+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2514246474 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f8005510 con 0x55797695e560 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2514246474 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc400027020 con 0x55797695e560 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc416a59740 10 client.26015 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc416a59740 10 client.26015 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc416a59740 20 client.26015 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc416a59740 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557976979540 0x55797697b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc416a59740 4 client.26015 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557976867310 con 0x557976979540 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.776+0000 7fc410c00640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557976979540 0x55797697b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.777+0000 7fc410c00640 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557976979540 0x55797697b950 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86c24eba62a3a95 server_cookie=734eb29d9a27f88c in_seq=0 out_seq=0 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.777+0000 7fc3f2a00640 10 client.26015 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.777+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2514246474 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557976867310 con 0x557976979540 2024-09-05T12:33:01.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.777+0000 7fc3f2a00640 10 client.26015 handle_command_reply: tid=0 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.777+0000 7fc416a59740 1 client.26015 shutdown 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 msgr2=0x557976966740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976966740 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc400004620 tx=0x7fc400004730 comp rx=0 tx=0).stop 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 shutdown_connections 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557976979540 0x55797697b950 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x5579769671c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695ee80 0x557976966c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 --2- 172.21.15.71:0/2514246474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976966740 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 >> 172.21.15.71:0/2514246474 conn(0x55797695aaf0 msgr2=0x5579769607d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:01.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc3f2a00640 0 client.26015 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:01.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 shutdown_connections 2024-09-05T12:33:01.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 1 -- 172.21.15.71:0/2514246474 wait complete. 2024-09-05T12:33:01.782 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.778+0000 7fc416a59740 20 client.26015 trim_cache size 0 max 16384 2024-09-05T12:33:01.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 Processor -- start 2024-09-05T12:33:01.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 -- start start 2024-09-05T12:33:01.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976965680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 0x557976981220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x557976981760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1acb10 con 0x557976969740 2024-09-05T12:33:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1ae6b0 con 0x55797695e560 2024-09-05T12:33:01.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.799+0000 7fc416a59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1ad1e0 con 0x55797695ee80 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc410c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 0x557976981220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976965680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976965680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51282/0 (socket says 172.21.15.71:51282) 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 -- 172.21.15.71:0/3191207356 learned_addr learned my addr 172.21.15.71:0/3191207356 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc410c00640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 0x557976981220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:43894/0 (socket says 172.21.15.71:43894) 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc412000640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x557976981760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 -- 172.21.15.71:0/3191207356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 msgr2=0x557976981220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 0x557976981220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 -- 172.21.15.71:0/3191207356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 msgr2=0x557976981760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x557976981760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 -- 172.21.15.71:0/3191207356 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557976955160 con 0x55797695e560 2024-09-05T12:33:01.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc412000640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x557976981760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:01.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc410c00640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 0x557976981220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:01.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.800+0000 7fc411600640 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976965680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc400019040 tx=0x7fc400012d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=512feca37df36ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:01.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.801+0000 7fc3fdc00640 1 -- 172.21.15.71:0/3191207356 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400029440 con 0x55797695e560 2024-09-05T12:33:01.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.801+0000 7fc3fdc00640 1 -- 172.21.15.71:0/3191207356 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4000295e0 con 0x55797695e560 2024-09-05T12:33:01.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.801+0000 7fc3fdc00640 1 -- 172.21.15.71:0/3191207356 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001f790 con 0x55797695e560 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.801+0000 7fc416a59740 1 -- 172.21.15.71:0/3191207356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 msgr2=0x557976965680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.801+0000 7fc416a59740 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976965680 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc400019040 tx=0x7fc400012d80 comp rx=0 tx=0).stop 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 -- 172.21.15.71:0/3191207356 shutdown_connections 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557976969740 0x557976981760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55797695ee80 0x557976981220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 --2- 172.21.15.71:0/3191207356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695e560 0x557976965680 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 -- 172.21.15.71:0/3191207356 >> 172.21.15.71:0/3191207356 conn(0x55797695aaf0 msgr2=0x557976966f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 -- 172.21.15.71:0/3191207356 shutdown_connections 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 -- 172.21.15.71:0/3191207356 wait complete. 2024-09-05T12:33:01.806 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 -- start start 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.802+0000 7fc416a59740 1 Processor -- start 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695e560 0x557976971680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976971bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x55797696e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1adfe0 con 0x55797695e560 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1ad910 con 0x55797695ee80 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c1aaaf0 con 0x557976969740 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc416a59740 10 client.0 _ll_register_callbacks cb 0x7fc400012f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc412000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x55797696e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc411600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695e560 0x557976971680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc410c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976971bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc412000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x55797696e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:43902/0 (socket says 172.21.15.71:43902) 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc412000640 1 -- 172.21.15.71:0/2383452508 learned_addr learned my addr 172.21.15.71:0/2383452508 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:01.807 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc410c00640 1 -- 172.21.15.71:0/2383452508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 msgr2=0x55797696e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc410c00640 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x55797696e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc410c00640 1 -- 172.21.15.71:0/2383452508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695e560 msgr2=0x557976971680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc410c00640 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695e560 0x557976971680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc410c00640 1 -- 172.21.15.71:0/2383452508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557976972230 con 0x55797695ee80 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.803+0000 7fc412000640 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x55797696e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc410c00640 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976971bc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f8004b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc2cc501edc45a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc3f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2383452508 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f804a040 con 0x55797695ee80 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4000295e0 con 0x55797695ee80 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2383452508 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc3f8004180 con 0x55797695ee80 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2383452508 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc3f8062050 con 0x55797695ee80 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc416a59740 10 client.26021 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:01.808 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.804+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5579769e47b0 con 0x55797695ee80 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2383452508 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4000295e0 con 0x55797695ee80 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2383452508 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc3f803a6e0 con 0x55797695ee80 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc416a59740 10 client.26021 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc416a59740 10 client.26021 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc416a59740 20 client.26021 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc416a59740 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5579769e4b70 0x5579769e6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc416a59740 4 client.26021 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5579768d1c60 con 0x5579769e4b70 2024-09-05T12:33:01.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.805+0000 7fc411600640 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5579769e4b70 0x5579769e6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:01.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.806+0000 7fc411600640 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5579769e4b70 0x5579769e6f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd0796c3065a2c74 server_cookie=253965d021b1879b in_seq=0 out_seq=0 2024-09-05T12:33:01.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.806+0000 7fc3f2a00640 10 client.26021 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:01.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc3f2a00640 1 -- 172.21.15.71:0/2383452508 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5579768d1c60 con 0x5579769e4b70 2024-09-05T12:33:01.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc3f2a00640 10 client.26021 handle_command_reply: tid=0 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 client.26021 shutdown 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 msgr2=0x557976971bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976971bc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8000c00 tx=0x7fc3f8004b80 comp rx=0 tx=0).stop 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 shutdown_connections 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5579769e4b70 0x5579769e6f90 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557976969740 0x55797696e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55797695ee80 0x557976971bc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc3f2a00640 0 client.26021 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 --2- 172.21.15.71:0/2383452508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55797695e560 0x557976971680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 >> 172.21.15.71:0/2383452508 conn(0x55797695aaf0 msgr2=0x557976959450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 shutdown_connections 2024-09-05T12:33:01.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.874+0000 7fc416a59740 1 -- 172.21.15.71:0/2383452508 wait complete. 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc416a59740 20 client.26021 trim_cache size 0 max 16384 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 36, 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 18, 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:33:01.877 INFO:teuthology.orchestra.run.smithi071.stdout: "84daed41": { 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 18, 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:33:01.878 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:01.879 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc3f40a2d70 msgr2=0x7fc3f40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc3f40a2d70 0x7fc3f40a5230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc400005bc0 tx=0x7fc400005f90 comp rx=0 tx=0).stop 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 msgr2=0x7fc40c19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 0x7fc40c19cb50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc40400eb10 tx=0x7fc40400c2d0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 shutdown_connections 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc40c1079d0 0x7fc40c19cb50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc3f40a2d70 0x7fc3f40a5230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc40c102370 0x7fc40c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 --2- 172.21.15.71:0/3236602070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc40c1019d0 0x7fc40c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.875+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 >> 172.21.15.71:0/3236602070 conn(0x7fc40c0fd710 msgr2=0x7fc40c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.876+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 shutdown_connections 2024-09-05T12:33:01.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:01.876+0000 7fc412a00640 1 -- 172.21.15.71:0/3236602070 wait complete. 2024-09-05T12:33:01.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:01 smithi071 ceph-mon[29845]: pgmap v161: 129 pgs: 129 active+clean; 70 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-09-05T12:33:01.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:01 smithi123 ceph-mon[34321]: pgmap v161: 129 pgs: 129 active+clean; 70 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-09-05T12:33:02.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:01 smithi174 ceph-mon[34313]: pgmap v161: 129 pgs: 129 active+clean; 70 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.0 MiB/s wr, 374 op/s 2024-09-05T12:33:02.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.87 MB/sec execute 118 sec latency 402.372 ms 2024-09-05T12:33:03.107 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '84daed41', quiesce took 6.2 seconds. Will release after: 23.8 2024-09-05T12:33:03.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.83 MB/sec execute 119 sec latency 1402.438 ms 2024-09-05T12:33:03.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:03 smithi123 ceph-mon[34321]: pgmap v162: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-09-05T12:33:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:03 smithi174 ceph-mon[34313]: pgmap v162: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-09-05T12:33:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:03 smithi071 ceph-mon[29845]: pgmap v162: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-09-05T12:33:04.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.79 MB/sec execute 120 sec latency 2402.533 ms 2024-09-05T12:33:05.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.75 MB/sec execute 121 sec latency 3402.599 ms 2024-09-05T12:33:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:05 smithi071 ceph-mon[29845]: pgmap v163: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-05T12:33:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:05 smithi123 ceph-mon[34321]: pgmap v163: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-05T12:33:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:05 smithi174 ceph-mon[34313]: pgmap v163: 129 pgs: 129 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-09-05T12:33:06.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.71 MB/sec execute 122 sec latency 4402.676 ms 2024-09-05T12:33:07.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.67 MB/sec execute 123 sec latency 5402.801 ms 2024-09-05T12:33:07.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:07 smithi123 ceph-mon[34321]: pgmap v164: 129 pgs: 129 active+clean; 72 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-09-05T12:33:08.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:07 smithi174 ceph-mon[34313]: pgmap v164: 129 pgs: 129 active+clean; 72 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-09-05T12:33:08.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:07 smithi071 ceph-mon[29845]: pgmap v164: 129 pgs: 129 active+clean; 72 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.6 MiB/s wr, 321 op/s 2024-09-05T12:33:08.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.64 MB/sec execute 124 sec latency 6402.929 ms 2024-09-05T12:33:09.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.60 MB/sec execute 125 sec latency 7403.083 ms 2024-09-05T12:33:09.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:09 smithi071 ceph-mon[29845]: pgmap v165: 129 pgs: 129 active+clean; 72 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-05T12:33:09.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:09 smithi123 ceph-mon[34321]: pgmap v165: 129 pgs: 129 active+clean; 72 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-05T12:33:10.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:09 smithi174 ceph-mon[34313]: pgmap v165: 129 pgs: 129 active+clean; 72 MiB data, 1024 MiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.6 MiB/s wr, 207 op/s 2024-09-05T12:33:10.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.56 MB/sec execute 126 sec latency 8403.202 ms 2024-09-05T12:33:11.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.53 MB/sec execute 127 sec latency 9403.244 ms 2024-09-05T12:33:11.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:11 smithi123 ceph-mon[34321]: pgmap v166: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-09-05T12:33:12.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:11 smithi174 ceph-mon[34313]: pgmap v166: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-09-05T12:33:12.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:11 smithi071 ceph-mon[29845]: pgmap v166: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-09-05T12:33:12.210 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.49 MB/sec execute 128 sec latency 10403.342 ms 2024-09-05T12:33:13.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.46 MB/sec execute 129 sec latency 11403.457 ms 2024-09-05T12:33:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 -- 172.21.15.71:0/441577953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150104900 msgr2=0x7f8150108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 --2- 172.21.15.71:0/441577953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150104900 0x7f8150108940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f813c009900 tx=0x7f813c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 -- 172.21.15.71:0/441577953 shutdown_connections 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 --2- 172.21.15.71:0/441577953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81501090a0 0x7f8150110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 --2- 172.21.15.71:0/441577953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8150104900 0x7f8150108940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 --2- 172.21.15.71:0/441577953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f8150104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 -- 172.21.15.71:0/441577953 >> 172.21.15.71:0/441577953 conn(0x7f81500ff9e0 msgr2=0x7f8150101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 -- 172.21.15.71:0/441577953 shutdown_connections 2024-09-05T12:33:13.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.665+0000 7f8158200640 1 -- 172.21.15.71:0/441577953 wait complete. 2024-09-05T12:33:13.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.666+0000 7f8158200640 1 Processor -- start 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.666+0000 7f8158200640 1 -- start start 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.666+0000 7f8158200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f815019a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8158200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81501090a0 0x7f815019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8158200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815019ee30 0x7f815019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8158200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f81501140f0 con 0x7f8150103f50 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8158200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8150113f70 con 0x7f81501090a0 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8158200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8150114270 con 0x7f815019ee30 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f815019a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f815019a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42864/0 (socket says 172.21.15.71:42864) 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 -- 172.21.15.71:0/4068912468 learned_addr learned my addr 172.21.15.71:0/4068912468 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8154e00640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81501090a0 0x7f815019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8156200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815019ee30 0x7f815019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 -- 172.21.15.71:0/4068912468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815019ee30 msgr2=0x7f815019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815019ee30 0x7f815019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 -- 172.21.15.71:0/4068912468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81501090a0 msgr2=0x7f815019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81501090a0 0x7f815019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8155800640 1 -- 172.21.15.71:0/4068912468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815019f920 con 0x7f8150103f50 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.667+0000 7f8156200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815019ee30 0x7f815019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.668+0000 7f8155800640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f815019a570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f814400ea60 tx=0x7f814400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.668+0000 7f814e000640 1 -- 172.21.15.71:0/4068912468 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814400cde0 con 0x7f8150103f50 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.668+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815019fc10 con 0x7f8150103f50 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.668+0000 7f814e000640 1 -- 172.21.15.71:0/4068912468 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81440035f0 con 0x7f8150103f50 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.668+0000 7f814e000640 1 -- 172.21.15.71:0/4068912468 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144005220 con 0x7f8150103f50 2024-09-05T12:33:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.668+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501a73d0 con 0x7f8150103f50 2024-09-05T12:33:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 Processor -- start 2024-09-05T12:33:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f814e000640 1 -- 172.21.15.71:0/4068912468 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f81440037a0 con 0x7f8150103f50 2024-09-05T12:33:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 -- start start 2024-09-05T12:33:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 0x563687b1c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1cb00 0x563687b26e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b29790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563687a82100 con 0x563687b1c1e0 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563687a2a1b0 con 0x563687b273a0 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.671+0000 7f815b078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563687a7e0c0 con 0x563687b1cb00 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 0x563687b1c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 0x563687b1c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42874/0 (socket says 172.21.15.71:42874) 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 -- 172.21.15.71:0/7326849 learned_addr learned my addr 172.21.15.71:0/7326849 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8155800640 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1cb00 0x563687b26e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8154e00640 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b29790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 -- 172.21.15.71:0/7326849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1cb00 msgr2=0x563687b26e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1cb00 0x563687b26e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 -- 172.21.15.71:0/7326849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 msgr2=0x563687b29790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b29790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.672+0000 7f8156200640 1 -- 172.21.15.71:0/7326849 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563687795210 con 0x563687b1c1e0 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f8156200640 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 0x563687b1c5c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8148009fd0 tx=0x7f8148010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94bb76aaec13f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f814e000640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f81380a2da0 0x7f81380a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f8142a00640 1 -- 172.21.15.71:0/7326849 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148011700 con 0x563687b1c1e0 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f8142a00640 1 -- 172.21.15.71:0/7326849 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8148011d80 con 0x563687b1c1e0 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f8142a00640 1 -- 172.21.15.71:0/7326849 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814800e3c0 con 0x563687b1c1e0 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f814e000640 1 -- 172.21.15.71:0/4068912468 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f81440cfa90 con 0x7f8150103f50 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.673+0000 7f8155800640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f81380a2da0 0x7f81380a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 -- 172.21.15.71:0/7326849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 msgr2=0x563687b1c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 0x563687b1c5c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8148009fd0 tx=0x7f8148010860 comp rx=0 tx=0).stop 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 -- 172.21.15.71:0/7326849 shutdown_connections 2024-09-05T12:33:13.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b29790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1cb00 0x563687b26e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 --2- 172.21.15.71:0/7326849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1c1e0 0x563687b1c5c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 -- 172.21.15.71:0/7326849 >> 172.21.15.71:0/7326849 conn(0x563687b18750 msgr2=0x563687b18b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f8155800640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f81380a2da0 0x7f81380a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81440985e0 tx=0x7f81440d3540 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 -- 172.21.15.71:0/7326849 shutdown_connections 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 -- 172.21.15.71:0/7326849 wait complete. 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 -- start start 2024-09-05T12:33:13.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.674+0000 7f815b078740 1 Processor -- start 2024-09-05T12:33:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b24d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563687b29fd0 con 0x563687b1cb00 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563687a0b090 con 0x563687b1c1e0 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563687b2a150 con 0x563687b273a0 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f815b078740 10 client.0 _ll_register_callbacks cb 0x563687a5ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f8154e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f8156200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f8156200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b242c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40948/0 (socket says 172.21.15.71:40948) 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f8156200640 1 -- 172.21.15.71:0/2741155869 learned_addr learned my addr 172.21.15.71:0/2741155869 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.675+0000 7f8156200640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b24d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8154e00640 1 -- 172.21.15.71:0/2741155869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 msgr2=0x563687b24d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8154e00640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b24d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8154e00640 1 -- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 msgr2=0x563687b242c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8154e00640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b242c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8154e00640 1 -- 172.21.15.71:0/2741155869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563687861620 con 0x563687b1cb00 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8156200640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8154e00640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b24800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f813c02a910 tx=0x7f813c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdc03484b487562 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8137400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8137400640 1 -- 172.21.15.71:0/2741155869 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c035040 con 0x563687b1cb00 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.676+0000 7f8137400640 1 -- 172.21.15.71:0/2741155869 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f813c005450 con 0x563687b1cb00 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.677+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8148011d80 con 0x563687b1cb00 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f8137400640 1 -- 172.21.15.71:0/2741155869 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f813c04f050 con 0x563687b1cb00 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f815b078740 10 client.15999 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:13.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563687b35b90 con 0x563687b1cb00 2024-09-05T12:33:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f8137400640 1 -- 172.21.15.71:0/2741155869 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8148011d80 con 0x563687b1cb00 2024-09-05T12:33:13.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f8137400640 1 -- 172.21.15.71:0/2741155869 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f813c003850 con 0x563687b1cb00 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f815b078740 10 client.15999 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.680+0000 7f815b078740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f815b078740 20 client.15999 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f815b078740 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687b37050 0x563687b39440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f815b078740 4 client.15999 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563687a8f830 con 0x563687b37050 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f8156200640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687b37050 0x563687b39440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f8156200640 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687b37050 0x563687b39440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25624eea1ccd3749 server_cookie=7c634dcdc586af62 in_seq=0 out_seq=0 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.681+0000 7f8137400640 10 client.15999 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.682+0000 7f8137400640 1 -- 172.21.15.71:0/2741155869 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563687a8f830 con 0x563687b37050 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.682+0000 7f8137400640 10 client.15999 handle_command_reply: tid=0 2024-09-05T12:33:13.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.682+0000 7f815b078740 1 client.15999 shutdown 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 msgr2=0x563687b24800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b24800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f813c02a910 tx=0x7f813c0043b0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 shutdown_connections 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687b37050 0x563687b39440 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b24d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b24800 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 --2- 172.21.15.71:0/2741155869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b242c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f8137400640 0 client.15999 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 >> 172.21.15.71:0/2741155869 conn(0x563687b18750 msgr2=0x563687b1e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 shutdown_connections 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 1 -- 172.21.15.71:0/2741155869 wait complete. 2024-09-05T12:33:13.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.683+0000 7f815b078740 20 client.15999 trim_cache size 0 max 16384 2024-09-05T12:33:13.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.698+0000 7f815b078740 1 Processor -- start 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.698+0000 7f815b078740 1 -- start start 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.698+0000 7f815b078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b1e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.698+0000 7f815b078740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b3ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f815b078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b3f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f815b078740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b10d0 con 0x563687b1cb00 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f815b078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b2c70 con 0x563687b1c1e0 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f815b078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b17a0 con 0x563687b273a0 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b3ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b3ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42880/0 (socket says 172.21.15.71:42880) 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 -- 172.21.15.71:0/959239300 learned_addr learned my addr 172.21.15.71:0/959239300 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:13.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8156200640 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b1e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8156200640 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b3f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 -- 172.21.15.71:0/959239300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 msgr2=0x563687b3f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b3f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 -- 172.21.15.71:0/959239300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 msgr2=0x563687b1e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b1e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 -- 172.21.15.71:0/959239300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563687b1c930 con 0x563687b1cb00 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.699+0000 7f8154e00640 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b3ece0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f813c003dc0 tx=0x7f813c0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6c4612c9290edc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f8142000640 1 -- 172.21.15.71:0/959239300 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c035030 con 0x563687b1cb00 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f8142000640 1 -- 172.21.15.71:0/959239300 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c041070 con 0x563687b1cb00 2024-09-05T12:33:13.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f8142000640 1 -- 172.21.15.71:0/959239300 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0317c0 con 0x563687b1cb00 2024-09-05T12:33:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f815b078740 1 -- 172.21.15.71:0/959239300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 msgr2=0x563687b3ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f815b078740 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b3ece0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f813c003dc0 tx=0x7f813c0045b0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f815b078740 1 -- 172.21.15.71:0/959239300 shutdown_connections 2024-09-05T12:33:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f815b078740 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b273a0 0x563687b3f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f815b078740 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b3ece0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.700+0000 7f815b078740 1 --2- 172.21.15.71:0/959239300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b1c1e0 0x563687b1e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.701+0000 7f815b078740 1 -- 172.21.15.71:0/959239300 >> 172.21.15.71:0/959239300 conn(0x563687b18750 msgr2=0x563687b1dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.701+0000 7f815b078740 1 -- 172.21.15.71:0/959239300 shutdown_connections 2024-09-05T12:33:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.701+0000 7f815b078740 1 -- 172.21.15.71:0/959239300 wait complete. 2024-09-05T12:33:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.701+0000 7f815b078740 1 -- start start 2024-09-05T12:33:13.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.701+0000 7f815b078740 1 Processor -- start 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1c1e0 0x563687b2a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b9fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b25a0 con 0x563687b1cb00 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81501b1ed0 con 0x563687b273a0 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81501af0b0 con 0x563687b1c1e0 2024-09-05T12:33:13.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f815b078740 10 client.0 _ll_register_callbacks cb 0x7f813c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b2ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42886/0 (socket says 172.21.15.71:42886) 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 -- 172.21.15.71:0/1745149465 learned_addr learned my addr 172.21.15.71:0/1745149465 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8156200640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1c1e0 0x563687b2a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8156200640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b9fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 -- 172.21.15.71:0/1745149465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1c1e0 msgr2=0x563687b2a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1c1e0 0x563687b2a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 -- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 msgr2=0x563687b9fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b9fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 -- 172.21.15.71:0/1745149465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563687ba02e0 con 0x563687b1cb00 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8154e00640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b2ae80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f813c02a9b0 tx=0x7f813c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa944a4650eb93d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:13.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.702+0000 7f8137400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f8137400640 1 -- 172.21.15.71:0/1745149465 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0038d0 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f8137400640 1 -- 172.21.15.71:0/1745149465 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f813c005250 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f813c041070 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f8137400640 1 -- 172.21.15.71:0/1745149465 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f813c005450 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f815b078740 10 client.16005 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563687ba15f0 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.703+0000 7f8137400640 1 -- 172.21.15.71:0/1745149465 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c041070 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f8137400640 1 -- 172.21.15.71:0/1745149465 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f813c0047e0 con 0x563687b1cb00 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f815b078740 10 client.16005 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f815b078740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f815b078740 20 client.16005 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f815b078740 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687ba1cd0 0x563687ba40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:13.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f815b078740 4 client.16005 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:33:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563687a89b80 con 0x563687ba1cd0 2024-09-05T12:33:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f8156200640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687ba1cd0 0x563687ba40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f8156200640 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687ba1cd0 0x563687ba40f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3972966a3e4201f server_cookie=303b210f3302f01b in_seq=0 out_seq=0 2024-09-05T12:33:13.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.704+0000 7f8137400640 10 client.16005 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f8137400640 1 -- 172.21.15.71:0/1745149465 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563687a89b80 con 0x563687ba1cd0 2024-09-05T12:33:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f8137400640 10 client.16005 handle_command_reply: tid=0 2024-09-05T12:33:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 client.16005 shutdown 2024-09-05T12:33:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 msgr2=0x563687b2ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b2ae80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f813c02a9b0 tx=0x7f813c0315b0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 shutdown_connections 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563687ba1cd0 0x563687ba40f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563687b273a0 0x563687b9fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563687b1cb00 0x563687b2ae80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f8137400640 0 client.16005 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 --2- 172.21.15.71:0/1745149465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563687b1c1e0 0x563687b2a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 >> 172.21.15.71:0/1745149465 conn(0x563687b18750 msgr2=0x563687b1e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 shutdown_connections 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 1 -- 172.21.15.71:0/1745149465 wait complete. 2024-09-05T12:33:13.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.729+0000 7f815b078740 20 client.16005 trim_cache size 0 max 16384 2024-09-05T12:33:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:33:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:33:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:33:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:33:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f81380a2da0 msgr2=0x7f81380a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f81380a2da0 0x7f81380a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f81440985e0 tx=0x7f81440d3540 comp rx=0 tx=0).stop 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 msgr2=0x7f815019a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f815019a570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f814400ea60 tx=0x7f814400ef20 comp rx=0 tx=0).stop 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 shutdown_connections 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815019ee30 0x7f815019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81501090a0 0x7f815019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f81380a2da0 0x7f81380a5260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 --2- 172.21.15.71:0/4068912468 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8150103f50 0x7f815019a570 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 >> 172.21.15.71:0/4068912468 conn(0x7f81500ff9e0 msgr2=0x7f8150100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 shutdown_connections 2024-09-05T12:33:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:13.730+0000 7f8158200640 1 -- 172.21.15.71:0/4068912468 wait complete. 2024-09-05T12:33:14.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:13 smithi174 ceph-mon[34313]: pgmap v167: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-05T12:33:14.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:13 smithi071 ceph-mon[29845]: pgmap v167: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-05T12:33:14.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:13 smithi123 ceph-mon[34321]: pgmap v167: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-09-05T12:33:14.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.42 MB/sec execute 130 sec latency 12403.550 ms 2024-09-05T12:33:15.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.39 MB/sec execute 131 sec latency 13403.615 ms 2024-09-05T12:33:15.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:15 smithi071 ceph-mon[29845]: pgmap v168: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-05T12:33:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:15 smithi174 ceph-mon[34313]: pgmap v168: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-05T12:33:16.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:15 smithi123 ceph-mon[34321]: pgmap v168: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 1.1 MiB/s wr, 100 op/s 2024-09-05T12:33:16.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.36 MB/sec execute 132 sec latency 14403.676 ms 2024-09-05T12:33:17.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.32 MB/sec execute 133 sec latency 15403.716 ms 2024-09-05T12:33:18.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:17 smithi174 ceph-mon[34313]: pgmap v169: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-09-05T12:33:18.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:17 smithi071 ceph-mon[29845]: pgmap v169: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-09-05T12:33:18.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:17 smithi123 ceph-mon[34321]: pgmap v169: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.7 MiB/s wr, 135 op/s 2024-09-05T12:33:18.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.29 MB/sec execute 134 sec latency 16403.822 ms 2024-09-05T12:33:19.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.26 MB/sec execute 135 sec latency 17403.939 ms 2024-09-05T12:33:19.963 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:19 smithi071 ceph-mon[29845]: pgmap v170: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.1 MiB/s wr, 61 op/s 2024-09-05T12:33:19.981 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:19 smithi123 ceph-mon[34321]: pgmap v170: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.1 MiB/s wr, 61 op/s 2024-09-05T12:33:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:19 smithi174 ceph-mon[34313]: pgmap v170: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 1.1 MiB/s wr, 61 op/s 2024-09-05T12:33:20.211 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.23 MB/sec execute 136 sec latency 18404.012 ms 2024-09-05T12:33:21.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.20 MB/sec execute 137 sec latency 19404.081 ms 2024-09-05T12:33:21.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:21 smithi071 ceph-mon[29845]: pgmap v171: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-05T12:33:22.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:21 smithi174 ceph-mon[34313]: pgmap v171: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-05T12:33:22.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:21 smithi123 ceph-mon[34321]: pgmap v171: 129 pgs: 129 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-05T12:33:22.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.17 MB/sec execute 138 sec latency 20404.503 ms 2024-09-05T12:33:23.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.14 MB/sec execute 139 sec latency 21404.619 ms 2024-09-05T12:33:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:23 smithi174 ceph-mon[34313]: pgmap v172: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 65 op/s 2024-09-05T12:33:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:23 smithi071 ceph-mon[29845]: pgmap v172: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 65 op/s 2024-09-05T12:33:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:23 smithi123 ceph-mon[34321]: pgmap v172: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 1.2 MiB/s wr, 65 op/s 2024-09-05T12:33:24.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.11 MB/sec execute 140 sec latency 22404.737 ms 2024-09-05T12:33:25.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.08 MB/sec execute 141 sec latency 23404.826 ms 2024-09-05T12:33:25.815 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:25 smithi071 ceph-mon[29845]: pgmap v173: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2024-09-05T12:33:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:25 smithi174 ceph-mon[34313]: pgmap v173: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2024-09-05T12:33:26.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:25 smithi123 ceph-mon[34321]: pgmap v173: 129 pgs: 129 active+clean; 73 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 1.2 MiB/s wr, 62 op/s 2024-09-05T12:33:26.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.05 MB/sec execute 142 sec latency 24404.927 ms 2024-09-05T12:33:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 -- 172.21.15.71:0/29071848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64010bf00 msgr2=0x7fc64010e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 --2- 172.21.15.71:0/29071848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64010bf00 0x7fc64010e370 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc638009ca0 tx=0x7fc63802a950 comp rx=0 tx=0).stop 2024-09-05T12:33:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 -- 172.21.15.71:0/29071848 shutdown_connections 2024-09-05T12:33:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 --2- 172.21.15.71:0/29071848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64010bf00 0x7fc64010e370 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 --2- 172.21.15.71:0/29071848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 0x7fc64010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 --2- 172.21.15.71:0/29071848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc640102ae0 0x7fc640102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 -- 172.21.15.71:0/29071848 >> 172.21.15.71:0/29071848 conn(0x7fc6400777c0 msgr2=0x7fc6400feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 -- 172.21.15.71:0/29071848 shutdown_connections 2024-09-05T12:33:26.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.845+0000 7fc63f200640 1 -- 172.21.15.71:0/29071848 wait complete. 2024-09-05T12:33:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.846+0000 7fc63f200640 1 Processor -- start 2024-09-05T12:33:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.846+0000 7fc63f200640 1 -- start start 2024-09-05T12:33:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.846+0000 7fc63f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 0x7fc64019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.846+0000 7fc63f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 0x7fc64019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.846+0000 7fc63f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64010bf00 0x7fc64019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc640111d10 con 0x7fc640102ae0 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc640111b90 con 0x7fc64010bf00 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc640111e90 con 0x7fc640103400 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 0x7fc64019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 0x7fc64019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46888/0 (socket says 172.21.15.71:46888) 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 -- 172.21.15.71:0/3357879289 learned_addr learned my addr 172.21.15.71:0/3357879289 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63e800640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64010bf00 0x7fc64019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63d400640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 0x7fc64019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 -- 172.21.15.71:0/3357879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 msgr2=0x7fc64019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 0x7fc64019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 -- 172.21.15.71:0/3357879289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64010bf00 msgr2=0x7fc64019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64010bf00 0x7fc64019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 -- 172.21.15.71:0/3357879289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64019f3f0 con 0x7fc640102ae0 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63d400640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 0x7fc64019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.847+0000 7fc63de00640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 0x7fc64019a400 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc63000e9f0 tx=0x7fc63000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.848+0000 7fc636a00640 1 -- 172.21.15.71:0/3357879289 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc63000cd80 con 0x7fc640102ae0 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.848+0000 7fc636a00640 1 -- 172.21.15.71:0/3357879289 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc630010910 con 0x7fc640102ae0 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.848+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64019f6e0 con 0x7fc640102ae0 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.848+0000 7fc636a00640 1 -- 172.21.15.71:0/3357879289 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc630018990 con 0x7fc640102ae0 2024-09-05T12:33:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.848+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6401a6f70 con 0x7fc640102ae0 2024-09-05T12:33:26.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.852+0000 7fc647dbc740 1 Processor -- start 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.852+0000 7fc647dbc740 1 -- start start 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.852+0000 7fc636a00640 1 -- 172.21.15.71:0/3357879289 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc630010ac0 con 0x7fc640102ae0 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.852+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe43e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.852+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe449030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe449570 0x55defe44b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc647dbc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55defe3a4100 con 0x55defe43e3b0 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc647dbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55defe34c1b0 con 0x55defe449570 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc647dbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55defe3a00c0 con 0x55defe43ecd0 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe43e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe43e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46892/0 (socket says 172.21.15.71:46892) 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 -- 172.21.15.71:0/3885912718 learned_addr learned my addr 172.21.15.71:0/3885912718 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63de00640 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe449030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63d400640 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe449570 0x55defe44b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 -- 172.21.15.71:0/3885912718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 msgr2=0x55defe449030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe449030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 -- 172.21.15.71:0/3885912718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe449570 msgr2=0x55defe44b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe449570 0x55defe44b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.853+0000 7fc63e800640 1 -- 172.21.15.71:0/3885912718 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55defe0b7210 con 0x55defe43e3b0 2024-09-05T12:33:26.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc63e800640 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe43e790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc638009ca0 tx=0x7fc638005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2124e58a76209d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc636a00640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6240a2d70 0x7fc6240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc62b400640 1 -- 172.21.15.71:0/3885912718 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638048070 con 0x55defe43e3b0 2024-09-05T12:33:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc62b400640 1 -- 172.21.15.71:0/3885912718 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638003e50 con 0x55defe43e3b0 2024-09-05T12:33:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc62b400640 1 -- 172.21.15.71:0/3885912718 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638034040 con 0x55defe43e3b0 2024-09-05T12:33:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc636a00640 1 -- 172.21.15.71:0/3357879289 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc6300cfab0 con 0x7fc640102ae0 2024-09-05T12:33:26.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.854+0000 7fc63de00640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6240a2d70 0x7fc6240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 -- 172.21.15.71:0/3885912718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 msgr2=0x55defe43e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe43e790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc638009ca0 tx=0x7fc638005f80 comp rx=0 tx=0).stop 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc63de00640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6240a2d70 0x7fc6240a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6300ccf70 tx=0x7fc6300d35f0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 -- 172.21.15.71:0/3885912718 shutdown_connections 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe449570 0x55defe44b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe449030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3885912718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe43e790 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 -- 172.21.15.71:0/3885912718 >> 172.21.15.71:0/3885912718 conn(0x55defe43a920 msgr2=0x55defe43ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 -- 172.21.15.71:0/3885912718 shutdown_connections 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 -- 172.21.15.71:0/3885912718 wait complete. 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 -- start start 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.855+0000 7fc647dbc740 1 Processor -- start 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe446560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 0x55defe446aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe446fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55defe44c080 con 0x55defe43ecd0 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55defe32d090 con 0x55defe43e3b0 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55defe44c200 con 0x55defe449570 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc647dbc740 10 client.0 _ll_register_callbacks cb 0x55defe381f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 0x55defe446aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 0x55defe446aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46894/0 (socket says 172.21.15.71:46894) 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 -- 172.21.15.71:0/2708674213 learned_addr learned my addr 172.21.15.71:0/2708674213 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63e800640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe446560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63e800640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe446fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 -- 172.21.15.71:0/2708674213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 msgr2=0x55defe446fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe446fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 -- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 msgr2=0x55defe446560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe446560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.856+0000 7fc63d400640 1 -- 172.21.15.71:0/2708674213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55defe183620 con 0x55defe43ecd0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc63d400640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 0x55defe446aa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0126b0 tx=0x7fc62c012b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65d2f5b062e9d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc61fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc61fe00640 1 -- 172.21.15.71:0/2708674213 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c011bb0 con 0x55defe43ecd0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc61fe00640 1 -- 172.21.15.71:0/2708674213 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc62c011d50 con 0x55defe43ecd0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc638003e50 con 0x55defe43ecd0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc61fe00640 1 -- 172.21.15.71:0/2708674213 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc62c005020 con 0x55defe43ecd0 2024-09-05T12:33:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc647dbc740 10 client.16017 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55defe457e20 con 0x55defe43ecd0 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.857+0000 7fc61fe00640 1 -- 172.21.15.71:0/2708674213 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638003e50 con 0x55defe43ecd0 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc61fe00640 1 -- 172.21.15.71:0/2708674213 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc62c01c470 con 0x55defe43ecd0 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc647dbc740 10 client.16017 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc647dbc740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc647dbc740 20 client.16017 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc647dbc740 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4592e0 0x55defe45b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc647dbc740 4 client.16017 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55defe3b1830 con 0x55defe4592e0 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.858+0000 7fc63e800640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4592e0 0x55defe45b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.859+0000 7fc63e800640 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4592e0 0x55defe45b740 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e83277bef4c25e server_cookie=306b5f7770e47402 in_seq=0 out_seq=0 2024-09-05T12:33:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.859+0000 7fc61fe00640 10 client.16017 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.859+0000 7fc61fe00640 1 -- 172.21.15.71:0/2708674213 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55defe3b1830 con 0x55defe4592e0 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.859+0000 7fc61fe00640 10 client.16017 handle_command_reply: tid=0 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.859+0000 7fc647dbc740 1 client.16017 shutdown 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 msgr2=0x55defe446aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 0x55defe446aa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0126b0 tx=0x7fc62c012b70 comp rx=0 tx=0).stop 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 shutdown_connections 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4592e0 0x55defe45b740 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe446fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43ecd0 0x55defe446aa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 --2- 172.21.15.71:0/2708674213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe446560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 >> 172.21.15.71:0/2708674213 conn(0x55defe43a920 msgr2=0x55defe440630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc61fe00640 0 client.16017 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 shutdown_connections 2024-09-05T12:33:26.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 1 -- 172.21.15.71:0/2708674213 wait complete. 2024-09-05T12:33:26.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.860+0000 7fc647dbc740 20 client.16017 trim_cache size 0 max 16384 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 Processor -- start 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 -- start start 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe4483c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe461050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 0x55defe461590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401b0dc0 con 0x55defe449570 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401b2960 con 0x55defe43e3b0 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc647dbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401b1490 con 0x55defe43ecd0 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc63e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 0x55defe461590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc63e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 0x55defe461590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46902/0 (socket says 172.21.15.71:46902) 2024-09-05T12:33:26.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc63e800640 1 -- 172.21.15.71:0/3541526982 learned_addr learned my addr 172.21.15.71:0/3541526982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc63d400640 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe461050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.875+0000 7fc63e800640 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe4483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc63e800640 1 -- 172.21.15.71:0/3541526982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 msgr2=0x55defe461050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc63e800640 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe461050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc63e800640 1 -- 172.21.15.71:0/3541526982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 msgr2=0x55defe4483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc63e800640 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe4483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc63e800640 1 -- 172.21.15.71:0/3541526982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55defe43b9d0 con 0x55defe449570 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc63e800640 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 0x55defe461590 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc63802fd00 tx=0x7fc63802fd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b9e4f2b1e4e035 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc62aa00640 1 -- 172.21.15.71:0/3541526982 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638048040 con 0x55defe449570 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc62aa00640 1 -- 172.21.15.71:0/3541526982 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638049040 con 0x55defe449570 2024-09-05T12:33:26.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.876+0000 7fc62aa00640 1 -- 172.21.15.71:0/3541526982 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6380046d0 con 0x55defe449570 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 -- 172.21.15.71:0/3541526982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 msgr2=0x55defe461590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 0x55defe461590 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc63802fd00 tx=0x7fc63802fd30 comp rx=0 tx=0).stop 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 -- 172.21.15.71:0/3541526982 shutdown_connections 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe449570 0x55defe461590 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe43ecd0 0x55defe461050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 --2- 172.21.15.71:0/3541526982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43e3b0 0x55defe4483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 -- 172.21.15.71:0/3541526982 >> 172.21.15.71:0/3541526982 conn(0x55defe43a920 msgr2=0x55defe447cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 -- 172.21.15.71:0/3541526982 shutdown_connections 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 -- 172.21.15.71:0/3541526982 wait complete. 2024-09-05T12:33:26.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 -- start start 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.877+0000 7fc647dbc740 1 Processor -- start 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe44c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43ecd0 0x55defe44cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe4c2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401b2290 con 0x55defe43e3b0 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401b1bc0 con 0x55defe43ecd0 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6401aeda0 con 0x55defe449570 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc647dbc740 10 client.0 _ll_register_callbacks cb 0x7fc62c0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe44c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe44c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46914/0 (socket says 172.21.15.71:46914) 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 -- 172.21.15.71:0/492310150 learned_addr learned my addr 172.21.15.71:0/492310150 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:26.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63d400640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43ecd0 0x55defe44cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe4c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 -- 172.21.15.71:0/492310150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 msgr2=0x55defe4c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe4c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 -- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43ecd0 msgr2=0x55defe44cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43ecd0 0x55defe44cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.878+0000 7fc63e800640 1 -- 172.21.15.71:0/492310150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55defe4c2550 con 0x55defe43e3b0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc63e800640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe44c7e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc638004af0 tx=0x7fc6380042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521a4c1cb08df08d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc61fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc61fe00640 1 -- 172.21.15.71:0/492310150 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc638048070 con 0x55defe43e3b0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc61fe00640 1 -- 172.21.15.71:0/492310150 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc638004d50 con 0x55defe43e3b0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc638049040 con 0x55defe43e3b0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc61fe00640 1 -- 172.21.15.71:0/492310150 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc638034070 con 0x55defe43e3b0 2024-09-05T12:33:26.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc647dbc740 10 client.16023 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55defe4c37d0 con 0x55defe43e3b0 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.879+0000 7fc61fe00640 1 -- 172.21.15.71:0/492310150 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc638049040 con 0x55defe43e3b0 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc61fe00640 1 -- 172.21.15.71:0/492310150 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fc638003b30 con 0x55defe43e3b0 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc647dbc740 10 client.16023 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc647dbc740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc647dbc740 20 client.16023 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc647dbc740 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4c39f0 0x55defe4c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc647dbc740 4 client.16023 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55defe3abb80 con 0x55defe4c39f0 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.880+0000 7fc63d400640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4c39f0 0x55defe4c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.881+0000 7fc63d400640 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4c39f0 0x55defe4c5de0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c7d19eb40857a7 server_cookie=2e265a3af74505f in_seq=0 out_seq=0 2024-09-05T12:33:26.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.881+0000 7fc61fe00640 10 client.16023 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc61fe00640 1 -- 172.21.15.71:0/492310150 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55defe3abb80 con 0x55defe4c39f0 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc61fe00640 10 client.16023 handle_command_reply: tid=0 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc647dbc740 1 client.16023 shutdown 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 msgr2=0x55defe44c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc647dbc740 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe44c7e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc638004af0 tx=0x7fc6380042b0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 shutdown_connections 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.900+0000 7fc647dbc740 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55defe4c39f0 0x55defe4c5de0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55defe449570 0x55defe4c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55defe43ecd0 0x55defe44cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc61fe00640 0 client.16023 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 1 --2- 172.21.15.71:0/492310150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55defe43e3b0 0x55defe44c7e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 >> 172.21.15.71:0/492310150 conn(0x55defe43a920 msgr2=0x55defe440630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 shutdown_connections 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 1 -- 172.21.15.71:0/492310150 wait complete. 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc647dbc740 20 client.16023 trim_cache size 0 max 16384 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:33:26.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:33:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6240a2d70 msgr2=0x7fc6240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc63f200640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6240a2d70 0x7fc6240a5230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc6300ccf70 tx=0x7fc6300d35f0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 msgr2=0x7fc64019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.901+0000 7fc63f200640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 0x7fc64019a400 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc63000e9f0 tx=0x7fc63000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 shutdown_connections 2024-09-05T12:33:26.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64010bf00 0x7fc64019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc640103400 0x7fc64019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6240a2d70 0x7fc6240a5230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 --2- 172.21.15.71:0/3357879289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc640102ae0 0x7fc64019a400 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 >> 172.21.15.71:0/3357879289 conn(0x7fc6400777c0 msgr2=0x7fc640100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 shutdown_connections 2024-09-05T12:33:26.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:26.902+0000 7fc63f200640 1 -- 172.21.15.71:0/3357879289 wait complete. 2024-09-05T12:33:26.909 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:33:27.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 4.02 MB/sec execute 143 sec latency 25404.974 ms 2024-09-05T12:33:27.278 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:33:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 -- 172.21.15.71:0/3521552281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54112870 msgr2=0x7f4e54114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 --2- 172.21.15.71:0/3521552281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54112870 0x7f4e54114c60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009ca0 tx=0x7f4e4c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:33:27.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 -- 172.21.15.71:0/3521552281 shutdown_connections 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 --2- 172.21.15.71:0/3521552281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54112870 0x7f4e54114c60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 --2- 172.21.15.71:0/3521552281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e54077040 0x7f4e54112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 --2- 172.21.15.71:0/3521552281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e54076720 0x7f4e54076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 -- 172.21.15.71:0/3521552281 >> 172.21.15.71:0/3521552281 conn(0x7f4e540ff8d0 msgr2=0x7f4e54101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 -- 172.21.15.71:0/3521552281 shutdown_connections 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.879+0000 7f4e53200640 1 -- 172.21.15.71:0/3521552281 wait complete. 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.880+0000 7f4e53200640 1 Processor -- start 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.880+0000 7f4e53200640 1 -- start start 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e53200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e54076720 0x7f4e541a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e53200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 0x7f4e541a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:27.885 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e53200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e54112870 0x7f4e5419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e53200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54118400 con 0x7f4e54077040 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e53200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54118280 con 0x7f4e54076720 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e53200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54118580 con 0x7f4e54112870 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e51400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 0x7f4e541a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e51400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 0x7f4e541a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46920/0 (socket says 172.21.15.71:46920) 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e51400640 1 -- 172.21.15.71:0/4245065908 learned_addr learned my addr 172.21.15.71:0/4245065908 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e52800640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e54112870 0x7f4e5419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.881+0000 7f4e51e00640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e54076720 0x7f4e541a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e51400640 1 -- 172.21.15.71:0/4245065908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e54112870 msgr2=0x7f4e5419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e51400640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e54112870 0x7f4e5419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e51400640 1 -- 172.21.15.71:0/4245065908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e54076720 msgr2=0x7f4e541a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:27.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e51400640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e54076720 0x7f4e541a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e51400640 1 -- 172.21.15.71:0/4245065908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5419efa0 con 0x7f4e54077040 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e51400640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 0x7f4e541a4ba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000d950 tx=0x7f4e4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e40017070 con 0x7f4e54077040 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e40009c30 con 0x7f4e54077040 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4000f6b0 con 0x7f4e54077040 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5419f290 con 0x7f4e54077040 2024-09-05T12:33:27.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.882+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e541ab5e0 con 0x7f4e54077040 2024-09-05T12:33:27.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.883+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e54077e60 con 0x7f4e54077040 2024-09-05T12:33:27.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.885+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4e400109d0 con 0x7f4e54077040 2024-09-05T12:33:27.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.886+0000 7f4e4aa00640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4e380a2d70 0x7f4e380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:27.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.887+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f4e400cf290 con 0x7f4e54077040 2024-09-05T12:33:27.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.887+0000 7f4e51e00640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4e380a2d70 0x7f4e380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:27.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.887+0000 7f4e51e00640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4e380a2d70 0x7f4e380a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009bd0 tx=0x7f4e44005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:27.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:27.891+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e40009de0 con 0x7f4e54077040 2024-09-05T12:33:28.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:27 smithi174 ceph-mon[34313]: pgmap v174: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-05T12:33:28.103 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.102+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e5410da00 con 0x7f4e54077040 2024-09-05T12:33:28.104 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:27 smithi071 ceph-mon[29845]: pgmap v174: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-05T12:33:28.104 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.102+0000 7f4e4aa00640 1 -- 172.21.15.71:0/4245065908 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11971 (secure 0 0 0) 0x7f4e4009b4c0 con 0x7f4e54077040 2024-09-05T12:33:28.105 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:33:28.105 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"btime":"2024-09-05T12:32:37:493040+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:32:36.945626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:33:28.105 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4e380a2d70 msgr2=0x7f4e380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4e380a2d70 0x7f4e380a5230 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009bd0 tx=0x7f4e44005f90 comp rx=0 tx=0).stop 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 msgr2=0x7f4e541a4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 0x7f4e541a4ba0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4e4000d950 tx=0x7f4e4000de10 comp rx=0 tx=0).stop 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 shutdown_connections 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e54112870 0x7f4e5419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4e380a2d70 0x7f4e380a5230 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4e54077040 0x7f4e541a4ba0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 --2- 172.21.15.71:0/4245065908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e54076720 0x7f4e541a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:28.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 >> 172.21.15.71:0/4245065908 conn(0x7f4e540ff8d0 msgr2=0x7f4e54101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:28.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.105+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 shutdown_connections 2024-09-05T12:33:28.108 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:28.106+0000 7f4e53200640 1 -- 172.21.15.71:0/4245065908 wait complete. 2024-09-05T12:33:28.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:27 smithi123 ceph-mon[34321]: pgmap v174: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 1.7 MiB/s wr, 89 op/s 2024-09-05T12:33:28.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 3.99 MB/sec execute 144 sec latency 26405.071 ms 2024-09-05T12:33:28.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:28 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4245065908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:33:28.996 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 84daed41 --release --await' 2024-09-05T12:33:28.996 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 84daed41 --release --await 2024-09-05T12:33:29.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:28 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4245065908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:33:29.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:28 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4245065908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:33:29.212 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 3.97 MB/sec execute 145 sec latency 27405.180 ms 2024-09-05T12:33:29.364 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:33:29.896 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:29 smithi071 ceph-mon[29845]: pgmap v175: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-09-05T12:33:30.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:29 smithi174 ceph-mon[34313]: pgmap v175: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.012+0000 7f74c7800640 1 -- 172.21.15.71:0/2279314035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0106f10 msgr2=0x7f74c010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.012+0000 7f74c7800640 1 --2- 172.21.15.71:0/2279314035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0106f10 0x7f74c010ea90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009ca0 tx=0x7f74b802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 -- 172.21.15.71:0/2279314035 shutdown_connections 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 --2- 172.21.15.71:0/2279314035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0106f10 0x7f74c010ea90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 --2- 172.21.15.71:0/2279314035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c0102790 0x7f74c01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 --2- 172.21.15.71:0/2279314035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101de0 0x7f74c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 -- 172.21.15.71:0/2279314035 >> 172.21.15.71:0/2279314035 conn(0x7f74c00fd8c0 msgr2=0x7f74c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 -- 172.21.15.71:0/2279314035 shutdown_connections 2024-09-05T12:33:30.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.013+0000 7f74c7800640 1 -- 172.21.15.71:0/2279314035 wait complete. 2024-09-05T12:33:30.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 Processor -- start 2024-09-05T12:33:30.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 -- start start 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0101de0 0x7f74c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 0x7f74c019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c0106f10 0x7f74c019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111f70 con 0x7f74c0101de0 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111df0 con 0x7f74c0106f10 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.014+0000 7f74c7800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01120f0 con 0x7f74c0102790 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 0x7f74c019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74c4e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0101de0 0x7f74c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 0x7f74c019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34776/0 (socket says 172.21.15.71:34776) 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 -- 172.21.15.71:0/631493367 learned_addr learned my addr 172.21.15.71:0/631493367 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:30.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74c5800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c0106f10 0x7f74c019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 -- 172.21.15.71:0/631493367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c0106f10 msgr2=0x7f74c019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c0106f10 0x7f74c019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 -- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0101de0 msgr2=0x7f74c019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0101de0 0x7f74c019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 -- 172.21.15.71:0/631493367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c019f620 con 0x7f74c0102790 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.015+0000 7f74b7e00640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 0x7f74c019abb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f74a800d950 tx=0x7f74a800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:30.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.016+0000 7f74be000640 1 -- 172.21.15.71:0/631493367 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8017030 con 0x7f74c0102790 2024-09-05T12:33:30.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.016+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c019f910 con 0x7f74c0102790 2024-09-05T12:33:30.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.016+0000 7f74be000640 1 -- 172.21.15.71:0/631493367 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74a8004020 con 0x7f74c0102790 2024-09-05T12:33:30.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.016+0000 7f74be000640 1 -- 172.21.15.71:0/631493367 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a80047a0 con 0x7f74c0102790 2024-09-05T12:33:30.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.016+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01a71f0 con 0x7f74c0102790 2024-09-05T12:33:30.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 Processor -- start 2024-09-05T12:33:30.021 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 -- start start 2024-09-05T12:33:30.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9310 0x5634f6ca96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6cb44f0 0x5634f6cb68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5634f6c0f650 con 0x5634f6ca9310 2024-09-05T12:33:30.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634f6bb6f10 con 0x5634f6cb44f0 2024-09-05T12:33:30.022 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.020+0000 7f74ca266740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5634f6c0b610 con 0x5634f6ca9c30 2024-09-05T12:33:30.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c4e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34784/0 (socket says 172.21.15.71:34784) 2024-09-05T12:33:30.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c5800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9310 0x5634f6ca96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c4e00640 1 -- 172.21.15.71:0/1032758675 learned_addr learned my addr 172.21.15.71:0/1032758675 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:30.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c5800640 1 -- 172.21.15.71:0/1032758675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 msgr2=0x5634f6cb3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:30.023 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c5800640 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c5800640 1 -- 172.21.15.71:0/1032758675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6cb44f0 msgr2=0x5634f6cb68e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c5800640 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6cb44f0 0x5634f6cb68e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74be000640 1 -- 172.21.15.71:0/631493367 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f74a80041d0 con 0x7f74c0102790 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.021+0000 7f74c5800640 1 -- 172.21.15.71:0/1032758675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634f691e580 con 0x5634f6ca9310 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.022+0000 7f74c5800640 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9310 0x5634f6ca96f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74b8006010 tx=0x7f74b8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bc15ca9b8859d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.022+0000 7f74b6000640 1 -- 172.21.15.71:0/1032758675 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8031db0 con 0x5634f6ca9310 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.022+0000 7f74b6000640 1 -- 172.21.15.71:0/1032758675 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b8036040 con 0x5634f6ca9310 2024-09-05T12:33:30.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.022+0000 7f74b6000640 1 -- 172.21.15.71:0/1032758675 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8004330 con 0x5634f6ca9310 2024-09-05T12:33:30.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74be000640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 -- 172.21.15.71:0/1032758675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9310 msgr2=0x5634f6ca96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9310 0x5634f6ca96f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f74b8006010 tx=0x7f74b8009800 comp rx=0 tx=0).stop 2024-09-05T12:33:30.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74be000640 1 -- 172.21.15.71:0/631493367 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f74a80cef80 con 0x7f74c0102790 2024-09-05T12:33:30.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 -- 172.21.15.71:0/1032758675 shutdown_connections 2024-09-05T12:33:30.025 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6cb44f0 0x5634f6cb68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 --2- 172.21.15.71:0/1032758675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9310 0x5634f6ca96f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.023+0000 7f74ca266740 1 -- 172.21.15.71:0/1032758675 >> 172.21.15.71:0/1032758675 conn(0x5634f6ca58a0 msgr2=0x5634f6ca5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:30.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.024+0000 7f74c4e00640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.024+0000 7f74ca266740 1 -- 172.21.15.71:0/1032758675 shutdown_connections 2024-09-05T12:33:30.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.024+0000 7f74ca266740 1 -- 172.21.15.71:0/1032758675 wait complete. 2024-09-05T12:33:30.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.025+0000 7f74ca266740 1 -- start start 2024-09-05T12:33:30.027 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.025+0000 7f74ca266740 1 Processor -- start 2024-09-05T12:33:30.028 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cb1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cb1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5634f6cb6ee0 con 0x5634f6cb44f0 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634f6bca350 con 0x5634f6ca9310 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5634f6cb7060 con 0x5634f6ca9c30 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.026+0000 7f74ca266740 10 client.0 _ll_register_callbacks cb 0x5634f6bed290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.027+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.027+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34790/0 (socket says 172.21.15.71:34790) 2024-09-05T12:33:30.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.027+0000 7f74b7e00640 1 -- 172.21.15.71:0/2279408009 learned_addr learned my addr 172.21.15.71:0/2279408009 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.027+0000 7f74b7e00640 1 -- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 msgr2=0x5634f6cb1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.027+0000 7f74b7e00640 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cb1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.027+0000 7f74b7e00640 1 -- 172.21.15.71:0/2279408009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 msgr2=0x5634f6cb1e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:33:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.028+0000 7f74b7e00640 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cb1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.028+0000 7f74b7e00640 1 -- 172.21.15.71:0/2279408009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634f69e8960 con 0x5634f6ca9c30 2024-09-05T12:33:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.029+0000 7f74c4e00640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009d60 tx=0x7f74b0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.029+0000 7f74b7e00640 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb1940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74a80cc390 tx=0x7f74a809ad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=762cfbab2dcb438a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:30.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.029+0000 7f74aea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:33:30.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.030+0000 7f74aea00640 1 -- 172.21.15.71:0/2279408009 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a809f4b0 con 0x5634f6ca9c30 2024-09-05T12:33:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.030+0000 7f74aea00640 1 -- 172.21.15.71:0/2279408009 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f74a809f120 con 0x5634f6ca9c30 2024-09-05T12:33:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.030+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b8036040 con 0x5634f6ca9c30 2024-09-05T12:33:30.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.031+0000 7f74aea00640 1 -- 172.21.15.71:0/2279408009 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f74a80a3d70 con 0x5634f6ca9c30 2024-09-05T12:33:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.031+0000 7f74ca266740 10 client.26089 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.031+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634f6cb9ad0 con 0x5634f6ca9c30 2024-09-05T12:33:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.032+0000 7f74aea00640 1 -- 172.21.15.71:0/2279408009 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b8036040 con 0x5634f6ca9c30 2024-09-05T12:33:30.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.033+0000 7f74aea00640 1 -- 172.21.15.71:0/2279408009 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f74a8099fc0 con 0x5634f6ca9c30 2024-09-05T12:33:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.033+0000 7f74ca266740 10 client.26089 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.033+0000 7f74ca266740 10 client.26089 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:33:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.033+0000 7f74ca266740 20 client.26089 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.033+0000 7f74ca266740 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6cbb030 0x5634f6cbd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.034+0000 7f74ca266740 4 client.26089 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:30.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.034+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634f6c1cbc0 con 0x5634f6cbb030 2024-09-05T12:33:30.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.035+0000 7f74c4e00640 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6cbb030 0x5634f6cbd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.037+0000 7f74c4e00640 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6cbb030 0x5634f6cbd4d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc8480c38c4b052 server_cookie=938d3e0fed988385 in_seq=0 out_seq=0 2024-09-05T12:33:30.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.038+0000 7f74aea00640 10 client.26089 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:30.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.039+0000 7f74aea00640 1 -- 172.21.15.71:0/2279408009 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634f6c1cbc0 con 0x5634f6cbb030 2024-09-05T12:33:30.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.039+0000 7f74aea00640 10 client.26089 handle_command_reply: tid=0 2024-09-05T12:33:30.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.039+0000 7f74ca266740 1 client.26089 shutdown 2024-09-05T12:33:30.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.041+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 msgr2=0x5634f6cb1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.041+0000 7f74ca266740 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb1940 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74a80cc390 tx=0x7f74a809ad70 comp rx=0 tx=0).stop 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.041+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 shutdown_connections 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.041+0000 7f74ca266740 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6cbb030 0x5634f6cbd4d0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.041+0000 7f74ca266740 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cb1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.042+0000 7f74ca266740 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cb1940 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.042+0000 7f74ca266740 1 --2- 172.21.15.71:0/2279408009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cb1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.042+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 >> 172.21.15.71:0/2279408009 conn(0x5634f6ca58a0 msgr2=0x5634f6cab620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.042+0000 7f74aea00640 0 client.26089 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:30.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.042+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 shutdown_connections 2024-09-05T12:33:30.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.042+0000 7f74ca266740 1 -- 172.21.15.71:0/2279408009 wait complete. 2024-09-05T12:33:30.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.043+0000 7f74ca266740 20 client.26089 trim_cache size 0 max 16384 2024-09-05T12:33:30.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 Processor -- start 2024-09-05T12:33:30.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 -- start start 2024-09-05T12:33:30.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cc82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9c30 0x5634f6cdec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 0x5634f6cdf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f749c00fda0 con 0x5634f6ca9c30 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f749c011940 con 0x5634f6ca9310 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.076+0000 7f74ca266740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f749c010470 con 0x5634f6cb44f0 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 0x5634f6cdf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9c30 0x5634f6cdec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 0x5634f6cdf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34800/0 (socket says 172.21.15.71:34800) 2024-09-05T12:33:30.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 -- 172.21.15.71:0/3742474854 learned_addr learned my addr 172.21.15.71:0/3742474854 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 -- 172.21.15.71:0/3742474854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 msgr2=0x5634f6cc82b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cc82b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 -- 172.21.15.71:0/3742474854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9c30 msgr2=0x5634f6cdec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9c30 0x5634f6cdec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 -- 172.21.15.71:0/3742474854 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634f6ca1260 con 0x5634f6cb44f0 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.081+0000 7f74c5800640 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 0x5634f6cdf140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009620 tx=0x7f74b80099f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f373d558a5abc0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:30.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.082+0000 7f74b5600640 1 -- 172.21.15.71:0/3742474854 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8031db0 con 0x5634f6cb44f0 2024-09-05T12:33:30.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.082+0000 7f74b5600640 1 -- 172.21.15.71:0/3742474854 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b8036040 con 0x5634f6cb44f0 2024-09-05T12:33:30.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.082+0000 7f74b5600640 1 -- 172.21.15.71:0/3742474854 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b80042a0 con 0x5634f6cb44f0 2024-09-05T12:33:30.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.084+0000 7f74ca266740 1 -- 172.21.15.71:0/3742474854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 msgr2=0x5634f6cdf140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.084+0000 7f74ca266740 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 0x5634f6cdf140 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009620 tx=0x7f74b80099f0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- 172.21.15.71:0/3742474854 shutdown_connections 2024-09-05T12:33:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6cb44f0 0x5634f6cdf140 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6ca9c30 0x5634f6cdec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 --2- 172.21.15.71:0/3742474854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cc82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- 172.21.15.71:0/3742474854 >> 172.21.15.71:0/3742474854 conn(0x5634f6ca58a0 msgr2=0x5634f6cab620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:30.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- 172.21.15.71:0/3742474854 shutdown_connections 2024-09-05T12:33:30.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- 172.21.15.71:0/3742474854 wait complete. 2024-09-05T12:33:30.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- start start 2024-09-05T12:33:30.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 Processor -- start 2024-09-05T12:33:30.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cc3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f749c011270 con 0x5634f6cb44f0 2024-09-05T12:33:30.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f749c010ba0 con 0x5634f6ca9310 2024-09-05T12:33:30.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f749c00dd80 con 0x5634f6ca9c30 2024-09-05T12:33:30.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74ca266740 10 client.0 _ll_register_callbacks cb 0x7f74a80046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:30.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cae1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34814/0 (socket says 172.21.15.71:34814) 2024-09-05T12:33:30.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 -- 172.21.15.71:0/2554757647 learned_addr learned my addr 172.21.15.71:0/2554757647 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:30.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74c5800640 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 -- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 msgr2=0x5634f6cc3590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:33:30.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cc3590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 -- 172.21.15.71:0/2554757647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 msgr2=0x5634f6cae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:30.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 -- 172.21.15.71:0/2554757647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634f6caef40 con 0x5634f6ca9c30 2024-09-05T12:33:30.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74b7e00640 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cae1b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74a8005fd0 tx=0x7f74a80bb2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c6ad44421812a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:30.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.085+0000 7f74aea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:33:30.092 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74aea00640 1 -- 172.21.15.71:0/2554757647 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8014020 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74aea00640 1 -- 172.21.15.71:0/2554757647 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f74a809a550 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b8036040 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74aea00640 1 -- 172.21.15.71:0/2554757647 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f74a809aed0 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 10 client.26095 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634f6d3fc90 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74aea00640 1 -- 172.21.15.71:0/2554757647 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b8036040 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74aea00640 1 -- 172.21.15.71:0/2554757647 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f74a80a0840 con 0x5634f6ca9c30 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 10 client.26095 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 10 client.26095 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:33:30.093 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 20 client.26095 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:30.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6d40430 0x5634f6d42850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:30.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 4 client.26095 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "84daed41", "release": true, "await": true}] 2024-09-05T12:33:30.094 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.090+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "84daed41", "release": true, "await": true}) -- 0x5634f6c170d0 con 0x5634f6d40430 2024-09-05T12:33:30.107 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.106+0000 7f74c4e00640 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6d40430 0x5634f6d42850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.130+0000 7f74c4e00640 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6d40430 0x5634f6d42850 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48010e515adcdb7 server_cookie=57dce242eda31117 in_seq=0 out_seq=0 2024-09-05T12:33:30.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.130+0000 7f74aea00640 10 client.26095 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:30.138 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74aea00640 1 -- 172.21.15.71:0/2554757647 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5634f6c170d0 con 0x5634f6d40430 2024-09-05T12:33:30.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74aea00640 10 client.26095 handle_command_reply: tid=0 2024-09-05T12:33:30.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 client.26095 shutdown 2024-09-05T12:33:30.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 msgr2=0x5634f6cae1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cae1b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f74a8005fd0 tx=0x7f74a80bb2e0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 shutdown_connections 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5634f6d40430 0x5634f6d42850 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634f6cb44f0 0x5634f6cae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634f6ca9c30 0x5634f6cae1b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 --2- 172.21.15.71:0/2554757647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5634f6ca9310 0x5634f6cc3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74aea00640 0 client.26095 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 >> 172.21.15.71:0/2554757647 conn(0x5634f6ca58a0 msgr2=0x5634f6cb13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 shutdown_connections 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 1 -- 172.21.15.71:0/2554757647 wait complete. 2024-09-05T12:33:30.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74ca266740 20 client.26095 trim_cache size 0 max 16384 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 36, 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 20, 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "84daed41": { 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 20, 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:33:30.141 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:30.142 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:30.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:30.143 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:30.143 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 msgr2=0x7f74a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009d60 tx=0x7f74b0005f90 comp rx=0 tx=0).stop 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 msgr2=0x7f74c019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 0x7f74c019abb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f74a800d950 tx=0x7f74a800de10 comp rx=0 tx=0).stop 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 shutdown_connections 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c0106f10 0x7f74c019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102790 0x7f74c019abb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 --2- 172.21.15.71:0/631493367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0101de0 0x7f74c019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 >> 172.21.15.71:0/631493367 conn(0x7f74c00fd8c0 msgr2=0x7f74c00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:30.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 shutdown_connections 2024-09-05T12:33:30.145 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:30.135+0000 7f74c7800640 1 -- 172.21.15.71:0/631493367 wait complete. 2024-09-05T12:33:30.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:29 smithi123 ceph-mon[34321]: pgmap v175: 129 pgs: 129 active+clean; 72 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 1.0 MiB/s wr, 55 op/s 2024-09-05T12:33:30.213 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202387 3.94 MB/sec execute 146 sec latency 28406.316 ms 2024-09-05T12:33:31.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 202455 3.91 MB/sec execute 147 sec latency 28452.464 ms 2024-09-05T12:33:31.399 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '84daed41', total seconds elapsed: 34.4 2024-09-05T12:33:31.399 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:33:31.399 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:33:31.825 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:33:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:31 smithi071 ceph-mon[29845]: pgmap v176: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-09-05T12:33:32.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:31 smithi123 ceph-mon[34321]: pgmap v176: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-09-05T12:33:32.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 203474 3.90 MB/sec execute 148 sec latency 43.746 ms 2024-09-05T12:33:32.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:31 smithi174 ceph-mon[34313]: pgmap v176: 129 pgs: 129 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 1.4 MiB/s wr, 81 op/s 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.507+0000 7fc966200640 1 -- 172.21.15.71:0/3783113451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc960074ee0 msgr2=0x7fc96010dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.507+0000 7fc966200640 1 --2- 172.21.15.71:0/3783113451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc960074ee0 0x7fc96010dee0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009900 tx=0x7fc94c02a480 comp rx=0 tx=0).stop 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.507+0000 7fc966200640 1 -- 172.21.15.71:0/3783113451 shutdown_connections 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.507+0000 7fc966200640 1 --2- 172.21.15.71:0/3783113451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc96010e420 0x7fc960110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.507+0000 7fc966200640 1 --2- 172.21.15.71:0/3783113451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc960074ee0 0x7fc96010dee0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.507+0000 7fc966200640 1 --2- 172.21.15.71:0/3783113451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9600745c0 0x7fc9600749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.508+0000 7fc966200640 1 -- 172.21.15.71:0/3783113451 >> 172.21.15.71:0/3783113451 conn(0x7fc9600fd8c0 msgr2=0x7fc9600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.508+0000 7fc966200640 1 -- 172.21.15.71:0/3783113451 shutdown_connections 2024-09-05T12:33:32.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.508+0000 7fc966200640 1 -- 172.21.15.71:0/3783113451 wait complete. 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 Processor -- start 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 -- start start 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 0x7fc96019a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc960074ee0 0x7fc96019ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc96010e420 0x7fc96019f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc9601141b0 con 0x7fc960074ee0 2024-09-05T12:33:32.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc960114030 con 0x7fc9600745c0 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.509+0000 7fc966200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc960114330 con 0x7fc96010e420 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 0x7fc96019a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc96010e420 0x7fc96019f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 0x7fc96019a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:46888/0 (socket says 172.21.15.71:46888) 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 -- 172.21.15.71:0/2095847798 learned_addr learned my addr 172.21.15.71:0/2095847798 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95ea00640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc960074ee0 0x7fc96019ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 -- 172.21.15.71:0/2095847798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc96010e420 msgr2=0x7fc96019f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc96010e420 0x7fc96019f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 -- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc960074ee0 msgr2=0x7fc96019ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc960074ee0 0x7fc96019ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 -- 172.21.15.71:0/2095847798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96019f700 con 0x7fc9600745c0 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95fe00640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc96010e420 0x7fc96019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95ea00640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc960074ee0 0x7fc96019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:32.513 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.510+0000 7fc95f400640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 0x7fc96019a750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc95000e9f0 tx=0x7fc95000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.511+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95000cd80 con 0x7fc9600745c0 2024-09-05T12:33:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.511+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc950010910 con 0x7fc9600745c0 2024-09-05T12:33:32.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.511+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96019f9f0 con 0x7fc9600745c0 2024-09-05T12:33:32.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.511+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950018990 con 0x7fc9600745c0 2024-09-05T12:33:32.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.511+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9601a72d0 con 0x7fc9600745c0 2024-09-05T12:33:32.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.512+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc940005290 con 0x7fc9600745c0 2024-09-05T12:33:32.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.515+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc950018b30 con 0x7fc9600745c0 2024-09-05T12:33:32.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.517+0000 7fc957e00640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc9440a2d70 0x7fc9440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:32.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.517+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc9500cf9b0 con 0x7fc9600745c0 2024-09-05T12:33:32.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.517+0000 7fc95ea00640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc9440a2d70 0x7fc9440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:32.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.519+0000 7fc95ea00640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc9440a2d70 0x7fc9440a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc94c000c00 tx=0x7fc94c005a40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:32.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.521+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc95009c230 con 0x7fc9600745c0 2024-09-05T12:33:32.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.776+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc940005850 con 0x7fc9600745c0 2024-09-05T12:33:32.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.777+0000 7fc957e00640 1 -- 172.21.15.71:0/2095847798 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11971 (secure 0 0 0) 0x7fc95009bb30 con 0x7fc9600745c0 2024-09-05T12:33:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:33:32.780 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":36,"btime":"2024-09-05T12:32:37:493040+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:32:36.945626+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:33:32.780 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 36 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.780+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc9440a2d70 msgr2=0x7fc9440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.780+0000 7fc966200640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc9440a2d70 0x7fc9440a5230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc94c000c00 tx=0x7fc94c005a40 comp rx=0 tx=0).stop 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.780+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 msgr2=0x7fc96019a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.780+0000 7fc966200640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 0x7fc96019a750 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc95000e9f0 tx=0x7fc95000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 shutdown_connections 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc96010e420 0x7fc96019f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc9440a2d70 0x7fc9440a5230 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc960074ee0 0x7fc96019ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 --2- 172.21.15.71:0/2095847798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9600745c0 0x7fc96019a750 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 >> 172.21.15.71:0/2095847798 conn(0x7fc9600fd8c0 msgr2=0x7fc9600fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 shutdown_connections 2024-09-05T12:33:32.783 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:32.781+0000 7fc966200640 1 -- 172.21.15.71:0/2095847798 wait complete. 2024-09-05T12:33:33.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:32 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2095847798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:33:33.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:32 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2095847798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:33:33.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 205692 3.98 MB/sec execute 149 sec latency 26.985 ms 2024-09-05T12:33:33.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:32 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2095847798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:33:33.721 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:33:33.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:33:34.154 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:33:34.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:33 smithi071 ceph-mon[29845]: pgmap v177: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.0 MiB/s wr, 63 op/s 2024-09-05T12:33:34.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:33 smithi123 ceph-mon[34321]: pgmap v177: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.0 MiB/s wr, 63 op/s 2024-09-05T12:33:34.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 206415 3.96 MB/sec execute 150 sec latency 206.223 ms 2024-09-05T12:33:34.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:33 smithi174 ceph-mon[34313]: pgmap v177: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 1.0 MiB/s wr, 63 op/s 2024-09-05T12:33:34.784 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 -- 172.21.15.71:0/2984072509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b381034b0 msgr2=0x7f2b3810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 --2- 172.21.15.71:0/2984072509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b381034b0 0x7f2b3810b9a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009900 tx=0x7f2b2002a480 comp rx=0 tx=0).stop 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 -- 172.21.15.71:0/2984072509 shutdown_connections 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 --2- 172.21.15.71:0/2984072509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 0x7f2b3810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 --2- 172.21.15.71:0/2984072509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b381034b0 0x7f2b3810b9a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 --2- 172.21.15.71:0/2984072509 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b38102b90 0x7f2b38102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 -- 172.21.15.71:0/2984072509 >> 172.21.15.71:0/2984072509 conn(0x7f2b380fc610 msgr2=0x7f2b380fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 -- 172.21.15.71:0/2984072509 shutdown_connections 2024-09-05T12:33:34.785 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.782+0000 7f2b37e00640 1 -- 172.21.15.71:0/2984072509 wait complete. 2024-09-05T12:33:34.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.783+0000 7f2b37e00640 1 Processor -- start 2024-09-05T12:33:34.786 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.783+0000 7f2b37e00640 1 -- start start 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 0x7f2b3819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b381034b0 0x7f2b3819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 0x7f2b3819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b38111dc0 con 0x7f2b381034b0 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b38111c40 con 0x7f2b38102b90 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b38111f40 con 0x7f2b3810bfc0 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 0x7f2b3819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 0x7f2b3819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 0x7f2b3819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57120/0 (socket says 172.21.15.71:57120) 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 -- 172.21.15.71:0/225648021 learned_addr learned my addr 172.21.15.71:0/225648021 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b2e000640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b381034b0 0x7f2b3819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 -- 172.21.15.71:0/225648021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 msgr2=0x7f2b3819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 0x7f2b3819ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 -- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b381034b0 msgr2=0x7f2b3819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b381034b0 0x7f2b3819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b36a00640 1 -- 172.21.15.71:0/225648021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3819f510 con 0x7f2b38102b90 2024-09-05T12:33:34.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.784+0000 7f2b37400640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 0x7f2b3819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:33:34.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.785+0000 7f2b36a00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 0x7f2b3819a560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800e9f0 tx=0x7f2b2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:34.788 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.785+0000 7f2b2fe00640 1 -- 172.21.15.71:0/225648021 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800cd80 con 0x7f2b38102b90 2024-09-05T12:33:34.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.785+0000 7f2b2fe00640 1 -- 172.21.15.71:0/225648021 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b28010910 con 0x7f2b38102b90 2024-09-05T12:33:34.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.785+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3819f800 con 0x7f2b38102b90 2024-09-05T12:33:34.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.785+0000 7f2b2fe00640 1 -- 172.21.15.71:0/225648021 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28018990 con 0x7f2b38102b90 2024-09-05T12:33:34.789 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.785+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b381a70e0 con 0x7f2b38102b90 2024-09-05T12:33:34.790 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.788+0000 7f2b2fe00640 1 -- 172.21.15.71:0/225648021 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2b28010ac0 con 0x7f2b38102b90 2024-09-05T12:33:34.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.789+0000 7f2b40217740 1 Processor -- start 2024-09-05T12:33:34.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.789+0000 7f2b40217740 1 -- start start 2024-09-05T12:33:34.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b40217740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 0x5650267b4940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b2fe00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2b1c0a2d70 0x7f2b1c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b40217740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b40217740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267bf740 0x5650267c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b40217740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56502671a530 con 0x5650267b4e80 2024-09-05T12:33:34.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b40217740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5650266c1860 con 0x5650267bf740 2024-09-05T12:33:34.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b40217740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650267164f0 con 0x5650267b4560 2024-09-05T12:33:34.792 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b2fe00640 1 -- 172.21.15.71:0/225648021 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f2b280cfa60 con 0x7f2b38102b90 2024-09-05T12:33:34.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 0x5650267b4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b2e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 0x5650267b4940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:57112/0 (socket says 172.21.15.71:57112) 2024-09-05T12:33:34.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 -- 172.21.15.71:0/4241392539 learned_addr learned my addr 172.21.15.71:0/4241392539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:34.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b37400640 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267bf740 0x5650267c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.793 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b37400640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2b1c0a2d70 0x7f2b1c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 -- 172.21.15.71:0/4241392539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267bf740 msgr2=0x5650267c1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267bf740 0x5650267c1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 -- 172.21.15.71:0/4241392539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 msgr2=0x5650267bf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.790+0000 7f2b36a00640 1 -- 172.21.15.71:0/4241392539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56502648bb90 con 0x5650267b4560 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.791+0000 7f2b36a00640 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 0x5650267b4940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b280980f0 tx=0x7f2b280a17e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba570f2669ede0f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.791+0000 7f2b27e00640 1 -- 172.21.15.71:0/4241392539 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2809ff90 con 0x5650267b4560 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.791+0000 7f2b37400640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2b1c0a2d70 0x7f2b1c0a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b30005c40 tx=0x7f2b30009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.791+0000 7f2b27e00640 1 -- 172.21.15.71:0/4241392539 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b280a2300 con 0x5650267b4560 2024-09-05T12:33:34.794 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.791+0000 7f2b27e00640 1 -- 172.21.15.71:0/4241392539 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b280a2b90 con 0x5650267b4560 2024-09-05T12:33:34.795 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 -- 172.21.15.71:0/4241392539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 msgr2=0x5650267b4940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 0x5650267b4940 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2b280980f0 tx=0x7f2b280a17e0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 -- 172.21.15.71:0/4241392539 shutdown_connections 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267bf740 0x5650267c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 --2- 172.21.15.71:0/4241392539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4560 0x5650267b4940 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 -- 172.21.15.71:0/4241392539 >> 172.21.15.71:0/4241392539 conn(0x5650267b0af0 msgr2=0x5650267b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 -- 172.21.15.71:0/4241392539 shutdown_connections 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.792+0000 7f2b40217740 1 -- 172.21.15.71:0/4241392539 wait complete. 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.793+0000 7f2b40217740 1 -- start start 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.793+0000 7f2b40217740 1 Processor -- start 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267bd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5650267c2310 con 0x5650267b4e80 2024-09-05T12:33:34.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5650266d4e60 con 0x5650267b4560 2024-09-05T12:33:34.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650267c2490 con 0x5650267bf740 2024-09-05T12:33:34.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b40217740 10 client.0 _ll_register_callbacks cb 0x5650266d9cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:34.797 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57144/0 (socket says 172.21.15.71:57144) 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 -- 172.21.15.71:0/1497800661 learned_addr learned my addr 172.21.15.71:0/1497800661 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b2e000640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b37400640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267bd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 -- 172.21.15.71:0/1497800661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 msgr2=0x5650267bd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267bd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 -- 172.21.15.71:0/1497800661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 msgr2=0x5650267bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b36a00640 1 -- 172.21.15.71:0/1497800661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650266e0990 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.794+0000 7f2b37400640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267bd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b36a00640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bc720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2b2809bc20 tx=0x7f2b280a1850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ba1012ac05d57f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b24c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b24c00640 1 -- 172.21.15.71:0/1497800661 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b280b9060 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b280a2300 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b24c00640 1 -- 172.21.15.71:0/1497800661 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f2b280a3030 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b24c00640 1 -- 172.21.15.71:0/1497800661 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2b2809ff90 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b40217740 10 client.26087 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5650267ce010 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b24c00640 1 -- 172.21.15.71:0/1497800661 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b280a2300 con 0x5650267b4560 2024-09-05T12:33:34.800 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.795+0000 7f2b24c00640 1 -- 172.21.15.71:0/1497800661 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f2b280a2780 con 0x5650267b4560 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b40217740 10 client.26087 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b40217740 10 client.26087 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b40217740 20 client.26087 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b40217740 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5650267cf190 0x5650267d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b40217740 4 client.26087 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650266bd310 con 0x5650267cf190 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.796+0000 7f2b2e000640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5650267cf190 0x5650267d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.797+0000 7f2b2e000640 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5650267cf190 0x5650267d1580 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d92700aabaa121aa server_cookie=3f8e6a203f7d68e in_seq=0 out_seq=0 2024-09-05T12:33:34.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.797+0000 7f2b24c00640 10 client.26087 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:34.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b24c00640 1 -- 172.21.15.71:0/1497800661 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650266bd310 con 0x5650267cf190 2024-09-05T12:33:34.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b24c00640 10 client.26087 handle_command_reply: tid=0 2024-09-05T12:33:34.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 client.26087 shutdown 2024-09-05T12:33:34.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 msgr2=0x5650267bc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bc720 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2b2809bc20 tx=0x7f2b280a1850 comp rx=0 tx=0).stop 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 shutdown_connections 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5650267cf190 0x5650267d1580 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267bd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 --2- 172.21.15.71:0/1497800661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bc720 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 >> 172.21.15.71:0/1497800661 conn(0x5650267b0af0 msgr2=0x5650267b6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b24c00640 0 client.26087 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 shutdown_connections 2024-09-05T12:33:34.803 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 1 -- 172.21.15.71:0/1497800661 wait complete. 2024-09-05T12:33:34.804 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.798+0000 7f2b40217740 20 client.26087 trim_cache size 0 max 16384 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 Processor -- start 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 -- start start 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267d41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381b1000 con 0x5650267b4e80 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381b2ba0 con 0x5650267b4560 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.827+0000 7f2b40217740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381b16d0 con 0x5650267bf740 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57146/0 (socket says 172.21.15.71:57146) 2024-09-05T12:33:34.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 -- 172.21.15.71:0/780787325 learned_addr learned my addr 172.21.15.71:0/780787325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b2e000640 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267d41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b37400640 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 -- 172.21.15.71:0/780787325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 msgr2=0x5650267d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 -- 172.21.15.71:0/780787325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 msgr2=0x5650267d41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267d41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 -- 172.21.15.71:0/780787325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650267bac70 con 0x5650267b4560 2024-09-05T12:33:34.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b37400640 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b36a00640 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bb5d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2b28098610 tx=0x7f2b28018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d5bf192693c1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b27400640 1 -- 172.21.15.71:0/780787325 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b280b9070 con 0x5650267b4560 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.828+0000 7f2b27400640 1 -- 172.21.15.71:0/780787325 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b280a2260 con 0x5650267b4560 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b27400640 1 -- 172.21.15.71:0/780787325 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b280a4580 con 0x5650267b4560 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 -- 172.21.15.71:0/780787325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 msgr2=0x5650267bb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bb5d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2b28098610 tx=0x7f2b28018420 comp rx=0 tx=0).stop 2024-09-05T12:33:34.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 -- 172.21.15.71:0/780787325 shutdown_connections 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267bf740 0x5650267d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267b4e80 0x5650267d41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 --2- 172.21.15.71:0/780787325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267bb5d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 -- 172.21.15.71:0/780787325 >> 172.21.15.71:0/780787325 conn(0x5650267b0af0 msgr2=0x5650267baec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 -- 172.21.15.71:0/780787325 shutdown_connections 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.829+0000 7f2b40217740 1 -- 172.21.15.71:0/780787325 wait complete. 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.830+0000 7f2b40217740 1 -- start start 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.830+0000 7f2b40217740 1 Processor -- start 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4e80 0x5650267d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267bf740 0x5650267d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381b24d0 con 0x5650267bf740 2024-09-05T12:33:34.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381b1e00 con 0x5650267b4560 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b381aefe0 con 0x5650267b4e80 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b40217740 10 client.0 _ll_register_callbacks cb 0x7f2b28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b2e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4e80 0x5650267d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57160/0 (socket says 172.21.15.71:57160) 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 -- 172.21.15.71:0/3161042729 learned_addr learned my addr 172.21.15.71:0/3161042729 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 -- 172.21.15.71:0/3161042729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4e80 msgr2=0x5650267d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4e80 0x5650267d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 -- 172.21.15.71:0/3161042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267bf740 msgr2=0x5650267d2b20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267bf740 0x5650267d2b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 -- 172.21.15.71:0/3161042729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565026838c30 con 0x5650267b4560 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b36a00640 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267d20a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b280cd410 tx=0x7f2b2809be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f514e1c487ae42b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:34.835 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.831+0000 7f2b24c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b24c00640 1 -- 172.21.15.71:0/3161042729 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b280b9070 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b24c00640 1 -- 172.21.15.71:0/3161042729 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f2b28018b30 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b280a2260 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b24c00640 1 -- 172.21.15.71:0/3161042729 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2b28099f90 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 10 client.26093 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565026838c30 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b24c00640 1 -- 172.21.15.71:0/3161042729 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b280a2260 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b24c00640 1 -- 172.21.15.71:0/3161042729 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f2b280ad020 con 0x5650267b4560 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 10 client.26093 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 10 client.26093 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 20 client.26093 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56502683a1f0 0x56502683c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:34.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 4 client.26093 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:33:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.832+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x565026727c60 con 0x56502683a1f0 2024-09-05T12:33:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.833+0000 7f2b2e000640 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56502683a1f0 0x56502683c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.833+0000 7f2b2e000640 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56502683a1f0 0x56502683c5e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93c2f29bf8825834 server_cookie=6a1a8dec424e7f4c in_seq=0 out_seq=0 2024-09-05T12:33:34.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.833+0000 7f2b24c00640 10 client.26093 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:34.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.903+0000 7f2b24c00640 1 -- 172.21.15.71:0/3161042729 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x565026727c60 con 0x56502683a1f0 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.903+0000 7f2b24c00640 10 client.26093 handle_command_reply: tid=0 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 client.26093 shutdown 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 msgr2=0x5650267d20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267d20a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2b280cd410 tx=0x7f2b2809be10 comp rx=0 tx=0).stop 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 shutdown_connections 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56502683a1f0 0x56502683c5e0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650267bf740 0x5650267d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650267b4e80 0x5650267d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 --2- 172.21.15.71:0/3161042729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650267b4560 0x5650267d20a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 >> 172.21.15.71:0/3161042729 conn(0x5650267b0af0 msgr2=0x5650267afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b24c00640 0 client.26093 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:34.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 shutdown_connections 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.904+0000 7f2b40217740 1 -- 172.21.15.71:0/3161042729 wait complete. 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.905+0000 7f2b40217740 20 client.26093 trim_cache size 0 max 16384 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 36, 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 22, 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:33:34.908 INFO:teuthology.orchestra.run.smithi071.stdout: "17248364": { 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 22, 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:33:34.909 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:33:34.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.905+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2b1c0a2d70 msgr2=0x7f2b1c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.905+0000 7f2b37e00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2b1c0a2d70 0x7f2b1c0a5230 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b30005c40 tx=0x7f2b30009500 comp rx=0 tx=0).stop 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.905+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 msgr2=0x7f2b3819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.905+0000 7f2b37e00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 0x7f2b3819a560 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800e9f0 tx=0x7f2b2800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 shutdown_connections 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b3810bfc0 0x7f2b3819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2b1c0a2d70 0x7f2b1c0a5230 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b381034b0 0x7f2b3819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 --2- 172.21.15.71:0/225648021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b38102b90 0x7f2b3819a560 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 >> 172.21.15.71:0/225648021 conn(0x7f2b380fc610 msgr2=0x7f2b3810d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 shutdown_connections 2024-09-05T12:33:34.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:33:34.906+0000 7f2b37e00640 1 -- 172.21.15.71:0/225648021 wait complete. 2024-09-05T12:33:35.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.97 MB/sec execute 151 sec latency 307.836 ms 2024-09-05T12:33:35.756 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '17248364', quiesce took 4.4 seconds. Will release after: 25.6 2024-09-05T12:33:36.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:35 smithi071 ceph-mon[29845]: pgmap v178: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 921 KiB/s wr, 59 op/s 2024-09-05T12:33:36.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:35 smithi123 ceph-mon[34321]: pgmap v178: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 921 KiB/s wr, 59 op/s 2024-09-05T12:33:36.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.95 MB/sec execute 152 sec latency 1307.945 ms 2024-09-05T12:33:36.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:35 smithi174 ceph-mon[34313]: pgmap v178: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 921 KiB/s wr, 59 op/s 2024-09-05T12:33:37.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.92 MB/sec execute 153 sec latency 2308.086 ms 2024-09-05T12:33:38.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:37 smithi071 ceph-mon[29845]: pgmap v179: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-09-05T12:33:38.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:37 smithi123 ceph-mon[34321]: pgmap v179: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-09-05T12:33:38.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.90 MB/sec execute 154 sec latency 3308.233 ms 2024-09-05T12:33:38.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:37 smithi174 ceph-mon[34313]: pgmap v179: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-09-05T12:33:39.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.87 MB/sec execute 155 sec latency 4308.424 ms 2024-09-05T12:33:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 -- 172.21.15.71:0/714847507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 msgr2=0x7f95f410e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 --2- 172.21.15.71:0/714847507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f410e780 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009cd0 tx=0x7f95ec02a8f0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.025 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 -- 172.21.15.71:0/714847507 shutdown_connections 2024-09-05T12:33:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 --2- 172.21.15.71:0/714847507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f410e780 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 --2- 172.21.15.71:0/714847507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f41018b0 0x7f95f4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 --2- 172.21.15.71:0/714847507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41078c0 0x7f95f4107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 -- 172.21.15.71:0/714847507 >> 172.21.15.71:0/714847507 conn(0x7f95f40fd5d0 msgr2=0x7f95f40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 -- 172.21.15.71:0/714847507 shutdown_connections 2024-09-05T12:33:40.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.024+0000 7f95f3200640 1 -- 172.21.15.71:0/714847507 wait complete. 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.025+0000 7f95f3200640 1 Processor -- start 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.025+0000 7f95f3200640 1 -- start start 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41018b0 0x7f95f40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f41078c0 0x7f95f4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4111d50 con 0x7f95f4102250 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4111bd0 con 0x7f95f41018b0 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f3200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95f4111ed0 con 0x7f95f41078c0 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41018b0 0x7f95f40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52480/0 (socket says 172.21.15.71:52480) 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f1400640 1 -- 172.21.15.71:0/2247563630 learned_addr learned my addr 172.21.15.71:0/2247563630 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.026+0000 7f95f2800640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f41078c0 0x7f95f4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f1400640 1 -- 172.21.15.71:0/2247563630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f41078c0 msgr2=0x7f95f4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f1400640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f41078c0 0x7f95f4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f1400640 1 -- 172.21.15.71:0/2247563630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41018b0 msgr2=0x7f95f40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f1400640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41018b0 0x7f95f40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f1400640 1 -- 172.21.15.71:0/2247563630 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f41a66c0 con 0x7f95f4102250 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f1400640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f4074a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f95e000d770 tx=0x7f95e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95eaa00640 1 -- 172.21.15.71:0/2247563630 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0017070 con 0x7f95f4102250 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95eaa00640 1 -- 172.21.15.71:0/2247563630 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e0005500 con 0x7f95f4102250 2024-09-05T12:33:40.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95eaa00640 1 -- 172.21.15.71:0/2247563630 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0004e00 con 0x7f95f4102250 2024-09-05T12:33:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f41a69b0 con 0x7f95f4102250 2024-09-05T12:33:40.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.027+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f41a6eb0 con 0x7f95f4102250 2024-09-05T12:33:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.030+0000 7f95fb5fd740 1 Processor -- start 2024-09-05T12:33:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.030+0000 7f95fb5fd740 1 -- start start 2024-09-05T12:33:40.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.030+0000 7f95eaa00640 1 -- 172.21.15.71:0/2247563630 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f95e0004fa0 con 0x7f95f4102250 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69254790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c69254cd0 0x561c6925f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c6925f570 0x561c69261960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561c691ba100 con 0x561c692543b0 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561c691621b0 con 0x561c6925f570 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c691b60c0 con 0x561c69254cd0 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95f2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69254790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95f2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69254790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52496/0 (socket says 172.21.15.71:52496) 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95f1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c69254cd0 0x561c6925f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95f2800640 1 -- 172.21.15.71:0/2887001834 learned_addr learned my addr 172.21.15.71:0/2887001834 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.031+0000 7f95f1400640 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c6925f570 0x561c69261960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f1e00640 1 -- 172.21.15.71:0/2887001834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c6925f570 msgr2=0x561c69261960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f1e00640 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c6925f570 0x561c69261960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f1e00640 1 -- 172.21.15.71:0/2887001834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 msgr2=0x561c69254790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f1e00640 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69254790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f1e00640 1 -- 172.21.15.71:0/2887001834 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c68ecd210 con 0x561c69254cd0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f1e00640 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c69254cd0 0x561c6925f030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009c20 tx=0x7f95e4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f45ace6319e01e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95eaa00640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f95d80a2d70 0x7f95d80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95df000640 1 -- 172.21.15.71:0/2887001834 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4004050 con 0x561c69254cd0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95df000640 1 -- 172.21.15.71:0/2887001834 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4005500 con 0x561c69254cd0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95df000640 1 -- 172.21.15.71:0/2887001834 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4004d90 con 0x561c69254cd0 2024-09-05T12:33:40.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95eaa00640 1 -- 172.21.15.71:0/2247563630 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f95e00cf110 con 0x7f95f4102250 2024-09-05T12:33:40.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.032+0000 7f95f2800640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f95d80a2d70 0x7f95d80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 -- 172.21.15.71:0/2887001834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c69254cd0 msgr2=0x561c6925f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c69254cd0 0x561c6925f030 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f95e4009c20 tx=0x7f95e4011810 comp rx=0 tx=0).stop 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 -- 172.21.15.71:0/2887001834 shutdown_connections 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c6925f570 0x561c69261960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c69254cd0 0x561c6925f030 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/2887001834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69254790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.038 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 -- 172.21.15.71:0/2887001834 >> 172.21.15.71:0/2887001834 conn(0x561c69250920 msgr2=0x561c69250d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 -- 172.21.15.71:0/2887001834 shutdown_connections 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95f2800640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f95d80a2d70 0x7f95d80a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009ca0 tx=0x7f95ec005d80 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.033+0000 7f95fb5fd740 1 -- 172.21.15.71:0/2887001834 wait complete. 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 -- start start 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 Processor -- start 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c6925c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c6925ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c6925cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561c69261ff0 con 0x561c69254cd0 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561c69143090 con 0x561c692543b0 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c69262170 con 0x561c6925f570 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95fb5fd740 10 client.0 _ll_register_callbacks cb 0x561c69197f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:40.039 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95f1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c6925ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95f1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c6925ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52506/0 (socket says 172.21.15.71:52506) 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.034+0000 7f95f1400640 1 -- 172.21.15.71:0/1482635309 learned_addr learned my addr 172.21.15.71:0/1482635309 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1e00640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c6925c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f2800640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c6925cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1400640 1 -- 172.21.15.71:0/1482635309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 msgr2=0x561c6925cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1400640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c6925cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1400640 1 -- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 msgr2=0x561c6925c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1400640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c6925c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1400640 1 -- 172.21.15.71:0/1482635309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c68f99620 con 0x561c69254cd0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95f1400640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c6925ca30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f95e009b040 tx=0x7f95e000df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8f9b92f422a332 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95d3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95d3400640 1 -- 172.21.15.71:0/1482635309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0020070 con 0x561c69254cd0 2024-09-05T12:33:40.040 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95d3400640 1 -- 172.21.15.71:0/1482635309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f95e00d2cd0 con 0x561c69254cd0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e4005500 con 0x561c69254cd0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.035+0000 7f95d3400640 1 -- 172.21.15.71:0/1482635309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f95e009aca0 con 0x561c69254cd0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 10 client.16077 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561c6926ddc0 con 0x561c69254cd0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95d3400640 1 -- 172.21.15.71:0/1482635309 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4005500 con 0x561c69254cd0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95d3400640 1 -- 172.21.15.71:0/1482635309 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f95e00979b0 con 0x561c69254cd0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 10 client.16077 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 20 client.16077 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c6926f230 0x561c69271620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 4 client.16077 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.036+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c691c7830 con 0x561c6926f230 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.037+0000 7f95f1e00640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c6926f230 0x561c69271620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.037+0000 7f95f1e00640 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c6926f230 0x561c69271620 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34aa31b3d0a2335a server_cookie=3d8838dafe7942b8 in_seq=0 out_seq=0 2024-09-05T12:33:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.037+0000 7f95d3400640 10 client.16077 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95d3400640 1 -- 172.21.15.71:0/1482635309 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c691c7830 con 0x561c6926f230 2024-09-05T12:33:40.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95d3400640 10 client.16077 handle_command_reply: tid=0 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 client.16077 shutdown 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 msgr2=0x561c6925ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c6925ca30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f95e009b040 tx=0x7f95e000df40 comp rx=0 tx=0).stop 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 shutdown_connections 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c6926f230 0x561c69271620 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c6925cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c6925ca30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/1482635309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c6925c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95d3400640 0 client.16077 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 >> 172.21.15.71:0/1482635309 conn(0x561c69250920 msgr2=0x561c69256680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 shutdown_connections 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 1 -- 172.21.15.71:0/1482635309 wait complete. 2024-09-05T12:33:40.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.038+0000 7f95fb5fd740 20 client.16077 trim_cache size 0 max 16384 2024-09-05T12:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.053+0000 7f95fb5fd740 1 Processor -- start 2024-09-05T12:33:40.055 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.053+0000 7f95fb5fd740 1 -- start start 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.053+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69276f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.053+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c69254cd0 0x561c69277450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c69255e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41b0e90 con 0x561c692543b0 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41b2a30 con 0x561c69254cd0 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41b1560 con 0x561c6925f570 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69276f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69276f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52520/0 (socket says 172.21.15.71:52520) 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 -- 172.21.15.71:0/3958815250 learned_addr learned my addr 172.21.15.71:0/3958815250 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1400640 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c69254cd0 0x561c69277450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f2800640 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c69255e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 -- 172.21.15.71:0/3958815250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 msgr2=0x561c69255e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.056 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c69255e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 -- 172.21.15.71:0/3958815250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c69254cd0 msgr2=0x561c69277450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c69254cd0 0x561c69277450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 -- 172.21.15.71:0/3958815250 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c692515b0 con 0x561c692543b0 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.054+0000 7f95f1e00640 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69276f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f95e4014040 tx=0x7f95e4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951104414c1b23e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95de600640 1 -- 172.21.15.71:0/3958815250 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4034070 con 0x561c692543b0 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95de600640 1 -- 172.21.15.71:0/3958815250 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4005a70 con 0x561c692543b0 2024-09-05T12:33:40.057 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95de600640 1 -- 172.21.15.71:0/3958815250 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e4019940 con 0x561c692543b0 2024-09-05T12:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 -- 172.21.15.71:0/3958815250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 msgr2=0x561c69276f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69276f10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f95e4014040 tx=0x7f95e4004610 comp rx=0 tx=0).stop 2024-09-05T12:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 -- 172.21.15.71:0/3958815250 shutdown_connections 2024-09-05T12:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c69255e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c69254cd0 0x561c69277450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/3958815250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c692543b0 0x561c69276f10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 -- 172.21.15.71:0/3958815250 >> 172.21.15.71:0/3958815250 conn(0x561c69250920 msgr2=0x561c692577b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 -- 172.21.15.71:0/3958815250 shutdown_connections 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.055+0000 7f95fb5fd740 1 -- 172.21.15.71:0/3958815250 wait complete. 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 -- start start 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 Processor -- start 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c69267990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c69262cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c692631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41b2360 con 0x561c69254cd0 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41b1c90 con 0x561c692543b0 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95f41aee70 con 0x561c6925f570 2024-09-05T12:33:40.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.056+0000 7f95fb5fd740 10 client.0 _ll_register_callbacks cb 0x7f95e000d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c69262cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f2800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c692631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c69267990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c69262cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52536/0 (socket says 172.21.15.71:52536) 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 -- 172.21.15.71:0/949227379 learned_addr learned my addr 172.21.15.71:0/949227379 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 -- 172.21.15.71:0/949227379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 msgr2=0x561c692631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c692631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 -- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 msgr2=0x561c69267990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c69267990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 -- 172.21.15.71:0/949227379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c69263760 con 0x561c69254cd0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95f1400640 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c69262cb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x561c6925bd60 tx=0x7f95e00d2bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecea5e3769f967b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95d3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:40.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.057+0000 7f95d3400640 1 -- 172.21.15.71:0/949227379 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0020040 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95d3400640 1 -- 172.21.15.71:0/949227379 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f95e00d2850 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e4005a70 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95d3400640 1 -- 172.21.15.71:0/949227379 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f95e009b110 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 10 client.16083 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561c692d9c50 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95d3400640 1 -- 172.21.15.71:0/949227379 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95e4005a70 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95d3400640 1 -- 172.21.15.71:0/949227379 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7f95e00bb0b0 con 0x561c69254cd0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 10 client.16083 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 20 client.16083 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c692da3d0 0x561c692dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 4 client.16083 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.058+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c691c1b80 con 0x561c692da3d0 2024-09-05T12:33:40.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.059+0000 7f95f1e00640 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c692da3d0 0x561c692dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.059+0000 7f95f1e00640 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c692da3d0 0x561c692dc7f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792fcc69fd6d1931 server_cookie=826e78f5bbd6e8cc in_seq=0 out_seq=0 2024-09-05T12:33:40.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.059+0000 7f95d3400640 10 client.16083 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95d3400640 1 -- 172.21.15.71:0/949227379 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c691c1b80 con 0x561c692da3d0 2024-09-05T12:33:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95d3400640 10 client.16083 handle_command_reply: tid=0 2024-09-05T12:33:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 client.16083 shutdown 2024-09-05T12:33:40.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 msgr2=0x561c69262cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c69262cb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x561c6925bd60 tx=0x7f95e00d2bc0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 shutdown_connections 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561c692da3d0 0x561c692dc7f0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c6925f570 0x561c692631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561c69254cd0 0x561c69262cb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 --2- 172.21.15.71:0/949227379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c692543b0 0x561c69267990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95d3400640 0 client.16083 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 >> 172.21.15.71:0/949227379 conn(0x561c69250920 msgr2=0x561c6925db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 shutdown_connections 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 1 -- 172.21.15.71:0/949227379 wait complete. 2024-09-05T12:33:40.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.098+0000 7f95fb5fd740 20 client.16083 trim_cache size 0 max 16384 2024-09-05T12:33:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:33:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:33:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:33:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:33:40.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f95d80a2d70 msgr2=0x7f95d80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f95d80a2d70 0x7f95d80a5230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f95ec009ca0 tx=0x7f95ec005d80 comp rx=0 tx=0).stop 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 msgr2=0x7f95f4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f4074a30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f95e000d770 tx=0x7f95e000dc30 comp rx=0 tx=0).stop 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 shutdown_connections 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f41078c0 0x7f95f4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f95d80a2d70 0x7f95d80a5230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95f4102250 0x7f95f4074a30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 --2- 172.21.15.71:0/2247563630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f41018b0 0x7f95f40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 >> 172.21.15.71:0/2247563630 conn(0x7f95f40fd5d0 msgr2=0x7f95f40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 shutdown_connections 2024-09-05T12:33:40.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:40.099+0000 7f95f3200640 1 -- 172.21.15.71:0/2247563630 wait complete. 2024-09-05T12:33:40.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:39 smithi071 ceph-mon[29845]: pgmap v180: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 902 KiB/s wr, 138 op/s 2024-09-05T12:33:40.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:39 smithi123 ceph-mon[34321]: pgmap v180: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 902 KiB/s wr, 138 op/s 2024-09-05T12:33:40.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.85 MB/sec execute 156 sec latency 5308.471 ms 2024-09-05T12:33:40.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:39 smithi174 ceph-mon[34313]: pgmap v180: 129 pgs: 129 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 902 KiB/s wr, 138 op/s 2024-09-05T12:33:41.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.82 MB/sec execute 157 sec latency 6308.514 ms 2024-09-05T12:33:42.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:41 smithi071 ceph-mon[29845]: pgmap v181: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 934 KiB/s wr, 158 op/s 2024-09-05T12:33:42.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:41 smithi123 ceph-mon[34321]: pgmap v181: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 934 KiB/s wr, 158 op/s 2024-09-05T12:33:42.214 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.80 MB/sec execute 158 sec latency 7308.630 ms 2024-09-05T12:33:42.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:41 smithi174 ceph-mon[34313]: pgmap v181: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 934 KiB/s wr, 158 op/s 2024-09-05T12:33:43.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.77 MB/sec execute 159 sec latency 8308.786 ms 2024-09-05T12:33:44.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:43 smithi071 ceph-mon[29845]: pgmap v182: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 520 KiB/s wr, 135 op/s 2024-09-05T12:33:44.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:43 smithi123 ceph-mon[34321]: pgmap v182: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 520 KiB/s wr, 135 op/s 2024-09-05T12:33:44.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.75 MB/sec execute 160 sec latency 9308.963 ms 2024-09-05T12:33:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:43 smithi174 ceph-mon[34313]: pgmap v182: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 520 KiB/s wr, 135 op/s 2024-09-05T12:33:45.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.73 MB/sec execute 161 sec latency 10309.070 ms 2024-09-05T12:33:46.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:45 smithi071 ceph-mon[29845]: pgmap v183: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 508 KiB/s wr, 130 op/s 2024-09-05T12:33:46.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:45 smithi123 ceph-mon[34321]: pgmap v183: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 508 KiB/s wr, 130 op/s 2024-09-05T12:33:46.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.70 MB/sec execute 162 sec latency 11309.182 ms 2024-09-05T12:33:46.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:45 smithi174 ceph-mon[34313]: pgmap v183: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 508 KiB/s wr, 130 op/s 2024-09-05T12:33:47.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.68 MB/sec execute 163 sec latency 12309.310 ms 2024-09-05T12:33:48.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:47 smithi071 ceph-mon[29845]: pgmap v184: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 514 KiB/s wr, 135 op/s 2024-09-05T12:33:48.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:47 smithi123 ceph-mon[34321]: pgmap v184: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 514 KiB/s wr, 135 op/s 2024-09-05T12:33:48.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.66 MB/sec execute 164 sec latency 13309.436 ms 2024-09-05T12:33:48.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:47 smithi174 ceph-mon[34313]: pgmap v184: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 514 KiB/s wr, 135 op/s 2024-09-05T12:33:49.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.64 MB/sec execute 165 sec latency 14309.555 ms 2024-09-05T12:33:50.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:49 smithi071 ceph-mon[29845]: pgmap v185: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-09-05T12:33:50.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:49 smithi123 ceph-mon[34321]: pgmap v185: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-09-05T12:33:50.215 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.62 MB/sec execute 166 sec latency 15309.664 ms 2024-09-05T12:33:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:49 smithi174 ceph-mon[34313]: pgmap v185: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 41 KiB/s wr, 29 op/s 2024-09-05T12:33:51.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.59 MB/sec execute 167 sec latency 16309.710 ms 2024-09-05T12:33:52.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:51 smithi071 ceph-mon[29845]: pgmap v186: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 262 KiB/s wr, 40 op/s 2024-09-05T12:33:52.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:51 smithi123 ceph-mon[34321]: pgmap v186: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 262 KiB/s wr, 40 op/s 2024-09-05T12:33:52.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.57 MB/sec execute 168 sec latency 17309.764 ms 2024-09-05T12:33:52.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:51 smithi174 ceph-mon[34313]: pgmap v186: 129 pgs: 129 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 262 KiB/s wr, 40 op/s 2024-09-05T12:33:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 -- 172.21.15.71:0/716678882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 msgr2=0x7fbed4107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 --2- 172.21.15.71:0/716678882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4107ca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009900 tx=0x7fbec402a400 comp rx=0 tx=0).stop 2024-09-05T12:33:53.215 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 -- 172.21.15.71:0/716678882 shutdown_connections 2024-09-05T12:33:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 --2- 172.21.15.71:0/716678882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed4102250 0x7fbed410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 --2- 172.21.15.71:0/716678882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed41018b0 0x7fbed4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 --2- 172.21.15.71:0/716678882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4107ca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 -- 172.21.15.71:0/716678882 >> 172.21.15.71:0/716678882 conn(0x7fbed40fd5d0 msgr2=0x7fbed40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 -- 172.21.15.71:0/716678882 shutdown_connections 2024-09-05T12:33:53.216 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.214+0000 7fbed3200640 1 -- 172.21.15.71:0/716678882 wait complete. 2024-09-05T12:33:53.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.55 MB/sec execute 169 sec latency 18309.818 ms 2024-09-05T12:33:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.215+0000 7fbed3200640 1 Processor -- start 2024-09-05T12:33:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.215+0000 7fbed3200640 1 -- start start 2024-09-05T12:33:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed41018b0 0x7fbed40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.218 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed4102250 0x7fbed4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4111d50 con 0x7fbed41018b0 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4111bd0 con 0x7fbed41078c0 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed3200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbed4111ed0 con 0x7fbed4102250 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed41018b0 0x7fbed40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:39592/0 (socket says 172.21.15.71:39592) 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 -- 172.21.15.71:0/70384973 learned_addr learned my addr 172.21.15.71:0/70384973 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed1400640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed4102250 0x7fbed4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 -- 172.21.15.71:0/70384973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed4102250 msgr2=0x7fbed4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed4102250 0x7fbed4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 -- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed41018b0 msgr2=0x7fbed40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed41018b0 0x7fbed40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.216+0000 7fbed2800640 1 -- 172.21.15.71:0/70384973 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed41a6720 con 0x7fbed41078c0 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.217+0000 7fbed2800640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4074f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00eaa0 tx=0x7fbecc00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.217+0000 7fbecaa00640 1 -- 172.21.15.71:0/70384973 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc00cdb0 con 0x7fbed41078c0 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.217+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed41a6a10 con 0x7fbed41078c0 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.217+0000 7fbecaa00640 1 -- 172.21.15.71:0/70384973 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbecc005020 con 0x7fbed41078c0 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.217+0000 7fbecaa00640 1 -- 172.21.15.71:0/70384973 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc0116b0 con 0x7fbed41078c0 2024-09-05T12:33:53.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.217+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed41a6f60 con 0x7fbed41078c0 2024-09-05T12:33:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 Processor -- start 2024-09-05T12:33:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbecaa00640 1 -- 172.21.15.71:0/70384973 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbecc005560 con 0x7fbed41078c0 2024-09-05T12:33:53.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 -- start start 2024-09-05T12:33:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128f2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 0x5630128fd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563012858100 con 0x5630128fd570 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630128001b0 con 0x5630128f2cd0 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.220+0000 7fbedb848740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5630128540c0 con 0x5630128f23b0 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 0x5630128fd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 0x5630128fd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:39594/0 (socket says 172.21.15.71:39594) 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 -- 172.21.15.71:0/3268820380 learned_addr learned my addr 172.21.15.71:0/3268820380 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1400640 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 -- 172.21.15.71:0/3268820380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 msgr2=0x5630128f2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128f2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 -- 172.21.15.71:0/3268820380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 msgr2=0x5630128ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbed1e00640 1 -- 172.21.15.71:0/3268820380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56301256b210 con 0x5630128f2cd0 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.221+0000 7fbecaa00640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbeb80a2cf0 0x7fbeb80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbed1e00640 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 0x5630128fd030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009230 tx=0x7fbec4004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3b3c94cd1fe1150 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbecaa00640 1 -- 172.21.15.71:0/70384973 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbecc0cf920 con 0x7fbed41078c0 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbed1400640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbeb80a2cf0 0x7fbeb80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbebf000640 1 -- 172.21.15.71:0/3268820380 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4035040 con 0x5630128f2cd0 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbebf000640 1 -- 172.21.15.71:0/3268820380 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec4005520 con 0x5630128f2cd0 2024-09-05T12:33:53.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbebf000640 1 -- 172.21.15.71:0/3268820380 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec40035f0 con 0x5630128f2cd0 2024-09-05T12:33:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbedb848740 1 -- 172.21.15.71:0/3268820380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 msgr2=0x5630128fd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbedb848740 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 0x5630128fd030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009230 tx=0x7fbec4004040 comp rx=0 tx=0).stop 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.222+0000 7fbed1400640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbeb80a2cf0 0x7fbeb80a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5630128efca0 tx=0x7fbec00096c0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 -- 172.21.15.71:0/3268820380 shutdown_connections 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f2cd0 0x5630128fd030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 --2- 172.21.15.71:0/3268820380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128f2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 -- 172.21.15.71:0/3268820380 >> 172.21.15.71:0/3268820380 conn(0x5630128ee920 msgr2=0x5630128eed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 -- 172.21.15.71:0/3268820380 shutdown_connections 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 -- 172.21.15.71:0/3268820380 wait complete. 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 -- start start 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.223+0000 7fbedb848740 1 Processor -- start 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x5630128fa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f2cd0 0x5630128faae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5630128ffff0 con 0x5630128fd570 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5630127e1090 con 0x5630128f23b0 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563012900170 con 0x5630128f2cd0 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbedb848740 10 client.0 _ll_register_callbacks cb 0x563012835f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x5630128fa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f2cd0 0x5630128faae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed2800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x5630128fa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:39600/0 (socket says 172.21.15.71:39600) 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f2cd0 0x5630128faae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36298/0 (socket says 172.21.15.71:36298) 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 -- 172.21.15.71:0/1871613028 learned_addr learned my addr 172.21.15.71:0/1871613028 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 -- 172.21.15.71:0/1871613028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f2cd0 msgr2=0x5630128faae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f2cd0 0x5630128faae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 -- 172.21.15.71:0/1871613028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 msgr2=0x5630128fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.224+0000 7fbed1e00640 1 -- 172.21.15.71:0/1871613028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563012637620 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbed1e00640 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x5630128fa5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbec4005c40 tx=0x7fbec4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfaca1b2fea8ccaa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbeb3400640 1 -- 172.21.15.71:0/1871613028 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4053060 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec4005520 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbeb3400640 1 -- 172.21.15.71:0/1871613028 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbec4004cd0 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbeb3400640 1 -- 172.21.15.71:0/1871613028 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbec4035050 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbedb848740 10 client.26123 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56301290bda0 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.225+0000 7fbeb3400640 1 -- 172.21.15.71:0/1871613028 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec4005520 con 0x5630128f23b0 2024-09-05T12:33:53.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbeb3400640 1 -- 172.21.15.71:0/1871613028 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fbec40039f0 con 0x5630128f23b0 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbedb848740 10 client.26123 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbedb848740 10 client.26123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbedb848740 20 client.26123 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbedb848740 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56301290d0f0 0x56301290f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbedb848740 4 client.26123 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563012865830 con 0x56301290d0f0 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.226+0000 7fbed1400640 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56301290d0f0 0x56301290f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.227+0000 7fbed1400640 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56301290d0f0 0x56301290f4e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22314e151d35f771 server_cookie=4578dbdc6d72b119 in_seq=0 out_seq=0 2024-09-05T12:33:53.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.227+0000 7fbeb3400640 10 client.26123 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.227+0000 7fbeb3400640 1 -- 172.21.15.71:0/1871613028 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563012865830 con 0x56301290d0f0 2024-09-05T12:33:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.227+0000 7fbeb3400640 10 client.26123 handle_command_reply: tid=0 2024-09-05T12:33:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.227+0000 7fbedb848740 1 client.26123 shutdown 2024-09-05T12:33:53.232 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 msgr2=0x5630128fa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x5630128fa5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbec4005c40 tx=0x7fbec4009260 comp rx=0 tx=0).stop 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 shutdown_connections 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56301290d0f0 0x56301290f4e0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128fd570 0x5630128fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbeb3400640 0 client.26123 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f2cd0 0x5630128faae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 --2- 172.21.15.71:0/1871613028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x5630128fa5a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 >> 172.21.15.71:0/1871613028 conn(0x5630128ee920 msgr2=0x5630128f4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 shutdown_connections 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 1 -- 172.21.15.71:0/1871613028 wait complete. 2024-09-05T12:33:53.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.228+0000 7fbedb848740 20 client.26123 trim_cache size 0 max 16384 2024-09-05T12:33:53.248 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 Processor -- start 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 -- start start 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128e98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012914ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128fd570 0x563012914ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41b0ec0 con 0x5630128f2cd0 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41b2a60 con 0x5630128fd570 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.243+0000 7fbedb848740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41b1590 con 0x5630128f23b0 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012914ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012914ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40956/0 (socket says 172.21.15.71:40956) 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 -- 172.21.15.71:0/2090454208 learned_addr learned my addr 172.21.15.71:0/2090454208 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1e00640 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128e98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.249 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed2800640 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128fd570 0x563012914ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 -- 172.21.15.71:0/2090454208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 msgr2=0x5630128e98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128e98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 -- 172.21.15.71:0/2090454208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128fd570 msgr2=0x563012914ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128fd570 0x563012914ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 -- 172.21.15.71:0/2090454208 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630128f9530 con 0x5630128f2cd0 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.244+0000 7fbed1400640 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012914ab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1a7b8360debe4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbebe600640 1 -- 172.21.15.71:0/2090454208 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003d40 con 0x5630128f2cd0 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbebe600640 1 -- 172.21.15.71:0/2090454208 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec002c910 con 0x5630128f2cd0 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbebe600640 1 -- 172.21.15.71:0/2090454208 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0020810 con 0x5630128f2cd0 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 -- 172.21.15.71:0/2090454208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 msgr2=0x563012914ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012914ab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009fd0 tx=0x7fbec00046c0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 -- 172.21.15.71:0/2090454208 shutdown_connections 2024-09-05T12:33:53.250 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128fd570 0x563012914ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012914ab0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 --2- 172.21.15.71:0/2090454208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128f23b0 0x5630128e98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 -- 172.21.15.71:0/2090454208 >> 172.21.15.71:0/2090454208 conn(0x5630128ee920 msgr2=0x5630128fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 -- 172.21.15.71:0/2090454208 shutdown_connections 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.245+0000 7fbedb848740 1 -- 172.21.15.71:0/2090454208 wait complete. 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 -- start start 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 Processor -- start 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x563012901bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012902130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128fd570 0x563012905250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41b2390 con 0x5630128f2cd0 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41b1cc0 con 0x5630128f23b0 2024-09-05T12:33:53.251 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbed41aeea0 con 0x5630128fd570 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.246+0000 7fbedb848740 10 client.0 _ll_register_callbacks cb 0x7fbec0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012902130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012902130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40960/0 (socket says 172.21.15.71:40960) 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 -- 172.21.15.71:0/1795663715 learned_addr learned my addr 172.21.15.71:0/1795663715 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed2800640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128fd570 0x563012905250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1e00640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x563012901bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 -- 172.21.15.71:0/1795663715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128fd570 msgr2=0x563012905250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128fd570 0x563012905250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 -- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 msgr2=0x563012901bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x563012901bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 -- 172.21.15.71:0/1795663715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630128f2b00 con 0x5630128f2cd0 2024-09-05T12:33:53.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbed1400640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012902130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009730 tx=0x7fbec0009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9eefc5c2a3b6637 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbeb3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.247+0000 7fbeb3400640 1 -- 172.21.15.71:0/1795663715 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0003e00 con 0x5630128f2cd0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbeb3400640 1 -- 172.21.15.71:0/1795663715 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fbec002cac0 con 0x5630128f2cd0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec002c910 con 0x5630128f2cd0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbeb3400640 1 -- 172.21.15.71:0/1795663715 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbec002ccc0 con 0x5630128f2cd0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 10 client.16101 fetch_fsmap learned FSMap version 36 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5630129783b0 con 0x5630128f2cd0 2024-09-05T12:33:53.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbeb3400640 1 -- 172.21.15.71:0/1795663715 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbec002c910 con 0x5630128f2cd0 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbeb3400640 1 -- 172.21.15.71:0/1795663715 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 36) ==== 8378+0+0 (secure 0 0 0) 0x7fbec001d070 con 0x5630128f2cd0 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 10 client.16101 fetch_fsmap finished waiting for FSMap version 36 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 20 client.16101 populate_metadata read hostname 'smithi071' 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563012978a60 0x56301297ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 4 client.16101 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:33:53.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.248+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56301285fb80 con 0x563012978a60 2024-09-05T12:33:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.249+0000 7fbed1e00640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563012978a60 0x56301297ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:33:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.249+0000 7fbed1e00640 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563012978a60 0x56301297ae80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac649f6052a7316b server_cookie=27b7b31afbc142d8 in_seq=0 out_seq=0 2024-09-05T12:33:53.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.249+0000 7fbeb3400640 10 client.16101 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbeb3400640 1 -- 172.21.15.71:0/1795663715 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56301285fb80 con 0x563012978a60 2024-09-05T12:33:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbeb3400640 10 client.16101 handle_command_reply: tid=0 2024-09-05T12:33:53.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 client.16101 shutdown 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 msgr2=0x563012902130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012902130 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009730 tx=0x7fbec0009ac0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 shutdown_connections 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563012978a60 0x56301297ae80 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5630128fd570 0x563012905250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5630128f2cd0 0x563012902130 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 --2- 172.21.15.71:0/1795663715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5630128f23b0 0x563012901bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 >> 172.21.15.71:0/1795663715 conn(0x5630128ee920 msgr2=0x5630128fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbeb3400640 0 client.16101 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 shutdown_connections 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 1 -- 172.21.15.71:0/1795663715 wait complete. 2024-09-05T12:33:53.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.274+0000 7fbedb848740 20 client.16101 trim_cache size 0 max 16384 2024-09-05T12:33:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:33:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:33:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:33:53.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbeb80a2cf0 msgr2=0x7fbeb80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbeb80a2cf0 0x7fbeb80a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5630128efca0 tx=0x7fbec00096c0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 msgr2=0x7fbed4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4074f70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbecc00eaa0 tx=0x7fbecc00ef60 comp rx=0 tx=0).stop 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 shutdown_connections 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbed41078c0 0x7fbed4074f70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed4102250 0x7fbed4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbeb80a2cf0 0x7fbeb80a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 --2- 172.21.15.71:0/70384973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed41018b0 0x7fbed40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 >> 172.21.15.71:0/70384973 conn(0x7fbed40fd5d0 msgr2=0x7fbed40fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 shutdown_connections 2024-09-05T12:33:53.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:33:53.275+0000 7fbed3200640 1 -- 172.21.15.71:0/70384973 wait complete. 2024-09-05T12:33:54.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:53 smithi071 ceph-mon[29845]: pgmap v187: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 361 KiB/s wr, 22 op/s 2024-09-05T12:33:54.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:53 smithi123 ceph-mon[34321]: pgmap v187: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 361 KiB/s wr, 22 op/s 2024-09-05T12:33:54.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.53 MB/sec execute 170 sec latency 19309.930 ms 2024-09-05T12:33:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:53 smithi174 ceph-mon[34313]: pgmap v187: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 361 KiB/s wr, 22 op/s 2024-09-05T12:33:55.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.51 MB/sec execute 171 sec latency 20310.074 ms 2024-09-05T12:33:56.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:55 smithi071 ceph-mon[29845]: pgmap v188: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 357 KiB/s wr, 18 op/s 2024-09-05T12:33:56.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:55 smithi123 ceph-mon[34321]: pgmap v188: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 357 KiB/s wr, 18 op/s 2024-09-05T12:33:56.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.49 MB/sec execute 172 sec latency 21310.218 ms 2024-09-05T12:33:56.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:55 smithi174 ceph-mon[34313]: pgmap v188: 129 pgs: 129 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 357 KiB/s wr, 18 op/s 2024-09-05T12:33:57.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:56 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:33:57.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:56 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:33:57.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.47 MB/sec execute 173 sec latency 22310.360 ms 2024-09-05T12:33:57.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:56 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:33:58.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:57 smithi071 ceph-mon[29845]: pgmap v189: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 395 KiB/s wr, 39 op/s 2024-09-05T12:33:58.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:57 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:33:58.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:57 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:33:58.164 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:57 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:33:58.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:57 smithi123 ceph-mon[34321]: pgmap v189: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 395 KiB/s wr, 39 op/s 2024-09-05T12:33:58.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:57 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:33:58.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:57 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:33:58.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:57 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:33:58.216 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.45 MB/sec execute 174 sec latency 23310.539 ms 2024-09-05T12:33:58.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:57 smithi174 ceph-mon[34313]: pgmap v189: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 395 KiB/s wr, 39 op/s 2024-09-05T12:33:58.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:57 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:33:58.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:57 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:33:58.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:57 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:33:59.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.43 MB/sec execute 175 sec latency 24310.661 ms 2024-09-05T12:34:00.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:33:59 smithi071 ceph-mon[29845]: pgmap v190: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 389 KiB/s wr, 34 op/s 2024-09-05T12:34:00.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:33:59 smithi123 ceph-mon[34321]: pgmap v190: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 389 KiB/s wr, 34 op/s 2024-09-05T12:34:00.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.41 MB/sec execute 176 sec latency 25310.775 ms 2024-09-05T12:34:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:33:59 smithi174 ceph-mon[34313]: pgmap v190: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 389 KiB/s wr, 34 op/s 2024-09-05T12:34:01.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.39 MB/sec execute 177 sec latency 26310.911 ms 2024-09-05T12:34:01.358 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:34:01.736 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:34:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:01 smithi071 ceph-mon[29845]: pgmap v191: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 425 KiB/s wr, 41 op/s 2024-09-05T12:34:02.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:01 smithi123 ceph-mon[34321]: pgmap v191: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 425 KiB/s wr, 41 op/s 2024-09-05T12:34:02.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.37 MB/sec execute 178 sec latency 27310.975 ms 2024-09-05T12:34:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:01 smithi174 ceph-mon[34313]: pgmap v191: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 425 KiB/s wr, 41 op/s 2024-09-05T12:34:02.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.366+0000 7f5b8de00640 1 -- 172.21.15.71:0/4204439436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 msgr2=0x7f5b881043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:02.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.366+0000 7f5b8de00640 1 --2- 172.21.15.71:0/4204439436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b881043d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009900 tx=0x7f5b7802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 -- 172.21.15.71:0/4204439436 shutdown_connections 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 --2- 172.21.15.71:0/4204439436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b88109120 0x7f5b88110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 --2- 172.21.15.71:0/4204439436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b881049a0 0x7f5b881089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 --2- 172.21.15.71:0/4204439436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b881043d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 -- 172.21.15.71:0/4204439436 >> 172.21.15.71:0/4204439436 conn(0x7f5b880ffa90 msgr2=0x7f5b88101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 -- 172.21.15.71:0/4204439436 shutdown_connections 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.367+0000 7f5b8de00640 1 -- 172.21.15.71:0/4204439436 wait complete. 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 Processor -- start 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 -- start start 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b8819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b881049a0 0x7f5b8819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:02.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b88109120 0x7f5b8819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b88114170 con 0x7f5b88103ff0 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b88113ff0 con 0x7f5b881049a0 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b8de00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b881142f0 con 0x7f5b88109120 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b86a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b8819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b86a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b8819a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40992/0 (socket says 172.21.15.71:40992) 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b86a00640 1 -- 172.21.15.71:0/1429928829 learned_addr learned my addr 172.21.15.71:0/1429928829 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.368+0000 7f5b7fe00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b881049a0 0x7f5b8819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b87400640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b88109120 0x7f5b8819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b86a00640 1 -- 172.21.15.71:0/1429928829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b88109120 msgr2=0x7f5b8819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b86a00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b88109120 0x7f5b8819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b86a00640 1 -- 172.21.15.71:0/1429928829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b881049a0 msgr2=0x7f5b8819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b86a00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b881049a0 0x7f5b8819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b86a00640 1 -- 172.21.15.71:0/1429928829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8819f5c0 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b86a00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b8819a610 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b78004770 tx=0x7f5b78003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7803f070 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b78031ca0 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b8819f850 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.369+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78035030 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.370+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b881a7190 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.371+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b881054d0 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.372+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5b78042070 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.374+0000 7f5b8ca00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b6c0a2cf0 0x7f5b6c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.374+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f5b780efdb0 con 0x7f5b88103ff0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.374+0000 7f5b7fe00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b6c0a2cf0 0x7f5b6c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:02.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.375+0000 7f5b7fe00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b6c0a2cf0 0x7f5b6c0a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5b880fff20 tx=0x7f5b70005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:02.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.378+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b780f4070 con 0x7f5b88103ff0 2024-09-05T12:34:02.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.589+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b8810c300 con 0x7f5b88103ff0 2024-09-05T12:34:02.591 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.590+0000 7f5b8ca00640 1 -- 172.21.15.71:0/1429928829 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11969 (secure 0 0 0) 0x7f5b780bc630 con 0x7f5b88103ff0 2024-09-05T12:34:02.592 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:34:02.592 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":37,"btime":"2024-09-05T12:34:01:910782+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:34:01.825966+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:34:02.592 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 37 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b6c0a2cf0 msgr2=0x7f5b6c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b6c0a2cf0 0x7f5b6c0a51b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5b880fff20 tx=0x7f5b70005f90 comp rx=0 tx=0).stop 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 msgr2=0x7f5b8819a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b8819a610 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5b78004770 tx=0x7f5b78003c10 comp rx=0 tx=0).stop 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 shutdown_connections 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b88109120 0x7f5b8819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b881049a0 0x7f5b8819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b6c0a2cf0 0x7f5b6c0a51b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 --2- 172.21.15.71:0/1429928829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b88103ff0 0x7f5b8819a610 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:02.595 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 >> 172.21.15.71:0/1429928829 conn(0x7f5b880ffa90 msgr2=0x7f5b88100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.594+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 shutdown_connections 2024-09-05T12:34:02.596 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:02.595+0000 7f5b8de00640 1 -- 172.21.15.71:0/1429928829 wait complete. 2024-09-05T12:34:03.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:02 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:03.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:02 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1429928829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:34:03.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.35 MB/sec execute 179 sec latency 28311.074 ms 2024-09-05T12:34:03.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:02 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:03.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:02 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1429928829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:34:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:02 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:02 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1429928829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:34:03.501 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 17248364 --release --await' 2024-09-05T12:34:03.501 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 17248364 --release --await 2024-09-05T12:34:03.869 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:34:04.190 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:03 smithi071 ceph-mon[29845]: pgmap v192: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 227 KiB/s wr, 45 op/s 2024-09-05T12:34:04.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 207913 3.33 MB/sec execute 180 sec latency 29311.230 ms 2024-09-05T12:34:04.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:03 smithi174 ceph-mon[34313]: pgmap v192: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 227 KiB/s wr, 45 op/s 2024-09-05T12:34:04.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:03 smithi123 ceph-mon[34321]: pgmap v192: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 227 KiB/s wr, 45 op/s 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 -- 172.21.15.71:0/1501685141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96441034b0 msgr2=0x7f964410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 --2- 172.21.15.71:0/1501685141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96441034b0 0x7f964410b9a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9630009860 tx=0x7f963002a3d0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 -- 172.21.15.71:0/1501685141 shutdown_connections 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 --2- 172.21.15.71:0/1501685141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410bfc0 0x7f964410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 --2- 172.21.15.71:0/1501685141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96441034b0 0x7f964410b9a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 --2- 172.21.15.71:0/1501685141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f9644102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 -- 172.21.15.71:0/1501685141 >> 172.21.15.71:0/1501685141 conn(0x7f96440fc610 msgr2=0x7f96440fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.501+0000 7f964a200640 1 -- 172.21.15.71:0/1501685141 shutdown_connections 2024-09-05T12:34:04.503 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.502+0000 7f964a200640 1 -- 172.21.15.71:0/1501685141 wait complete. 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 Processor -- start 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 -- start start 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f96441007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96441034b0 0x7f9644100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f964410bfc0 0x7f9644101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9644111dc0 con 0x7f9644102b90 2024-09-05T12:34:04.504 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9644111c40 con 0x7f96441034b0 2024-09-05T12:34:04.505 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.503+0000 7f964a200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9644111f40 con 0x7f964410bfc0 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f96441007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f96441007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41794/0 (socket says 172.21.15.71:41794) 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 -- 172.21.15.71:0/573736701 learned_addr learned my addr 172.21.15.71:0/573736701 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9643e00640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96441034b0 0x7f9644100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9649800640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f964410bfc0 0x7f9644101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 -- 172.21.15.71:0/573736701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f964410bfc0 msgr2=0x7f9644101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f964410bfc0 0x7f9644101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 -- 172.21.15.71:0/573736701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96441034b0 msgr2=0x7f9644100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96441034b0 0x7f9644100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 -- 172.21.15.71:0/573736701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a67c0 con 0x7f9644102b90 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.504+0000 7f9648e00640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f96441007f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9c0 tx=0x7f963800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:04.506 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.505+0000 7f9641600640 1 -- 172.21.15.71:0/573736701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800cde0 con 0x7f9644102b90 2024-09-05T12:34:04.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.505+0000 7f9641600640 1 -- 172.21.15.71:0/573736701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96380035f0 con 0x7f9644102b90 2024-09-05T12:34:04.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.505+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a6ab0 con 0x7f9644102b90 2024-09-05T12:34:04.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.505+0000 7f9641600640 1 -- 172.21.15.71:0/573736701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638010690 con 0x7f9644102b90 2024-09-05T12:34:04.507 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.505+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441a6ff0 con 0x7f9644102b90 2024-09-05T12:34:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.509+0000 7f964e257740 1 Processor -- start 2024-09-05T12:34:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.509+0000 7f964e257740 1 -- start start 2024-09-05T12:34:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.509+0000 7f9641600640 1 -- 172.21.15.71:0/573736701 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f96380037a0 con 0x7f9644102b90 2024-09-05T12:34:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f964e257740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x55956006d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f964e257740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.510 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f964e257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x55956007aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f964e257740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55955ffd3650 con 0x55956006d520 2024-09-05T12:34:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f964e257740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55955ff7af10 con 0x55956006de40 2024-09-05T12:34:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f964e257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55955ffcf610 con 0x559560078700 2024-09-05T12:34:04.511 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9649800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x55956006d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9648e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9649800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x55956006d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41806/0 (socket says 172.21.15.71:41806) 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9649800640 1 -- 172.21.15.71:0/2586890907 learned_addr learned my addr 172.21.15.71:0/2586890907 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9643e00640 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x55956007aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9649800640 1 -- 172.21.15.71:0/2586890907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 msgr2=0x55956007aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9649800640 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x55956007aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.510+0000 7f9649800640 1 -- 172.21.15.71:0/2586890907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 msgr2=0x5595600781c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9649800640 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600781c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9649800640 1 -- 172.21.15.71:0/2586890907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55955fce2580 con 0x55956006d520 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9643e00640 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x55956007aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:04.512 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9649800640 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x55956006d900 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f963c009f40 tx=0x7f963c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb9a2c5d5f72f09 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9636000640 1 -- 172.21.15.71:0/2586890907 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c003b70 con 0x55956006d520 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9636000640 1 -- 172.21.15.71:0/2586890907 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f963c003d10 con 0x55956006d520 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9636000640 1 -- 172.21.15.71:0/2586890907 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963c0119e0 con 0x55956006d520 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.511+0000 7f9641600640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f962c0a2cf0 0x7f962c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f9641600640 1 -- 172.21.15.71:0/573736701 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f96380cfb30 con 0x7f9644102b90 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f9648e00640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f962c0a2cf0 0x7f962c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 -- 172.21.15.71:0/2586890907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 msgr2=0x55956006d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x55956006d900 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f963c009f40 tx=0x7f963c010860 comp rx=0 tx=0).stop 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 -- 172.21.15.71:0/2586890907 shutdown_connections 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x55956007aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600781c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.514 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 --2- 172.21.15.71:0/2586890907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x55956006d900 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 -- 172.21.15.71:0/2586890907 >> 172.21.15.71:0/2586890907 conn(0x559560069ab0 msgr2=0x559560069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.512+0000 7f964e257740 1 -- 172.21.15.71:0/2586890907 shutdown_connections 2024-09-05T12:34:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.513+0000 7f9648e00640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f962c0a2cf0 0x7f962c0a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96380ccff0 tx=0x7f96380d3680 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.513+0000 7f964e257740 1 -- 172.21.15.71:0/2586890907 wait complete. 2024-09-05T12:34:04.515 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.513+0000 7f964e257740 1 -- start start 2024-09-05T12:34:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.513+0000 7f964e257740 1 Processor -- start 2024-09-05T12:34:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006d520 0x5595600756a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.516 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x559560075be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559560078700 0x559560076120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55956007b2d0 con 0x559560078700 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55955ff8e350 con 0x55956006de40 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55956007b450 con 0x55956006d520 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f964e257740 10 client.0 _ll_register_callbacks cb 0x55955ffb1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f9649800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006d520 0x5595600756a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f9643e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x559560075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.514+0000 7f9643e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x559560075be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51300/0 (socket says 172.21.15.71:51300) 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 -- 172.21.15.71:0/4199357446 learned_addr learned my addr 172.21.15.71:0/4199357446 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9649800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559560078700 0x559560076120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 -- 172.21.15.71:0/4199357446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006d520 msgr2=0x5595600756a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006d520 0x5595600756a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.517 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 -- 172.21.15.71:0/4199357446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559560078700 msgr2=0x559560076120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559560078700 0x559560076120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 -- 172.21.15.71:0/4199357446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55955fdac960 con 0x55956006de40 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9649800640 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006d520 0x5595600756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f9643e00640 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x559560075be0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9630034000 tx=0x7f9630009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6dcdbcbe054bc53 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.515+0000 7f962aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f962aa00640 1 -- 172.21.15.71:0/4199357446 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96300305b0 con 0x55956006de40 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f963c003d10 con 0x55956006de40 2024-09-05T12:34:04.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f962aa00640 1 -- 172.21.15.71:0/4199357446 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f9630030750 con 0x55956006de40 2024-09-05T12:34:04.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f962aa00640 1 -- 172.21.15.71:0/4199357446 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f963004e050 con 0x55956006de40 2024-09-05T12:34:04.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f964e257740 10 client.26147 fetch_fsmap learned FSMap version 37 2024-09-05T12:34:04.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559560086fe0 con 0x55956006de40 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.516+0000 7f962aa00640 1 -- 172.21.15.71:0/4199357446 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f963c003d10 con 0x55956006de40 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f962aa00640 1 -- 172.21.15.71:0/4199357446 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7f9630004cc0 con 0x55956006de40 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f964e257740 10 client.26147 fetch_fsmap finished waiting for FSMap version 37 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f964e257740 10 client.26147 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f964e257740 20 client.26147 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f964e257740 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600884b0 0x55956008a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f964e257740 4 client.26147 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55955ffe0bc0 con 0x5595600884b0 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.517+0000 7f9649800640 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600884b0 0x55956008a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.518+0000 7f9649800640 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600884b0 0x55956008a8d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=981ed04ebd830b6b server_cookie=3e146753e2c69d42 in_seq=0 out_seq=0 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.518+0000 7f962aa00640 10 client.26147 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.518+0000 7f962aa00640 1 -- 172.21.15.71:0/4199357446 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55955ffe0bc0 con 0x5595600884b0 2024-09-05T12:34:04.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.518+0000 7f962aa00640 10 client.26147 handle_command_reply: tid=0 2024-09-05T12:34:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 client.26147 shutdown 2024-09-05T12:34:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 msgr2=0x559560075be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x559560075be0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9630034000 tx=0x7f9630009580 comp rx=0 tx=0).stop 2024-09-05T12:34:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 shutdown_connections 2024-09-05T12:34:04.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600884b0 0x55956008a8d0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559560078700 0x559560076120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x559560075be0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 --2- 172.21.15.71:0/4199357446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006d520 0x5595600756a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 >> 172.21.15.71:0/4199357446 conn(0x559560069ab0 msgr2=0x559560068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f962aa00640 0 client.26147 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 shutdown_connections 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 1 -- 172.21.15.71:0/4199357446 wait complete. 2024-09-05T12:34:04.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.519+0000 7f964e257740 20 client.26147 trim_cache size 0 max 16384 2024-09-05T12:34:04.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.546+0000 7f964e257740 1 Processor -- start 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.546+0000 7f964e257740 1 -- start start 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f964e257740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560077500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f964e257740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f964e257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x5595600906e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f964e257740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b0f10 con 0x55956006d520 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f964e257740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b2a80 con 0x55956006de40 2024-09-05T12:34:04.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f964e257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b15e0 con 0x559560078700 2024-09-05T12:34:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f9649800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560077500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f9643e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f9649800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560077500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41830/0 (socket says 172.21.15.71:41830) 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f9649800640 1 -- 172.21.15.71:0/3667267692 learned_addr learned my addr 172.21.15.71:0/3667267692 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.547+0000 7f9649800640 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x5595600906e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9643e00640 1 -- 172.21.15.71:0/3667267692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 msgr2=0x5595600906e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9643e00640 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x5595600906e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9643e00640 1 -- 172.21.15.71:0/3667267692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 msgr2=0x559560077500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9643e00640 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560077500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9643e00640 1 -- 172.21.15.71:0/3667267692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956006dc70 con 0x55956006de40 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9649800640 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560077500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9643e00640 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600901a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9630005e00 tx=0x7f963002a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dda1574ba057318 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9635600640 1 -- 172.21.15.71:0/3667267692 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630005850 con 0x55956006de40 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9635600640 1 -- 172.21.15.71:0/3667267692 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630005520 con 0x55956006de40 2024-09-05T12:34:04.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.548+0000 7f9635600640 1 -- 172.21.15.71:0/3667267692 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630030810 con 0x55956006de40 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 -- 172.21.15.71:0/3667267692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 msgr2=0x5595600901a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600901a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9630005e00 tx=0x7f963002a3d0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 -- 172.21.15.71:0/3667267692 shutdown_connections 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559560078700 0x5595600906e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55956006de40 0x5595600901a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 --2- 172.21.15.71:0/3667267692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560077500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 -- 172.21.15.71:0/3667267692 >> 172.21.15.71:0/3667267692 conn(0x559560069ab0 msgr2=0x559560076df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.549+0000 7f964e257740 1 -- 172.21.15.71:0/3667267692 shutdown_connections 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.550+0000 7f964e257740 1 -- 172.21.15.71:0/3667267692 wait complete. 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.550+0000 7f964e257740 1 -- start start 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.550+0000 7f964e257740 1 Processor -- start 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560080d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 0x55956007c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559560078700 0x55956007c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b23b0 con 0x55956006d520 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96441b1ce0 con 0x559560078700 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96441aeef0 con 0x55956006de40 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f964e257740 10 client.0 _ll_register_callbacks cb 0x7f963c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f9643e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 0x55956007c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f9643e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 0x55956007c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47644/0 (socket says 172.21.15.71:47644) 2024-09-05T12:34:04.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f9643e00640 1 -- 172.21.15.71:0/4175252181 learned_addr learned my addr 172.21.15.71:0/4175252181 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:04.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f9649800640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559560078700 0x55956007c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.551+0000 7f9649800640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560080d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f9643e00640 1 -- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559560078700 msgr2=0x55956007c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f9643e00640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559560078700 0x55956007c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f9643e00640 1 -- 172.21.15.71:0/4175252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 msgr2=0x559560080d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f9643e00640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560080d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f9643e00640 1 -- 172.21.15.71:0/4175252181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956007ce60 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f9643e00640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 0x55956007c0f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f963002fe10 tx=0x7f9630005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9f120b583741711 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f962aa00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f962aa00640 1 -- 172.21.15.71:0/4175252181 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630005850 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9630005520 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.552+0000 7f962aa00640 1 -- 172.21.15.71:0/4175252181 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7f9630042040 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f962aa00640 1 -- 172.21.15.71:0/4175252181 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9630067050 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 10 client.26179 fetch_fsmap learned FSMap version 37 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5595600fb950 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f962aa00640 1 -- 172.21.15.71:0/4175252181 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9630005520 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f962aa00640 1 -- 172.21.15.71:0/4175252181 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7f96300539b0 con 0x55956006de40 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 10 client.26179 fetch_fsmap finished waiting for FSMap version 37 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 10 client.26179 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 20 client.26179 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600fc020 0x5595600fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:04.556 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 4 client.26179 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "17248364", "release": true, "await": true}] 2024-09-05T12:34:04.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.553+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "17248364", "release": true, "await": true}) -- 0x55955ffdb0d0 con 0x5595600fc020 2024-09-05T12:34:04.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.554+0000 7f9649800640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600fc020 0x5595600fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:04.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.554+0000 7f9649800640 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600fc020 0x5595600fe440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97ad221b70b9366 server_cookie=164158d1df5a4d6 in_seq=0 out_seq=0 2024-09-05T12:34:04.557 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.554+0000 7f962aa00640 10 client.26179 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:04.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.567+0000 7f962aa00640 1 -- 172.21.15.71:0/4175252181 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55955ffdb0d0 con 0x5595600fc020 2024-09-05T12:34:04.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.567+0000 7f962aa00640 10 client.26179 handle_command_reply: tid=0 2024-09-05T12:34:04.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.567+0000 7f964e257740 1 client.26179 shutdown 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.567+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 msgr2=0x55956007c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.567+0000 7f964e257740 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 0x55956007c0f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f963002fe10 tx=0x7f9630005740 comp rx=0 tx=0).stop 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.567+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 shutdown_connections 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595600fc020 0x5595600fe440 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559560078700 0x55956007c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55956006de40 0x55956007c0f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 --2- 172.21.15.71:0/4175252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55956006d520 0x559560080d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 >> 172.21.15.71:0/4175252181 conn(0x559560069ab0 msgr2=0x559560074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f962aa00640 0 client.26179 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 shutdown_connections 2024-09-05T12:34:04.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 1 -- 172.21.15.71:0/4175252181 wait complete. 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.568+0000 7f964e257740 20 client.26179 trim_cache size 0 max 16384 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 37, 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 24, 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "17248364": { 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 24, 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:34:04.570 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:34:04.571 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:34:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:34:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:34:04.572 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:34:04.572 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f962c0a2cf0 msgr2=0x7f962c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f962c0a2cf0 0x7f962c0a51b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96380ccff0 tx=0x7f96380d3680 comp rx=0 tx=0).stop 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 msgr2=0x7f96441007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f96441007f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9c0 tx=0x7f963800ee80 comp rx=0 tx=0).stop 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 shutdown_connections 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f964410bfc0 0x7f9644101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96441034b0 0x7f9644100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f962c0a2cf0 0x7f962c0a51b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 --2- 172.21.15.71:0/573736701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9644102b90 0x7f96441007f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 >> 172.21.15.71:0/573736701 conn(0x7f96440fc610 msgr2=0x7f96440fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 shutdown_connections 2024-09-05T12:34:04.573 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:34:04.569+0000 7f964a200640 1 -- 172.21.15.71:0/573736701 wait complete. 2024-09-05T12:34:05.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 208299 3.33 MB/sec execute 181 sec latency 30044.290 ms 2024-09-05T12:34:05.794 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '17248364', total seconds elapsed: 34.4 2024-09-05T12:34:06.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:05 smithi071 ceph-mon[29845]: pgmap v193: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 43 op/s 2024-09-05T12:34:06.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 209821 3.37 MB/sec execute 182 sec latency 12.012 ms 2024-09-05T12:34:06.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:05 smithi174 ceph-mon[34313]: pgmap v193: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 43 op/s 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 -- 172.21.15.71:0/1506803559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc308103b00 msgr2=0x7fc308103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 --2- 172.21.15.71:0/1506803559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc308103b00 0x7fc308103f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009900 tx=0x7fc2f402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 -- 172.21.15.71:0/1506803559 shutdown_connections 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 --2- 172.21.15.71:0/1506803559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3081044a0 0x7fc3081109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 --2- 172.21.15.71:0/1506803559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc308103b00 0x7fc308103f60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 --2- 172.21.15.71:0/1506803559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc308109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 -- 172.21.15.71:0/1506803559 >> 172.21.15.71:0/1506803559 conn(0x7fc3080ff7c0 msgr2=0x7fc308101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.408+0000 7fc30f000640 1 -- 172.21.15.71:0/1506803559 shutdown_connections 2024-09-05T12:34:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.409+0000 7fc30f000640 1 -- 172.21.15.71:0/1506803559 wait complete. 2024-09-05T12:34:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.409+0000 7fc30f000640 1 Processor -- start 2024-09-05T12:34:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 -- start start 2024-09-05T12:34:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc308103b00 0x7fc30819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081044a0 0x7fc30819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc30819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc308113f60 con 0x7fc308103b00 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc308113de0 con 0x7fc3081044a0 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30f000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081140e0 con 0x7fc308109ba0 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc30819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc308103b00 0x7fc30819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081044a0 0x7fc30819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc30819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47672/0 (socket says 172.21.15.71:47672) 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.410+0000 7fc30e600640 1 -- 172.21.15.71:0/2467857774 learned_addr learned my addr 172.21.15.71:0/2467857774 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30e600640 1 -- 172.21.15.71:0/2467857774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081044a0 msgr2=0x7fc30819a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30e600640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081044a0 0x7fc30819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30e600640 1 -- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc308103b00 msgr2=0x7fc30819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30e600640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc308103b00 0x7fc30819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30e600640 1 -- 172.21.15.71:0/2467857774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30819f440 con 0x7fc308109ba0 2024-09-05T12:34:06.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30dc00640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc308103b00 0x7fc30819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30e600640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc30819ed10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc30000eb10 tx=0x7fc30000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc306000640 1 -- 172.21.15.71:0/2467857774 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300004b20 con 0x7fc308109ba0 2024-09-05T12:34:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc306000640 1 -- 172.21.15.71:0/2467857774 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc300004cc0 con 0x7fc308109ba0 2024-09-05T12:34:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30819f730 con 0x7fc308109ba0 2024-09-05T12:34:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc306000640 1 -- 172.21.15.71:0/2467857774 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300010760 con 0x7fc308109ba0 2024-09-05T12:34:06.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.411+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a6fc0 con 0x7fc308109ba0 2024-09-05T12:34:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.414+0000 7fc306000640 1 -- 172.21.15.71:0/2467857774 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc300005020 con 0x7fc308109ba0 2024-09-05T12:34:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 Processor -- start 2024-09-05T12:34:06.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 -- start start 2024-09-05T12:34:06.416 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a336790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a341030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a343960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55568a29c100 con 0x55568a336cd0 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55568a2441b0 con 0x55568a3363b0 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.415+0000 7fc313485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55568a2980c0 con 0x55568a341570 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a336790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a336790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51342/0 (socket says 172.21.15.71:51342) 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 -- 172.21.15.71:0/3432263748 learned_addr learned my addr 172.21.15.71:0/3432263748 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a343960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30dc00640 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a341030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.417 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc306000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc2f00a2cf0 0x7fc2f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 -- 172.21.15.71:0/3432263748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 msgr2=0x55568a343960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a343960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.419 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 -- 172.21.15.71:0/3432263748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 msgr2=0x55568a341030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a341030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 -- 172.21.15.71:0/3432263748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555689faf210 con 0x55568a3363b0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc306000640 1 -- 172.21.15.71:0/2467857774 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc3000cfda0 con 0x7fc308109ba0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30e600640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc2f00a2cf0 0x7fc2f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.416+0000 7fc30d200640 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a336790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55568a333ca0 tx=0x7fc2f4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61d337d85ca35906 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.417+0000 7fc2faa00640 1 -- 172.21.15.71:0/3432263748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4035030 con 0x55568a3363b0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.417+0000 7fc2faa00640 1 -- 172.21.15.71:0/3432263748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f40046c0 con 0x55568a3363b0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.417+0000 7fc2faa00640 1 -- 172.21.15.71:0/3432263748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4003850 con 0x55568a3363b0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 -- 172.21.15.71:0/3432263748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 msgr2=0x55568a336790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a336790 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55568a333ca0 tx=0x7fc2f4005b50 comp rx=0 tx=0).stop 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc30e600640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc2f00a2cf0 0x7fc2f00a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc300098380 tx=0x7fc300005b40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 -- 172.21.15.71:0/3432263748 shutdown_connections 2024-09-05T12:34:06.420 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a343960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a341030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 --2- 172.21.15.71:0/3432263748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a336790 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 -- 172.21.15.71:0/3432263748 >> 172.21.15.71:0/3432263748 conn(0x55568a332920 msgr2=0x55568a332d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 -- 172.21.15.71:0/3432263748 shutdown_connections 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 -- 172.21.15.71:0/3432263748 wait complete. 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 -- start start 2024-09-05T12:34:06.421 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.418+0000 7fc313485740 1 Processor -- start 2024-09-05T12:34:06.424 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a3363b0 0x55568a33e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 0x55568a33ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a33f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55568a343ff0 con 0x55568a3363b0 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55568a225090 con 0x55568a336cd0 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55568a344170 con 0x55568a341570 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc313485740 10 client.0 _ll_register_callbacks cb 0x55568a279f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 0x55568a33ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 0x55568a33ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51346/0 (socket says 172.21.15.71:51346) 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 -- 172.21.15.71:0/1850723673 learned_addr learned my addr 172.21.15.71:0/1850723673 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a3363b0 0x55568a33e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30d200640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a33f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 -- 172.21.15.71:0/1850723673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 msgr2=0x55568a33f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a33f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 -- 172.21.15.71:0/1850723673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a3363b0 msgr2=0x55568a33e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a3363b0 0x55568a33e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30dc00640 1 -- 172.21.15.71:0/1850723673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568a07b620 con 0x55568a336cd0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.419+0000 7fc30d200640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a33f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc30dc00640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 0x55568a33ebe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc012d20 tx=0x7fc2fc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f845b41b709fc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc2ef400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc2ef400640 1 -- 172.21.15.71:0/1850723673 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc0120d0 con 0x55568a336cd0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f40046c0 con 0x55568a336cd0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc2ef400640 1 -- 172.21.15.71:0/1850723673 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc2fc010430 con 0x55568a336cd0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc2ef400640 1 -- 172.21.15.71:0/1850723673 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc012270 con 0x55568a336cd0 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc313485740 10 client.26165 fetch_fsmap learned FSMap version 37 2024-09-05T12:34:06.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55568a34fe80 con 0x55568a336cd0 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.420+0000 7fc2ef400640 1 -- 172.21.15.71:0/1850723673 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f40046c0 con 0x55568a336cd0 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc2ef400640 1 -- 172.21.15.71:0/1850723673 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7fc2fc01c830 con 0x55568a336cd0 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc313485740 10 client.26165 fetch_fsmap finished waiting for FSMap version 37 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc313485740 10 client.26165 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc313485740 20 client.26165 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc313485740 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a351390 0x55568a353780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc313485740 4 client.26165 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55568a2a9830 con 0x55568a351390 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.421+0000 7fc30d200640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a351390 0x55568a353780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.422+0000 7fc2ef400640 10 client.26165 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.422+0000 7fc30d200640 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a351390 0x55568a353780 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551e250487a95eb5 server_cookie=6584e6105a266100 in_seq=0 out_seq=0 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.422+0000 7fc2ef400640 1 -- 172.21.15.71:0/1850723673 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55568a2a9830 con 0x55568a351390 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.422+0000 7fc2ef400640 10 client.26165 handle_command_reply: tid=0 2024-09-05T12:34:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.422+0000 7fc313485740 1 client.26165 shutdown 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 msgr2=0x55568a33ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 0x55568a33ebe0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc012d20 tx=0x7fc2fc00f790 comp rx=0 tx=0).stop 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 shutdown_connections 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a351390 0x55568a353780 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a33f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a336cd0 0x55568a33ebe0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 --2- 172.21.15.71:0/1850723673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a3363b0 0x55568a33e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc2ef400640 0 client.26165 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 >> 172.21.15.71:0/1850723673 conn(0x55568a332920 msgr2=0x55568a338680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 shutdown_connections 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 1 -- 172.21.15.71:0/1850723673 wait complete. 2024-09-05T12:34:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.423+0000 7fc313485740 20 client.26165 trim_cache size 0 max 16384 2024-09-05T12:34:06.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:05 smithi123 ceph-mon[34321]: pgmap v193: 129 pgs: 129 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 97 KiB/s wr, 43 op/s 2024-09-05T12:34:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 Processor -- start 2024-09-05T12:34:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 -- start start 2024-09-05T12:34:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 0x55568a359090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a3595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a341570 0x55568a337e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b0f70 con 0x55568a336cd0 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b2ae0 con 0x55568a341570 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.439+0000 7fc313485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b1640 con 0x55568a3363b0 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 0x55568a359090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 0x55568a359090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47698/0 (socket says 172.21.15.71:47698) 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 -- 172.21.15.71:0/876755572 learned_addr learned my addr 172.21.15.71:0/876755572 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a341570 0x55568a337e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30dc00640 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a3595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 -- 172.21.15.71:0/876755572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a341570 msgr2=0x55568a337e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a341570 0x55568a337e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 -- 172.21.15.71:0/876755572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 msgr2=0x55568a3595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a3595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 -- 172.21.15.71:0/876755572 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55568a333460 con 0x55568a3363b0 2024-09-05T12:34:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc30d200640 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 0x55568a359090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005c40 tx=0x7fc2f4004eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dad59bb490742cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.440+0000 7fc2fa000640 1 -- 172.21.15.71:0/876755572 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4035040 con 0x55568a3363b0 2024-09-05T12:34:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc2fa000640 1 -- 172.21.15.71:0/876755572 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4041040 con 0x55568a3363b0 2024-09-05T12:34:06.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc2fa000640 1 -- 172.21.15.71:0/876755572 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4003850 con 0x55568a3363b0 2024-09-05T12:34:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 -- 172.21.15.71:0/876755572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 msgr2=0x55568a359090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 0x55568a359090 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4005c40 tx=0x7fc2f4004eb0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.446 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 -- 172.21.15.71:0/876755572 shutdown_connections 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a341570 0x55568a337e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a3595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 --2- 172.21.15.71:0/876755572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a3363b0 0x55568a359090 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 -- 172.21.15.71:0/876755572 >> 172.21.15.71:0/876755572 conn(0x55568a332920 msgr2=0x55568a337550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.441+0000 7fc313485740 1 -- 172.21.15.71:0/876755572 shutdown_connections 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.442+0000 7fc313485740 1 -- 172.21.15.71:0/876755572 wait complete. 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.442+0000 7fc313485740 1 -- start start 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.442+0000 7fc313485740 1 Processor -- start 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a3498e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a344c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a345140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b2410 con 0x55568a336cd0 2024-09-05T12:34:06.447 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081b1d40 con 0x55568a3363b0 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081aef50 con 0x55568a341570 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc313485740 10 client.0 _ll_register_callbacks cb 0x7fc2fc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc30d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a345140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc30dc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a344c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc30d200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a345140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47708/0 (socket says 172.21.15.71:47708) 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc30d200640 1 -- 172.21.15.71:0/3108265671 learned_addr learned my addr 172.21.15.71:0/3108265671 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.443+0000 7fc30d200640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a3498e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30dc00640 1 -- 172.21.15.71:0/3108265671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 msgr2=0x55568a345140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30dc00640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a345140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30dc00640 1 -- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 msgr2=0x55568a3498e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30dc00640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a3498e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30dc00640 1 -- 172.21.15.71:0/3108265671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55568a345700 con 0x55568a336cd0 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30d200640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a345140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc30dc00640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a344c00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55568a33fc50 tx=0x7fc2fc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11077eb6c2029905 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.444+0000 7fc2ef400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc2ef400640 1 -- 172.21.15.71:0/3108265671 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc005110 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc2ef400640 1 -- 172.21.15.71:0/3108265671 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9350+0+0 (secure 0 0 0) 0x7fc2fc0052b0 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f4041040 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc2ef400640 1 -- 172.21.15.71:0/3108265671 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc2fc0054b0 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc313485740 10 client.16143 fetch_fsmap learned FSMap version 37 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55568a3bbe40 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc2ef400640 1 -- 172.21.15.71:0/3108265671 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2f4041040 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.445+0000 7fc2ef400640 1 -- 172.21.15.71:0/3108265671 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 37) ==== 8374+0+0 (secure 0 0 0) 0x7fc2fc02a4c0 con 0x55568a336cd0 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc313485740 10 client.16143 fetch_fsmap finished waiting for FSMap version 37 2024-09-05T12:34:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc313485740 10 client.16143 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc313485740 20 client.16143 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc313485740 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a3bc640 0x55568a3bea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc313485740 4 client.16143 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:34:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55568a2a3b80 con 0x55568a3bc640 2024-09-05T12:34:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc30d200640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a3bc640 0x55568a3bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc30d200640 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a3bc640 0x55568a3bea60 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89be81224d2812a9 server_cookie=f79497831a7fa4f5 in_seq=0 out_seq=0 2024-09-05T12:34:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.446+0000 7fc2ef400640 10 client.16143 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.474+0000 7fc2ef400640 1 -- 172.21.15.71:0/3108265671 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55568a2a3b80 con 0x55568a3bc640 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.474+0000 7fc2ef400640 10 client.16143 handle_command_reply: tid=0 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.474+0000 7fc313485740 1 client.16143 shutdown 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.474+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 msgr2=0x55568a344c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.474+0000 7fc313485740 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a344c00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55568a33fc50 tx=0x7fc2fc00fd10 comp rx=0 tx=0).stop 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 shutdown_connections 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55568a3bc640 0x55568a3bea60 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55568a341570 0x55568a345140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55568a336cd0 0x55568a344c00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 --2- 172.21.15.71:0/3108265671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55568a3363b0 0x55568a3498e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc2ef400640 0 client.16143 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:06.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 >> 172.21.15.71:0/3108265671 conn(0x55568a332920 msgr2=0x55568a33e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 shutdown_connections 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 1 -- 172.21.15.71:0/3108265671 wait complete. 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.475+0000 7fc313485740 20 client.16143 trim_cache size 0 max 16384 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:34:06.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:34:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc2f00a2cf0 msgr2=0x7fc2f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc2f00a2cf0 0x7fc2f00a51b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc300098380 tx=0x7fc300005b40 comp rx=0 tx=0).stop 2024-09-05T12:34:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 msgr2=0x7fc30819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc30819ed10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc30000eb10 tx=0x7fc30000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 shutdown_connections 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308109ba0 0x7fc30819ed10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3081044a0 0x7fc30819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc2f00a2cf0 0x7fc2f00a51b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 --2- 172.21.15.71:0/2467857774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc308103b00 0x7fc30819a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 >> 172.21.15.71:0/2467857774 conn(0x7fc3080ff7c0 msgr2=0x7fc3080fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 shutdown_connections 2024-09-05T12:34:06.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:06.476+0000 7fc30f000640 1 -- 172.21.15.71:0/2467857774 wait complete. 2024-09-05T12:34:07.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 211787 3.39 MB/sec execute 183 sec latency 10.926 ms 2024-09-05T12:34:08.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 213162 3.44 MB/sec execute 184 sec latency 35.107 ms 2024-09-05T12:34:08.251 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:07 smithi071 ceph-mon[29845]: pgmap v194: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 192 KiB/s wr, 62 op/s 2024-09-05T12:34:08.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:07 smithi174 ceph-mon[34313]: pgmap v194: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 192 KiB/s wr, 62 op/s 2024-09-05T12:34:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:07 smithi123 ceph-mon[34321]: pgmap v194: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 192 KiB/s wr, 62 op/s 2024-09-05T12:34:09.217 INFO:tasks.workunit.client.0.smithi071.stdout: 1 214106 3.43 MB/sec execute 185 sec latency 32.237 ms 2024-09-05T12:34:09.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:08 smithi174 ceph-mon[34313]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T12:34:09.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:08 smithi071 ceph-mon[29845]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T12:34:09.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:08 smithi123 ceph-mon[34321]: osdmap e96: 12 total, 12 up, 12 in 2024-09-05T12:34:10.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 216224 3.48 MB/sec execute 186 sec latency 213.976 ms 2024-09-05T12:34:10.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:09 smithi174 ceph-mon[34313]: pgmap v196: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 184 KiB/s wr, 49 op/s 2024-09-05T12:34:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:09 smithi071 ceph-mon[29845]: pgmap v196: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 184 KiB/s wr, 49 op/s 2024-09-05T12:34:10.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:09 smithi123 ceph-mon[34321]: pgmap v196: 129 pgs: 129 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 184 KiB/s wr, 49 op/s 2024-09-05T12:34:11.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217016 3.49 MB/sec execute 187 sec latency 240.501 ms 2024-09-05T12:34:12.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217016 3.47 MB/sec execute 188 sec latency 1240.546 ms 2024-09-05T12:34:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:11 smithi174 ceph-mon[34313]: pgmap v197: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 777 KiB/s wr, 156 op/s 2024-09-05T12:34:12.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:11 smithi071 ceph-mon[29845]: pgmap v197: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 777 KiB/s wr, 156 op/s 2024-09-05T12:34:12.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:11 smithi123 ceph-mon[34321]: pgmap v197: 129 pgs: 129 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 777 KiB/s wr, 156 op/s 2024-09-05T12:34:13.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217016 3.45 MB/sec execute 189 sec latency 2240.669 ms 2024-09-05T12:34:13.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:12 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:13.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:12 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:13.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:12 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:14.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217016 3.43 MB/sec execute 190 sec latency 3240.790 ms 2024-09-05T12:34:14.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:13 smithi174 ceph-mon[34313]: pgmap v198: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 844 KiB/s wr, 151 op/s 2024-09-05T12:34:14.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:13 smithi071 ceph-mon[29845]: pgmap v198: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 844 KiB/s wr, 151 op/s 2024-09-05T12:34:14.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:13 smithi123 ceph-mon[34321]: pgmap v198: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 844 KiB/s wr, 151 op/s 2024-09-05T12:34:15.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217016 3.41 MB/sec execute 191 sec latency 4240.906 ms 2024-09-05T12:34:16.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217016 3.39 MB/sec execute 192 sec latency 5241.023 ms 2024-09-05T12:34:16.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:16 smithi071 ceph-mon[29845]: pgmap v199: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 844 KiB/s wr, 151 op/s 2024-09-05T12:34:16.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:16 smithi071 ceph-mon[29845]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T12:34:16.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:16 smithi123 ceph-mon[34321]: pgmap v199: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 844 KiB/s wr, 151 op/s 2024-09-05T12:34:16.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:16 smithi123 ceph-mon[34321]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T12:34:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:16 smithi174 ceph-mon[34313]: pgmap v199: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 844 KiB/s wr, 151 op/s 2024-09-05T12:34:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:16 smithi174 ceph-mon[34313]: osdmap e97: 12 total, 12 up, 12 in 2024-09-05T12:34:17.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 217349 3.38 MB/sec execute 193 sec latency 6002.818 ms 2024-09-05T12:34:18.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 219342 3.42 MB/sec execute 194 sec latency 16.908 ms 2024-09-05T12:34:18.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:18 smithi071 ceph-mon[29845]: pgmap v201: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 922 KiB/s wr, 171 op/s 2024-09-05T12:34:18.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:18 smithi123 ceph-mon[34321]: pgmap v201: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 922 KiB/s wr, 171 op/s 2024-09-05T12:34:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:18 smithi174 ceph-mon[34313]: pgmap v201: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 922 KiB/s wr, 171 op/s 2024-09-05T12:34:19.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 220687 3.44 MB/sec execute 195 sec latency 12.444 ms 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 -- 172.21.15.71:0/18331488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 msgr2=0x7efc04107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 --2- 172.21.15.71:0/18331488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 0x7efc04107ca0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009930 tx=0x7efbf402a440 comp rx=0 tx=0).stop 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 -- 172.21.15.71:0/18331488 shutdown_connections 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 --2- 172.21.15.71:0/18331488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc04102250 0x7efc0410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 --2- 172.21.15.71:0/18331488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc041018b0 0x7efc04101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 --2- 172.21.15.71:0/18331488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 0x7efc04107ca0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 -- 172.21.15.71:0/18331488 >> 172.21.15.71:0/18331488 conn(0x7efc040fd5d0 msgr2=0x7efc040ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 -- 172.21.15.71:0/18331488 shutdown_connections 2024-09-05T12:34:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.600+0000 7efc08a00640 1 -- 172.21.15.71:0/18331488 wait complete. 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 Processor -- start 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 -- start start 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 0x7efc0419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102250 0x7efc0419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 0x7efc0419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111d50 con 0x7efc04102250 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111bd0 con 0x7efc041018b0 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc08a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc04111ed0 con 0x7efc041078c0 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc02800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 0x7efc0419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc01e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102250 0x7efc0419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc02800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 0x7efc0419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60692/0 (socket says 172.21.15.71:60692) 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.601+0000 7efc02800640 1 -- 172.21.15.71:0/2830554982 learned_addr learned my addr 172.21.15.71:0/2830554982 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc03200640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 0x7efc0419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc01e00640 1 -- 172.21.15.71:0/2830554982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 msgr2=0x7efc0419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc01e00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 0x7efc0419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc01e00640 1 -- 172.21.15.71:0/2830554982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 msgr2=0x7efc0419a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc01e00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 0x7efc0419a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc01e00640 1 -- 172.21.15.71:0/2830554982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0419f400 con 0x7efc04102250 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc02800640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 0x7efc0419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc01e00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102250 0x7efc0419a950 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efbf000e9f0 tx=0x7efbf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efbfb400640 1 -- 172.21.15.71:0/2830554982 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000ccc0 con 0x7efc04102250 2024-09-05T12:34:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0419f6f0 con 0x7efc04102250 2024-09-05T12:34:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efbfb400640 1 -- 172.21.15.71:0/2830554982 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf0010910 con 0x7efc04102250 2024-09-05T12:34:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.602+0000 7efbfb400640 1 -- 172.21.15.71:0/2830554982 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0018960 con 0x7efc04102250 2024-09-05T12:34:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.603+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a6f80 con 0x7efc04102250 2024-09-05T12:34:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 Processor -- start 2024-09-05T12:34:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 -- start start 2024-09-05T12:34:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efbfb400640 1 -- 172.21.15.71:0/2830554982 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efbf0010ac0 con 0x7efc04102250 2024-09-05T12:34:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c205790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c210030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c210570 0x55e92c212960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e92c16b100 con 0x55e92c205cd0 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e92c1131b0 con 0x55e92c2053b0 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.606+0000 7efc0c29a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e92c1670c0 con 0x55e92c210570 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c205790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c205790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60704/0 (socket says 172.21.15.71:60704) 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc01e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c210570 0x55e92c212960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 -- 172.21.15.71:0/927008680 learned_addr learned my addr 172.21.15.71:0/927008680 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc02800640 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c210030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 -- 172.21.15.71:0/927008680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c210570 msgr2=0x55e92c212960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c210570 0x55e92c212960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 -- 172.21.15.71:0/927008680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 msgr2=0x55e92c210030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c210030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 -- 172.21.15.71:0/927008680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92be7e210 con 0x55e92c2053b0 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc02800640 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c210030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efc03200640 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c205790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009f50 tx=0x7efbfc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4027333e9a8ed26 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.607+0000 7efbef400640 1 -- 172.21.15.71:0/927008680 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc004d20 con 0x55e92c2053b0 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efbef400640 1 -- 172.21.15.71:0/927008680 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbfc004050 con 0x55e92c2053b0 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efbef400640 1 -- 172.21.15.71:0/927008680 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbfc005220 con 0x55e92c2053b0 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efbfb400640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efbe80a2cf0 0x7efbe80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efbfb400640 1 -- 172.21.15.71:0/2830554982 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7efbf00cfbb0 con 0x7efc04102250 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc02800640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efbe80a2cf0 0x7efbe80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 -- 172.21.15.71:0/927008680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 msgr2=0x55e92c205790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c205790 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009f50 tx=0x7efbfc0118d0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 -- 172.21.15.71:0/927008680 shutdown_connections 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c210570 0x55e92c212960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c210030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 --2- 172.21.15.71:0/927008680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c205790 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.608+0000 7efc0c29a740 1 -- 172.21.15.71:0/927008680 >> 172.21.15.71:0/927008680 conn(0x55e92c201920 msgr2=0x55e92c201d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.609+0000 7efc0c29a740 1 -- 172.21.15.71:0/927008680 shutdown_connections 2024-09-05T12:34:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.609+0000 7efc0c29a740 1 -- 172.21.15.71:0/927008680 wait complete. 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.609+0000 7efc02800640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efbe80a2cf0 0x7efbe80a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf4031510 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.609+0000 7efc0c29a740 1 -- start start 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.609+0000 7efc0c29a740 1 Processor -- start 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c20d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c20db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c20e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e92c212ff0 con 0x55e92c210570 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e92c0f4090 con 0x55e92c2053b0 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e92c213170 con 0x55e92c205cd0 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc0c29a740 10 client.0 _ll_register_callbacks cb 0x55e92c148f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc01e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c20db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc03200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c20e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc01e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c20db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46262/0 (socket says 172.21.15.71:46262) 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc01e00640 1 -- 172.21.15.71:0/1183190142 learned_addr learned my addr 172.21.15.71:0/1183190142 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.610+0000 7efc02800640 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c20d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc01e00640 1 -- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 msgr2=0x55e92c20d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc01e00640 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c20d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc01e00640 1 -- 172.21.15.71:0/1183190142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 msgr2=0x55e92c20e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc01e00640 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c20e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc01e00640 1 -- 172.21.15.71:0/1183190142 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92bf4a620 con 0x55e92c205cd0 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc01e00640 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c20db60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efbf009ba90 tx=0x7efbf00980f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6d5f191b4afa6a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efbe3e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efbe3e00640 1 -- 172.21.15.71:0/1183190142 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf00cd3a0 con 0x55e92c205cd0 2024-09-05T12:34:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efbe3e00640 1 -- 172.21.15.71:0/1183190142 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7efbf009b3e0 con 0x55e92c205cd0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbfc004050 con 0x55e92c205cd0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efbe3e00640 1 -- 172.21.15.71:0/1183190142 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efbf009bd90 con 0x55e92c205cd0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc0c29a740 10 client.26209 fetch_fsmap learned FSMap version 38 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.611+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e92c2275e0 con 0x55e92c205cd0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efbe3e00640 1 -- 172.21.15.71:0/1183190142 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbfc004050 con 0x55e92c205cd0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efbe3e00640 1 -- 172.21.15.71:0/1183190142 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7efbf0097ae0 con 0x55e92c205cd0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc0c29a740 10 client.26209 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc0c29a740 10 client.26209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc0c29a740 20 client.26209 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc0c29a740 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c228af0 0x55e92c22aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc0c29a740 4 client.26209 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e92c178830 con 0x55e92c228af0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.612+0000 7efc02800640 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c228af0 0x55e92c22aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.613+0000 7efc02800640 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c228af0 0x55e92c22aee0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1411dbaef0bba01 server_cookie=706a5f1ced6d10f0 in_seq=0 out_seq=0 2024-09-05T12:34:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.613+0000 7efbe3e00640 10 client.26209 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efbe3e00640 1 -- 172.21.15.71:0/1183190142 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e92c178830 con 0x55e92c228af0 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efbe3e00640 10 client.26209 handle_command_reply: tid=0 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 client.26209 shutdown 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 msgr2=0x55e92c20db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c20db60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efbf009ba90 tx=0x7efbf00980f0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 shutdown_connections 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c228af0 0x55e92c22aee0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c20e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c20db60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 --2- 172.21.15.71:0/1183190142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c20d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 >> 172.21.15.71:0/1183190142 conn(0x55e92c201920 msgr2=0x55e92c207680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.614+0000 7efbe3e00640 0 client.26209 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.615+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 shutdown_connections 2024-09-05T12:34:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.615+0000 7efc0c29a740 1 -- 172.21.15.71:0/1183190142 wait complete. 2024-09-05T12:34:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.615+0000 7efc0c29a740 20 client.26209 trim_cache size 0 max 16384 2024-09-05T12:34:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.631+0000 7efc0c29a740 1 Processor -- start 2024-09-05T12:34:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.631+0000 7efc0c29a740 1 -- start start 2024-09-05T12:34:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.631+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c2307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.631+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c230d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c206e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc0c29a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc041acc50 con 0x55e92c210570 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc0c29a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ae790 con 0x55e92c2053b0 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc0c29a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ad320 con 0x55e92c205cd0 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c2307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc01e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c230d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c2307d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60724/0 (socket says 172.21.15.71:60724) 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 -- 172.21.15.71:0/423228546 learned_addr learned my addr 172.21.15.71:0/423228546 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc03200640 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c206e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 -- 172.21.15.71:0/423228546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 msgr2=0x55e92c230d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c230d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 -- 172.21.15.71:0/423228546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 msgr2=0x55e92c206e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c206e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 -- 172.21.15.71:0/423228546 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92c205b00 con 0x55e92c2053b0 2024-09-05T12:34:19.636 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.632+0000 7efc02800640 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c2307d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006010 tx=0x7efbf403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f956ff1999473f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.633+0000 7efbeea00640 1 -- 172.21.15.71:0/423228546 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4056070 con 0x55e92c2053b0 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.633+0000 7efbeea00640 1 -- 172.21.15.71:0/423228546 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf40315b0 con 0x55e92c2053b0 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.633+0000 7efbeea00640 1 -- 172.21.15.71:0/423228546 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4035040 con 0x55e92c2053b0 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.633+0000 7efc0c29a740 1 -- 172.21.15.71:0/423228546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 msgr2=0x55e92c2307d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.633+0000 7efc0c29a740 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c2307d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efbf4006010 tx=0x7efbf403f000 comp rx=0 tx=0).stop 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.633+0000 7efc0c29a740 1 -- 172.21.15.71:0/423228546 shutdown_connections 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c210570 0x55e92c206e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c205cd0 0x55e92c230d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 --2- 172.21.15.71:0/423228546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c2053b0 0x55e92c2307d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 -- 172.21.15.71:0/423228546 >> 172.21.15.71:0/423228546 conn(0x55e92c201920 msgr2=0x55e92c208910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 -- 172.21.15.71:0/423228546 shutdown_connections 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 -- 172.21.15.71:0/423228546 wait complete. 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 -- start start 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.634+0000 7efc0c29a740 1 Processor -- start 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 0x55e92c221410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c21c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c210570 0x55e92c21ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ae0c0 con 0x55e92c205cd0 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc041ad9f0 con 0x55e92c210570 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc041aac90 con 0x55e92c2053b0 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc0c29a740 10 client.0 _ll_register_callbacks cb 0x7efbf00d3350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:19.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 0x55e92c221410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 0x55e92c221410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46282/0 (socket says 172.21.15.71:46282) 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 -- 172.21.15.71:0/229480371 learned_addr learned my addr 172.21.15.71:0/229480371 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc03200640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c210570 0x55e92c21ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc01e00640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c21c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 -- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c210570 msgr2=0x55e92c21ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c210570 0x55e92c21ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 -- 172.21.15.71:0/229480371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 msgr2=0x55e92c21c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c21c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 -- 172.21.15.71:0/229480371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92c21d4f0 con 0x55e92c2053b0 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc01e00640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c21c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.635+0000 7efc02800640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 0x55e92c221410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbf40092a0 tx=0x7efbf4005560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69f9ca6d7503a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efbe3e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:34:19.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efbe3e00640 1 -- 172.21.15.71:0/229480371 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4056070 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efbe3e00640 1 -- 172.21.15.71:0/229480371 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7efbf4031960 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf40315b0 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efbe3e00640 1 -- 172.21.15.71:0/229480371 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7efbf4031b60 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efc0c29a740 10 client.26215 fetch_fsmap learned FSMap version 38 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e92c293810 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.636+0000 7efbe3e00640 1 -- 172.21.15.71:0/229480371 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf40315b0 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efbe3e00640 1 -- 172.21.15.71:0/229480371 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 38) ==== 8370+0+0 (secure 0 0 0) 0x7efbf405d520 con 0x55e92c2053b0 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc0c29a740 10 client.26215 fetch_fsmap finished waiting for FSMap version 38 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc0c29a740 10 client.26215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc0c29a740 20 client.26215 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:19.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc0c29a740 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c293ed0 0x55e92c2962c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc0c29a740 4 client.26215 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:34:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e92c172b80 con 0x55e92c293ed0 2024-09-05T12:34:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.637+0000 7efc01e00640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c293ed0 0x55e92c2962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.638+0000 7efc01e00640 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c293ed0 0x55e92c2962c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88fce81683e0f210 server_cookie=2eef7ee99f2cdf70 in_seq=0 out_seq=0 2024-09-05T12:34:19.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.638+0000 7efbe3e00640 10 client.26215 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.688+0000 7efbe3e00640 1 -- 172.21.15.71:0/229480371 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e92c172b80 con 0x55e92c293ed0 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.688+0000 7efbe3e00640 10 client.26215 handle_command_reply: tid=0 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.688+0000 7efc0c29a740 1 client.26215 shutdown 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 msgr2=0x55e92c221410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 0x55e92c221410 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efbf40092a0 tx=0x7efbf4005560 comp rx=0 tx=0).stop 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 shutdown_connections 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e92c293ed0 0x55e92c2962c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92c210570 0x55e92c21ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e92c205cd0 0x55e92c21c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efbe3e00640 0 client.26215 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 --2- 172.21.15.71:0/229480371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92c2053b0 0x55e92c221410 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 >> 172.21.15.71:0/229480371 conn(0x55e92c201920 msgr2=0x55e92c206e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 shutdown_connections 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 1 -- 172.21.15.71:0/229480371 wait complete. 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.689+0000 7efc0c29a740 20 client.26215 trim_cache size 0 max 16384 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:34:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:34:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:34:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:34:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efbe80a2cf0 msgr2=0x7efbe80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efbe80a2cf0 0x7efbe80a51b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf4031510 comp rx=0 tx=0).stop 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102250 msgr2=0x7efc0419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102250 0x7efc0419a950 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efbf000e9f0 tx=0x7efbf000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 shutdown_connections 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc041078c0 0x7efc0419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efbe80a2cf0 0x7efbe80a51b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04102250 0x7efc0419a950 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 --2- 172.21.15.71:0/2830554982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc041018b0 0x7efc0419a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 >> 172.21.15.71:0/2830554982 conn(0x7efc040fd5d0 msgr2=0x7efc0410d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 shutdown_connections 2024-09-05T12:34:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:19.690+0000 7efc08a00640 1 -- 172.21.15.71:0/2830554982 wait complete. 2024-09-05T12:34:20.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 222457 3.46 MB/sec execute 196 sec latency 44.759 ms 2024-09-05T12:34:20.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:20 smithi071 ceph-mon[29845]: pgmap v202: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 801 KiB/s wr, 149 op/s 2024-09-05T12:34:20.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:20 smithi123 ceph-mon[34321]: pgmap v202: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 801 KiB/s wr, 149 op/s 2024-09-05T12:34:20.509 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:20 smithi174 ceph-mon[34313]: pgmap v202: 129 pgs: 129 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 801 KiB/s wr, 149 op/s 2024-09-05T12:34:21.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 223988 3.50 MB/sec execute 197 sec latency 11.058 ms 2024-09-05T12:34:22.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 225664 3.51 MB/sec execute 198 sec latency 19.745 ms 2024-09-05T12:34:22.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:22 smithi071 ceph-mon[29845]: pgmap v203: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 798 KiB/s wr, 132 op/s 2024-09-05T12:34:22.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:22 smithi123 ceph-mon[34321]: pgmap v203: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 798 KiB/s wr, 132 op/s 2024-09-05T12:34:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:22 smithi174 ceph-mon[34313]: pgmap v203: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 796 KiB/s rd, 798 KiB/s wr, 132 op/s 2024-09-05T12:34:23.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 227343 3.56 MB/sec execute 199 sec latency 99.316 ms 2024-09-05T12:34:24.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 228665 3.57 MB/sec execute 200 sec latency 14.937 ms 2024-09-05T12:34:24.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:24 smithi071 ceph-mon[29845]: pgmap v204: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 761 KiB/s wr, 132 op/s 2024-09-05T12:34:24.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:24 smithi123 ceph-mon[34321]: pgmap v204: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 761 KiB/s wr, 132 op/s 2024-09-05T12:34:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:24 smithi174 ceph-mon[34313]: pgmap v204: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 761 KiB/s wr, 132 op/s 2024-09-05T12:34:25.218 INFO:tasks.workunit.client.0.smithi071.stdout: 1 230799 3.62 MB/sec execute 201 sec latency 12.408 ms 2024-09-05T12:34:26.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 231888 3.62 MB/sec execute 202 sec latency 16.193 ms 2024-09-05T12:34:26.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:26 smithi071 ceph-mon[29845]: pgmap v205: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 761 KiB/s wr, 132 op/s 2024-09-05T12:34:26.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:26 smithi123 ceph-mon[34321]: pgmap v205: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 761 KiB/s wr, 132 op/s 2024-09-05T12:34:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:26 smithi174 ceph-mon[34313]: pgmap v205: 129 pgs: 129 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 761 KiB/s wr, 132 op/s 2024-09-05T12:34:27.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 233967 3.66 MB/sec execute 203 sec latency 16.100 ms 2024-09-05T12:34:28.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 235097 3.67 MB/sec execute 204 sec latency 16.005 ms 2024-09-05T12:34:28.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:28 smithi071 ceph-mon[29845]: pgmap v206: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-09-05T12:34:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:28 smithi123 ceph-mon[34321]: pgmap v206: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-09-05T12:34:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:28 smithi174 ceph-mon[34313]: pgmap v206: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 218 op/s 2024-09-05T12:34:29.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 236723 3.68 MB/sec execute 205 sec latency 257.364 ms 2024-09-05T12:34:30.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 238262 3.72 MB/sec execute 206 sec latency 10.876 ms 2024-09-05T12:34:30.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:30 smithi071 ceph-mon[29845]: pgmap v207: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-09-05T12:34:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:30 smithi123 ceph-mon[34321]: pgmap v207: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-09-05T12:34:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:30 smithi174 ceph-mon[34313]: pgmap v207: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 193 op/s 2024-09-05T12:34:31.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 239561 3.73 MB/sec execute 207 sec latency 21.668 ms 2024-09-05T12:34:32.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 241441 3.78 MB/sec execute 208 sec latency 28.594 ms 2024-09-05T12:34:32.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:32 smithi071 ceph-mon[29845]: pgmap v208: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-05T12:34:32.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:32 smithi123 ceph-mon[34321]: pgmap v208: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-05T12:34:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:32 smithi174 ceph-mon[34313]: pgmap v208: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.7 MiB/s wr, 290 op/s 2024-09-05T12:34:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 -- 172.21.15.71:0/1753053954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544810c110 msgr2=0x7f544810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 --2- 172.21.15.71:0/1753053954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544810c110 0x7f544810e560 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5440009ca0 tx=0x7f544002a8b0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 -- 172.21.15.71:0/1753053954 shutdown_connections 2024-09-05T12:34:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 --2- 172.21.15.71:0/1753053954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f544810c110 0x7f544810e560 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 --2- 172.21.15.71:0/1753053954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54481035c0 0x7f544810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 --2- 172.21.15.71:0/1753053954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5448102c10 0x7f5448102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 -- 172.21.15.71:0/1753053954 >> 172.21.15.71:0/1753053954 conn(0x7f54480fc6b0 msgr2=0x7f54480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 -- 172.21.15.71:0/1753053954 shutdown_connections 2024-09-05T12:34:32.833 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.830+0000 7f544fe00640 1 -- 172.21.15.71:0/1753053954 wait complete. 2024-09-05T12:34:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 Processor -- start 2024-09-05T12:34:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 -- start start 2024-09-05T12:34:32.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5448102c10 0x7f54481008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 0x7f5448100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f544810c110 0x7f5448101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5448111f10 con 0x7f544810c110 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5448111d90 con 0x7f54481035c0 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5448112090 con 0x7f5448102c10 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.831+0000 7f544d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5448102c10 0x7f54481008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 0x7f5448100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f544810c110 0x7f5448101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5448102c10 0x7f54481008f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48560/0 (socket says 172.21.15.71:48560) 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 0x7f5448100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54042/0 (socket says 172.21.15.71:54042) 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544d400640 1 -- 172.21.15.71:0/914373670 learned_addr learned my addr 172.21.15.71:0/914373670 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 -- 172.21.15.71:0/914373670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5448102c10 msgr2=0x7f54481008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5448102c10 0x7f54481008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 -- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f544810c110 msgr2=0x7f5448101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f544810c110 0x7f5448101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 -- 172.21.15.71:0/914373670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54481a6780 con 0x7f54481035c0 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544ca00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 0x7f5448100e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f543400e9f0 tx=0x7f543400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f5446000640 1 -- 172.21.15.71:0/914373670 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543400ccc0 con 0x7f54481035c0 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f5446000640 1 -- 172.21.15.71:0/914373670 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5434005020 con 0x7f54481035c0 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54481a6a70 con 0x7f54481035c0 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.832+0000 7f5446000640 1 -- 172.21.15.71:0/914373670 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340049e0 con 0x7f54481035c0 2024-09-05T12:34:32.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.833+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54481a6fc0 con 0x7f54481035c0 2024-09-05T12:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.835+0000 7f5452c84740 1 Processor -- start 2024-09-05T12:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.835+0000 7f5452c84740 1 -- start start 2024-09-05T12:34:32.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5446000640 1 -- 172.21.15.71:0/914373670 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5434005560 con 0x7f54481035c0 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5452c84740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915b1e0 0x55e3b915b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5452c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 0x55e3b9165e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5452c84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b9168790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5452c84740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3b90c1100 con 0x55e3b915b1e0 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5452c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3b90691b0 con 0x55e3b915bb00 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f5452c84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3b90bd0c0 con 0x55e3b91663a0 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 0x55e3b9165e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915b1e0 0x55e3b915b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.838 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 0x55e3b9165e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54046/0 (socket says 172.21.15.71:54046) 2024-09-05T12:34:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544d400640 1 -- 172.21.15.71:0/573760203 learned_addr learned my addr 172.21.15.71:0/573760203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:32.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544ca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b9168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544de00640 1 -- 172.21.15.71:0/573760203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 msgr2=0x55e3b9168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544de00640 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b9168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544de00640 1 -- 172.21.15.71:0/573760203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 msgr2=0x55e3b9165e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544de00640 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 0x55e3b9165e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544de00640 1 -- 172.21.15.71:0/573760203 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b8dd4210 con 0x55e3b915b1e0 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.836+0000 7f544d400640 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 0x55e3b9165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f544de00640 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915b1e0 0x55e3b915b5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5440005e30 tx=0x7f54400093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e309ea8c33e72b7d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f5446000640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54300a2e20 0x7f54300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f543a600640 1 -- 172.21.15.71:0/573760203 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440030760 con 0x55e3b915b1e0 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f5446000640 1 -- 172.21.15.71:0/914373670 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f54340cf6c0 con 0x7f54481035c0 2024-09-05T12:34:32.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f543a600640 1 -- 172.21.15.71:0/573760203 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5440036030 con 0x55e3b915b1e0 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f543a600640 1 -- 172.21.15.71:0/573760203 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440004330 con 0x55e3b915b1e0 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.837+0000 7f544d400640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54300a2e20 0x7f54300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 -- 172.21.15.71:0/573760203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915b1e0 msgr2=0x55e3b915b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915b1e0 0x55e3b915b5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5440005e30 tx=0x7f54400093e0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f544d400640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54300a2e20 0x7f54300a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f543c012640 tx=0x7f543c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 -- 172.21.15.71:0/573760203 shutdown_connections 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b9168790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915bb00 0x55e3b9165e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 --2- 172.21.15.71:0/573760203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915b1e0 0x55e3b915b5c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 -- 172.21.15.71:0/573760203 >> 172.21.15.71:0/573760203 conn(0x55e3b9157750 msgr2=0x55e3b9157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 -- 172.21.15.71:0/573760203 shutdown_connections 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.838+0000 7f5452c84740 1 -- 172.21.15.71:0/573760203 wait complete. 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 -- start start 2024-09-05T12:34:32.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 Processor -- start 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915b1e0 0x55e3b9163330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9163870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b91663a0 0x55e3b9163db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3b9168f70 con 0x55e3b915bb00 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3b904a090 con 0x55e3b91663a0 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.839+0000 7f5452c84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3b91690f0 con 0x55e3b915b1e0 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f5452c84740 10 client.0 _ll_register_callbacks cb 0x55e3b909ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:32.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915b1e0 0x55e3b9163330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915b1e0 0x55e3b9163330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48566/0 (socket says 172.21.15.71:48566) 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544d400640 1 -- 172.21.15.71:0/480651801 learned_addr learned my addr 172.21.15.71:0/480651801 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b91663a0 0x55e3b9163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544ca00640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 -- 172.21.15.71:0/480651801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915b1e0 msgr2=0x55e3b9163330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915b1e0 0x55e3b9163330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 -- 172.21.15.71:0/480651801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 msgr2=0x55e3b9163870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9163870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 -- 172.21.15.71:0/480651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b8ea0620 con 0x55e3b91663a0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f544de00640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b91663a0 0x55e3b9163db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f544002aea0 tx=0x7f5440009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9e7c7b7f86f30c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.840+0000 7f542ea00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f542ea00640 1 -- 172.21.15.71:0/480651801 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440052020 con 0x55e3b91663a0 2024-09-05T12:34:32.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5440036030 con 0x55e3b91663a0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f542ea00640 1 -- 172.21.15.71:0/480651801 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f5440030c00 con 0x55e3b91663a0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f542ea00640 1 -- 172.21.15.71:0/480651801 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f544005d050 con 0x55e3b91663a0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f5452c84740 10 client.26201 fetch_fsmap learned FSMap version 39 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e3b9168f70 con 0x55e3b91663a0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.841+0000 7f542ea00640 1 -- 172.21.15.71:0/480651801 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5440036030 con 0x55e3b91663a0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f542ea00640 1 -- 172.21.15.71:0/480651801 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f5440043070 con 0x55e3b91663a0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f5452c84740 10 client.26201 fetch_fsmap finished waiting for FSMap version 39 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f5452c84740 10 client.26201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f5452c84740 20 client.26201 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f5452c84740 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b9175ca0 0x55e3b9178090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f5452c84740 4 client.26201 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3b90ce830 con 0x55e3b9175ca0 2024-09-05T12:34:32.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.842+0000 7f544d400640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b9175ca0 0x55e3b9178090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.843+0000 7f544d400640 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b9175ca0 0x55e3b9178090 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67ad6dedb7bb33 server_cookie=b73c0af4b0f952f7 in_seq=0 out_seq=0 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.843+0000 7f542ea00640 10 client.26201 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.843+0000 7f542ea00640 1 -- 172.21.15.71:0/480651801 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3b90ce830 con 0x55e3b9175ca0 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.843+0000 7f542ea00640 10 client.26201 handle_command_reply: tid=0 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.843+0000 7f5452c84740 1 client.26201 shutdown 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b91663a0 msgr2=0x55e3b9163db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b91663a0 0x55e3b9163db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f544002aea0 tx=0x7f5440009bf0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 shutdown_connections 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b9175ca0 0x55e3b9178090 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b91663a0 0x55e3b9163db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9163870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f542ea00640 0 client.26201 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 --2- 172.21.15.71:0/480651801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915b1e0 0x55e3b9163330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 >> 172.21.15.71:0/480651801 conn(0x55e3b9157750 msgr2=0x55e3b915d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 shutdown_connections 2024-09-05T12:34:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 1 -- 172.21.15.71:0/480651801 wait complete. 2024-09-05T12:34:32.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.844+0000 7f5452c84740 20 client.26201 trim_cache size 0 max 16384 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 Processor -- start 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 -- start start 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b9163be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9165190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b917d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54481b0f80 con 0x55e3b915bb00 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54481b2b20 con 0x55e3b915b1e0 2024-09-05T12:34:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.861+0000 7f5452c84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54481b1650 con 0x55e3b91663a0 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b9163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b9163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:54056/0 (socket says 172.21.15.71:54056) 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 -- 172.21.15.71:0/863663748 learned_addr learned my addr 172.21.15.71:0/863663748 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544de00640 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b917d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544ca00640 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 -- 172.21.15.71:0/863663748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 msgr2=0x55e3b917d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b917d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 -- 172.21.15.71:0/863663748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 msgr2=0x55e3b9165190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9165190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544d400640 1 -- 172.21.15.71:0/863663748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3b9178620 con 0x55e3b915b1e0 2024-09-05T12:34:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.862+0000 7f544de00640 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b917d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.863+0000 7f544d400640 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b9163be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f543c009f30 tx=0x7f543c01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3de91b2fae4c2eae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.863+0000 7f5439c00640 1 -- 172.21.15.71:0/863663748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c003ed0 con 0x55e3b915b1e0 2024-09-05T12:34:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.863+0000 7f5439c00640 1 -- 172.21.15.71:0/863663748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f543c02b900 con 0x55e3b915b1e0 2024-09-05T12:34:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.863+0000 7f5439c00640 1 -- 172.21.15.71:0/863663748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543c00fd80 con 0x55e3b915b1e0 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 -- 172.21.15.71:0/863663748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 msgr2=0x55e3b9163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b9163be0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f543c009f30 tx=0x7f543c01b000 comp rx=0 tx=0).stop 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 -- 172.21.15.71:0/863663748 shutdown_connections 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b91663a0 0x55e3b917d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b915bb00 0x55e3b9165190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 --2- 172.21.15.71:0/863663748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b9163be0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 -- 172.21.15.71:0/863663748 >> 172.21.15.71:0/863663748 conn(0x55e3b9157750 msgr2=0x55e3b9164a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 -- 172.21.15.71:0/863663748 shutdown_connections 2024-09-05T12:34:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 -- 172.21.15.71:0/863663748 wait complete. 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.864+0000 7f5452c84740 1 -- start start 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 Processor -- start 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b916e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 0x55e3b915ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b91663a0 0x55e3b9160520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54481b23f0 con 0x55e3b91663a0 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54481b1d20 con 0x55e3b915b1e0 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54481aef80 con 0x55e3b915bb00 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f5452c84740 10 client.0 _ll_register_callbacks cb 0x7f543c009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f544ca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 0x55e3b915ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f544d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b916e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f544ca00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 0x55e3b915ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48582/0 (socket says 172.21.15.71:48582) 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f544ca00640 1 -- 172.21.15.71:0/2285994935 learned_addr learned my addr 172.21.15.71:0/2285994935 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.865+0000 7f544de00640 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b91663a0 0x55e3b9160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f544ca00640 1 -- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 msgr2=0x55e3b916e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f544ca00640 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b916e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f544ca00640 1 -- 172.21.15.71:0/2285994935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b91663a0 msgr2=0x55e3b9160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f544ca00640 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b91663a0 0x55e3b9160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f544ca00640 1 -- 172.21.15.71:0/2285994935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3b9160c90 con 0x55e3b915bb00 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f544ca00640 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 0x55e3b915ffe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f54340cca80 tx=0x7f543409b5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12584200774d414 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f542ea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f542ea00640 1 -- 172.21.15.71:0/2285994935 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54340ce070 con 0x55e3b915bb00 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f543c02b900 con 0x55e3b915bb00 2024-09-05T12:34:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.866+0000 7f542ea00640 1 -- 172.21.15.71:0/2285994935 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f54340cf290 con 0x55e3b915bb00 2024-09-05T12:34:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f542ea00640 1 -- 172.21.15.71:0/2285994935 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f543401c040 con 0x55e3b915bb00 2024-09-05T12:34:32.871 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 10 client.26233 fetch_fsmap learned FSMap version 39 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e3b91e0870 con 0x55e3b915bb00 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f542ea00640 1 -- 172.21.15.71:0/2285994935 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f543c02b900 con 0x55e3b915bb00 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f542ea00640 1 -- 172.21.15.71:0/2285994935 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 39) ==== 8378+0+0 (secure 0 0 0) 0x7f54340a3fd0 con 0x55e3b915bb00 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 10 client.26233 fetch_fsmap finished waiting for FSMap version 39 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 20 client.26233 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b91e1000 0x55e3b91e3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 4 client.26233 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.867+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3b9152d80 con 0x55e3b91e1000 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.868+0000 7f544d400640 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b91e1000 0x55e3b91e3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.868+0000 7f544d400640 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b91e1000 0x55e3b91e3420 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf946f2c5605dc8 server_cookie=ebb7908c920b7209 in_seq=0 out_seq=0 2024-09-05T12:34:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.868+0000 7f542ea00640 10 client.26233 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f542ea00640 1 -- 172.21.15.71:0/2285994935 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3b9152d80 con 0x55e3b91e1000 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f542ea00640 10 client.26233 handle_command_reply: tid=0 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 client.26233 shutdown 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 msgr2=0x55e3b915ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 0x55e3b915ffe0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f54340cca80 tx=0x7f543409b5c0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 shutdown_connections 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3b91e1000 0x55e3b91e3420 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f542ea00640 0 client.26233 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3b91663a0 0x55e3b9160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3b915bb00 0x55e3b915ffe0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.932+0000 7f5452c84740 1 --2- 172.21.15.71:0/2285994935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3b915b1e0 0x55e3b916e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.933+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 >> 172.21.15.71:0/2285994935 conn(0x55e3b9157750 msgr2=0x55e3b915cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.933+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 shutdown_connections 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.933+0000 7f5452c84740 1 -- 172.21.15.71:0/2285994935 wait complete. 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.933+0000 7f5452c84740 20 client.26233 trim_cache size 0 max 16384 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54300a2e20 msgr2=0x7f54300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54300a2e20 0x7f54300a52e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f543c012640 tx=0x7f543c005f90 comp rx=0 tx=0).stop 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 msgr2=0x7f5448100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 0x7f5448100e30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f543400e9f0 tx=0x7f543400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 shutdown_connections 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54300a2e20 0x7f54300a52e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f544810c110 0x7f5448101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54481035c0 0x7f5448100e30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 --2- 172.21.15.71:0/914373670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5448102c10 0x7f54481008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 >> 172.21.15.71:0/914373670 conn(0x7f54480fc6b0 msgr2=0x7f54480fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 shutdown_connections 2024-09-05T12:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:32.934+0000 7f544fe00640 1 -- 172.21.15.71:0/914373670 wait complete. 2024-09-05T12:34:33.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 242349 3.77 MB/sec execute 209 sec latency 46.710 ms 2024-09-05T12:34:33.312 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:33 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:33.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:33 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:33 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:34.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 244033 3.79 MB/sec execute 210 sec latency 111.016 ms 2024-09-05T12:34:34.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:34 smithi123 ceph-mon[34321]: pgmap v209: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-09-05T12:34:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:34 smithi174 ceph-mon[34313]: pgmap v209: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-09-05T12:34:34.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:34 smithi071 ceph-mon[29845]: pgmap v209: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.3 MiB/s wr, 220 op/s 2024-09-05T12:34:35.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 245453 3.81 MB/sec execute 211 sec latency 16.947 ms 2024-09-05T12:34:36.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 246318 3.82 MB/sec execute 212 sec latency 147.948 ms 2024-09-05T12:34:36.399 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:36 smithi071 ceph-mon[29845]: pgmap v210: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-09-05T12:34:36.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:36 smithi123 ceph-mon[34321]: pgmap v210: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-09-05T12:34:36.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:36 smithi174 ceph-mon[34313]: pgmap v210: 129 pgs: 129 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.2 MiB/s wr, 208 op/s 2024-09-05T12:34:37.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 247291 3.81 MB/sec execute 213 sec latency 332.380 ms 2024-09-05T12:34:38.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 248764 3.85 MB/sec execute 214 sec latency 293.119 ms 2024-09-05T12:34:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:38 smithi174 ceph-mon[34313]: pgmap v211: 129 pgs: 129 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.8 MiB/s wr, 340 op/s 2024-09-05T12:34:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:38 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:38.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:38 smithi071 ceph-mon[29845]: pgmap v211: 129 pgs: 129 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.8 MiB/s wr, 340 op/s 2024-09-05T12:34:38.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:38 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:38 smithi123 ceph-mon[34321]: pgmap v211: 129 pgs: 129 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.8 MiB/s wr, 340 op/s 2024-09-05T12:34:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:38 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:34:39.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 249561 3.84 MB/sec execute 215 sec latency 381.499 ms 2024-09-05T12:34:40.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 251523 3.88 MB/sec execute 216 sec latency 39.343 ms 2024-09-05T12:34:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:40 smithi174 ceph-mon[34313]: pgmap v212: 129 pgs: 129 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 241 op/s 2024-09-05T12:34:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:40 smithi071 ceph-mon[29845]: pgmap v212: 129 pgs: 129 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 241 op/s 2024-09-05T12:34:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:40 smithi123 ceph-mon[34321]: pgmap v212: 129 pgs: 129 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.3 MiB/s wr, 241 op/s 2024-09-05T12:34:41.219 INFO:tasks.workunit.client.0.smithi071.stdout: 1 252612 3.89 MB/sec execute 217 sec latency 39.444 ms 2024-09-05T12:34:42.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 252660 3.87 MB/sec execute 218 sec latency 890.117 ms 2024-09-05T12:34:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:42 smithi174 ceph-mon[34313]: pgmap v213: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-09-05T12:34:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:42 smithi071 ceph-mon[29845]: pgmap v213: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-09-05T12:34:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:42 smithi123 ceph-mon[34321]: pgmap v213: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.9 MiB/s wr, 362 op/s 2024-09-05T12:34:43.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 253572 3.87 MB/sec execute 219 sec latency 1114.950 ms 2024-09-05T12:34:44.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 253572 3.86 MB/sec execute 220 sec latency 1093.692 ms 2024-09-05T12:34:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:44 smithi174 ceph-mon[34313]: pgmap v214: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-09-05T12:34:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:44 smithi071 ceph-mon[29845]: pgmap v214: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-09-05T12:34:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:44 smithi123 ceph-mon[34321]: pgmap v214: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 275 op/s 2024-09-05T12:34:45.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 253572 3.84 MB/sec execute 221 sec latency 2093.829 ms 2024-09-05T12:34:46.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 -- 172.21.15.71:0/3733275358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 msgr2=0x7fd88810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 --2- 172.21.15.71:0/3733275358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88810e9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd880009cd0 tx=0x7fd88002a8f0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 -- 172.21.15.71:0/3733275358 shutdown_connections 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 --2- 172.21.15.71:0/3733275358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88810e9b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 --2- 172.21.15.71:0/3733275358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd888101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 --2- 172.21.15.71:0/3733275358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd888107b80 0x7fd888107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 -- 172.21.15.71:0/3733275358 >> 172.21.15.71:0/3733275358 conn(0x7fd8880fd7e0 msgr2=0x7fd8880ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 -- 172.21.15.71:0/3733275358 shutdown_connections 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.059+0000 7fd88e600640 1 -- 172.21.15.71:0/3733275358 wait complete. 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 Processor -- start 2024-09-05T12:34:46.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 -- start start 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd88819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd888107b80 0x7fd88819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd888111f80 con 0x7fd888102480 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd888111e00 con 0x7fd888101ae0 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.060+0000 7fd88e600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd888112100 con 0x7fd888107b80 2024-09-05T12:34:46.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd87ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd88819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd87ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88819aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57922/0 (socket says 172.21.15.71:57922) 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd88819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:41558/0 (socket says 172.21.15.71:41558) 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd888107b80 0x7fd88819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd87ea00640 1 -- 172.21.15.71:0/934436209 learned_addr learned my addr 172.21.15.71:0/934436209 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 -- 172.21.15.71:0/934436209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd888107b80 msgr2=0x7fd88819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd888107b80 0x7fd88819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 -- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 msgr2=0x7fd88819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 -- 172.21.15.71:0/934436209 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88819f540 con 0x7fd888101ae0 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd87ea00640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.061+0000 7fd887400640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd88819a550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd87800e9f0 tx=0x7fd87800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:46.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.062+0000 7fd884c00640 1 -- 172.21.15.71:0/934436209 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87800ccc0 con 0x7fd888101ae0 2024-09-05T12:34:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.062+0000 7fd884c00640 1 -- 172.21.15.71:0/934436209 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd878005020 con 0x7fd888101ae0 2024-09-05T12:34:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.062+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88819f830 con 0x7fd888101ae0 2024-09-05T12:34:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.062+0000 7fd884c00640 1 -- 172.21.15.71:0/934436209 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8780049e0 con 0x7fd888101ae0 2024-09-05T12:34:46.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.062+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8881a70c0 con 0x7fd888101ae0 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 Processor -- start 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 -- start start 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd884c00640 1 -- 172.21.15.71:0/934436209 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd878005560 con 0x7fd888101ae0 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56472181d100 con 0x5647218b7b00 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5647217c51b0 con 0x5647218b71e0 2024-09-05T12:34:46.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd8913bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647218190c0 con 0x5647218c23a0 2024-09-05T12:34:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd887e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd887400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.065+0000 7fd887e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:41562/0 (socket says 172.21.15.71:41562) 2024-09-05T12:34:46.068 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 -- 172.21.15.71:0/4110559790 learned_addr learned my addr 172.21.15.71:0/4110559790 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd87ea00640 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 -- 172.21.15.71:0/4110559790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 msgr2=0x5647218c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 -- 172.21.15.71:0/4110559790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 msgr2=0x5647218c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 -- 172.21.15.71:0/4110559790 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564721530210 con 0x5647218b71e0 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd87ea00640 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd887e00640 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218b75c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd880009cd0 tx=0x7fd880005f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed34e7f524013145 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd884c00640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd86c0a2da0 0x7fd86c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.066+0000 7fd87cc00640 1 -- 172.21.15.71:0/4110559790 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880030760 con 0x5647218b71e0 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd87cc00640 1 -- 172.21.15.71:0/4110559790 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880036030 con 0x5647218b71e0 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd87cc00640 1 -- 172.21.15.71:0/4110559790 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800043a0 con 0x5647218b71e0 2024-09-05T12:34:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd884c00640 1 -- 172.21.15.71:0/934436209 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fd8780cf710 con 0x7fd888101ae0 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd887400640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd86c0a2da0 0x7fd86c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 -- 172.21.15.71:0/4110559790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 msgr2=0x5647218b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218b75c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd880009cd0 tx=0x7fd880005f80 comp rx=0 tx=0).stop 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd887400640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd86c0a2da0 0x7fd86c0a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8780ccad0 tx=0x7fd8780d5530 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 -- 172.21.15.71:0/4110559790 shutdown_connections 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4110559790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218b75c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 -- 172.21.15.71:0/4110559790 >> 172.21.15.71:0/4110559790 conn(0x5647218b3750 msgr2=0x5647218b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.067+0000 7fd8913bf740 1 -- 172.21.15.71:0/4110559790 shutdown_connections 2024-09-05T12:34:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.068+0000 7fd8913bf740 1 -- 172.21.15.71:0/4110559790 wait complete. 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.068+0000 7fd8913bf740 1 -- start start 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.068+0000 7fd8913bf740 1 Processor -- start 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218bf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b7b00 0x5647218bf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5647218c4f70 con 0x5647218c23a0 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5647217a6090 con 0x5647218b71e0 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647218c50f0 con 0x5647218b7b00 2024-09-05T12:34:46.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd8913bf740 10 client.0 _ll_register_callbacks cb 0x5647217faf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd887e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd887e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57934/0 (socket says 172.21.15.71:57934) 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd887e00640 1 -- 172.21.15.71:0/2176877295 learned_addr learned my addr 172.21.15.71:0/2176877295 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd87ea00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b7b00 0x5647218bf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd887e00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218bf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd87ea00640 1 -- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 msgr2=0x5647218bf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd87ea00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218bf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd87ea00640 1 -- 172.21.15.71:0/2176877295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 msgr2=0x5647218bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd87ea00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd87ea00640 1 -- 172.21.15.71:0/2176877295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647215fc620 con 0x5647218b7b00 2024-09-05T12:34:46.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.069+0000 7fd887e00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd87ea00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b7b00 0x5647218bf8e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd870009fd0 tx=0x7fd870012e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4495f88a91b3d13 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd875600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd875600640 1 -- 172.21.15.71:0/2176877295 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87000fa00 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd880036030 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd875600640 1 -- 172.21.15.71:0/2176877295 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fd87000fba0 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd875600640 1 -- 172.21.15.71:0/2176877295 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd870024050 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd8913bf740 10 client.26245 fetch_fsmap learned FSMap version 40 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5647218d0c40 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.070+0000 7fd875600640 1 -- 172.21.15.71:0/2176877295 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880036030 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd875600640 1 -- 172.21.15.71:0/2176877295 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fd870010430 con 0x5647218b7b00 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd8913bf740 10 client.26245 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd8913bf740 10 client.26245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:46.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd8913bf740 20 client.26245 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd8913bf740 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5647218d21d0 0x5647218d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd8913bf740 4 client.26245 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56472182a830 con 0x5647218d21d0 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.071+0000 7fd887e00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5647218d21d0 0x5647218d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.072+0000 7fd887e00640 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5647218d21d0 0x5647218d45f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2599673460180f server_cookie=6e358eecc3a00f27 in_seq=0 out_seq=0 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.072+0000 7fd875600640 10 client.26245 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd875600640 1 -- 172.21.15.71:0/2176877295 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56472182a830 con 0x5647218d21d0 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd875600640 10 client.26245 handle_command_reply: tid=0 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 client.26245 shutdown 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b7b00 msgr2=0x5647218bf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b7b00 0x5647218bf8e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd870009fd0 tx=0x7fd870012e70 comp rx=0 tx=0).stop 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 shutdown_connections 2024-09-05T12:34:46.078 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5647218d21d0 0x5647218d45f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd875600640 0 client.26245 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b7b00 0x5647218bf8e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 --2- 172.21.15.71:0/2176877295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218bf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 >> 172.21.15.71:0/2176877295 conn(0x5647218b3750 msgr2=0x5647218b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 shutdown_connections 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 1 -- 172.21.15.71:0/2176877295 wait complete. 2024-09-05T12:34:46.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.073+0000 7fd8913bf740 20 client.26245 trim_cache size 0 max 16384 2024-09-05T12:34:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.088+0000 7fd8913bf740 1 Processor -- start 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 -- start start 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b71e0 0x5647218d9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b7b00 0x5647218da3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881b10e0 con 0x5647218c23a0 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881b2ce0 con 0x5647218b7b00 2024-09-05T12:34:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd8913bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881b17b0 con 0x5647218b71e0 2024-09-05T12:34:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd887e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd887e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57944/0 (socket says 172.21.15.71:57944) 2024-09-05T12:34:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd887e00640 1 -- 172.21.15.71:0/1145569929 learned_addr learned my addr 172.21.15.71:0/1145569929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd887e00640 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b71e0 0x5647218d9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.089+0000 7fd87ea00640 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b7b00 0x5647218da3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd887e00640 1 -- 172.21.15.71:0/1145569929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b71e0 msgr2=0x5647218d9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd887e00640 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b71e0 0x5647218d9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd887e00640 1 -- 172.21.15.71:0/1145569929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b7b00 msgr2=0x5647218da3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd887e00640 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b7b00 0x5647218da3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd887e00640 1 -- 172.21.15.71:0/1145569929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647218d0a50 con 0x5647218c23a0 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd87ea00640 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b7b00 0x5647218da3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd887e00640 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218c0af0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd880030f30 tx=0x7fd880005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e6b082a3027452 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd877e00640 1 -- 172.21.15.71:0/1145569929 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880003970 con 0x5647218c23a0 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd877e00640 1 -- 172.21.15.71:0/1145569929 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880036070 con 0x5647218c23a0 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.090+0000 7fd877e00640 1 -- 172.21.15.71:0/1145569929 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800316a0 con 0x5647218c23a0 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 -- 172.21.15.71:0/1145569929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 msgr2=0x5647218c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218c0af0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd880030f30 tx=0x7fd880005d80 comp rx=0 tx=0).stop 2024-09-05T12:34:46.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 -- 172.21.15.71:0/1145569929 shutdown_connections 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218c23a0 0x5647218c0af0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b7b00 0x5647218da3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 --2- 172.21.15.71:0/1145569929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218b71e0 0x5647218d9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 -- 172.21.15.71:0/1145569929 >> 172.21.15.71:0/1145569929 conn(0x5647218b3750 msgr2=0x5647218bdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 -- 172.21.15.71:0/1145569929 shutdown_connections 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 -- 172.21.15.71:0/1145569929 wait complete. 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 -- start start 2024-09-05T12:34:46.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.091+0000 7fd8913bf740 1 Processor -- start 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218bc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881b2550 con 0x5647218b7b00 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881b1e80 con 0x5647218b71e0 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8881af0e0 con 0x5647218c23a0 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd8913bf740 10 client.0 _ll_register_callbacks cb 0x7fd870009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd87ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd87ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218bbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57952/0 (socket says 172.21.15.71:57952) 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd87ea00640 1 -- 172.21.15.71:0/4219604595 learned_addr learned my addr 172.21.15.71:0/4219604595 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd887e00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218bc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.092+0000 7fd887e00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd87ea00640 1 -- 172.21.15.71:0/4219604595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 msgr2=0x5647218bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd87ea00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218bc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd87ea00640 1 -- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 msgr2=0x5647218ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd87ea00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd87ea00640 1 -- 172.21.15.71:0/4219604595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647218bc8a0 con 0x5647218b7b00 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd887e00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd87ea00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218bbe20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd870010850 tx=0x7fd870005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0311866adcabd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd875600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd875600640 1 -- 172.21.15.71:0/4219604595 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870010430 con 0x5647218b7b00 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd875600640 1 -- 172.21.15.71:0/4219604595 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7fd8700105d0 con 0x5647218b7b00 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd880036070 con 0x5647218b7b00 2024-09-05T12:34:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd875600640 1 -- 172.21.15.71:0/4219604595 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd870035050 con 0x5647218b7b00 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd8913bf740 10 client.16197 fetch_fsmap learned FSMap version 40 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.093+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56472193cac0 con 0x5647218b7b00 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd875600640 1 -- 172.21.15.71:0/4219604595 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd880036070 con 0x5647218b7b00 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd875600640 1 -- 172.21.15.71:0/4219604595 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fd8700036f0 con 0x5647218b7b00 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd8913bf740 10 client.16197 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd8913bf740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd8913bf740 20 client.16197 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56472193d340 0x56472193f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd8913bf740 4 client.16197 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564721824b80 con 0x56472193d340 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.094+0000 7fd887e00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56472193d340 0x56472193f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.095+0000 7fd887e00640 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56472193d340 0x56472193f760 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec8392c4b226bf0a server_cookie=878e0d2bdfcf67a9 in_seq=0 out_seq=0 2024-09-05T12:34:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.095+0000 7fd875600640 10 client.16197 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.135+0000 7fd875600640 1 -- 172.21.15.71:0/4219604595 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564721824b80 con 0x56472193d340 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.135+0000 7fd875600640 10 client.16197 handle_command_reply: tid=0 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.135+0000 7fd8913bf740 1 client.16197 shutdown 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 msgr2=0x5647218bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218bbe20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd870010850 tx=0x7fd870005d60 comp rx=0 tx=0).stop 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 shutdown_connections 2024-09-05T12:34:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56472193d340 0x56472193f760 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647218c23a0 0x5647218bc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647218b7b00 0x5647218bbe20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 --2- 172.21.15.71:0/4219604595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647218b71e0 0x5647218ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd875600640 0 client.16197 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 >> 172.21.15.71:0/4219604595 conn(0x5647218b3750 msgr2=0x5647218b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 shutdown_connections 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 1 -- 172.21.15.71:0/4219604595 wait complete. 2024-09-05T12:34:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.136+0000 7fd8913bf740 20 client.16197 trim_cache size 0 max 16384 2024-09-05T12:34:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:34:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:34:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:34:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:34:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd86c0a2da0 msgr2=0x7fd86c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd86c0a2da0 0x7fd86c0a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8780ccad0 tx=0x7fd8780d5530 comp rx=0 tx=0).stop 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 msgr2=0x7fd88819a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd88819a550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd87800e9f0 tx=0x7fd87800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 shutdown_connections 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd888107b80 0x7fd88819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd86c0a2da0 0x7fd86c0a5260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd888102480 0x7fd88819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 --2- 172.21.15.71:0/934436209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd888101ae0 0x7fd88819a550 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 >> 172.21.15.71:0/934436209 conn(0x7fd8880fd7e0 msgr2=0x7fd8880fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 shutdown_connections 2024-09-05T12:34:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:46.137+0000 7fd88e600640 1 -- 172.21.15.71:0/934436209 wait complete. 2024-09-05T12:34:46.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 253572 3.82 MB/sec execute 222 sec latency 3093.957 ms 2024-09-05T12:34:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:46 smithi174 ceph-mon[34313]: pgmap v215: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:34:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:46 smithi071 ceph-mon[29845]: pgmap v215: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:34:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:46 smithi123 ceph-mon[34321]: pgmap v215: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:34:47.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 255429 3.86 MB/sec execute 223 sec latency 3252.148 ms 2024-09-05T12:34:48.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 256575 3.86 MB/sec execute 224 sec latency 17.854 ms 2024-09-05T12:34:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:48 smithi174 ceph-mon[34313]: pgmap v216: 129 pgs: 129 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-09-05T12:34:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:48 smithi071 ceph-mon[29845]: pgmap v216: 129 pgs: 129 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-09-05T12:34:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:48 smithi123 ceph-mon[34321]: pgmap v216: 129 pgs: 129 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 318 op/s 2024-09-05T12:34:49.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 257113 3.85 MB/sec execute 225 sec latency 518.760 ms 2024-09-05T12:34:50.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 257113 3.84 MB/sec execute 226 sec latency 1518.939 ms 2024-09-05T12:34:50.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:50 smithi071 ceph-mon[29845]: pgmap v217: 129 pgs: 129 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-09-05T12:34:50.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:50 smithi123 ceph-mon[34321]: pgmap v217: 129 pgs: 129 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-09-05T12:34:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:50 smithi174 ceph-mon[34313]: pgmap v217: 129 pgs: 129 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 186 op/s 2024-09-05T12:34:51.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 257113 3.82 MB/sec execute 227 sec latency 2519.014 ms 2024-09-05T12:34:52.220 INFO:tasks.workunit.client.0.smithi071.stdout: 1 257232 3.80 MB/sec execute 228 sec latency 2678.016 ms 2024-09-05T12:34:52.493 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:52 smithi071 ceph-mon[29845]: pgmap v218: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-09-05T12:34:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:52 smithi174 ceph-mon[34313]: pgmap v218: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-09-05T12:34:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:52 smithi123 ceph-mon[34321]: pgmap v218: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 247 op/s 2024-09-05T12:34:53.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 257967 3.80 MB/sec execute 229 sec latency 1288.697 ms 2024-09-05T12:34:54.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 259474 3.83 MB/sec execute 230 sec latency 14.557 ms 2024-09-05T12:34:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:54 smithi174 ceph-mon[34313]: pgmap v219: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T12:34:54.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:54 smithi071 ceph-mon[29845]: pgmap v219: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T12:34:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:54 smithi123 ceph-mon[34321]: pgmap v219: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 134 op/s 2024-09-05T12:34:55.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 260661 3.84 MB/sec execute 231 sec latency 219.798 ms 2024-09-05T12:34:56.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 260661 3.82 MB/sec execute 232 sec latency 1219.909 ms 2024-09-05T12:34:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:56 smithi174 ceph-mon[34313]: pgmap v220: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1004 KiB/s wr, 123 op/s 2024-09-05T12:34:56.574 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:56 smithi071 ceph-mon[29845]: pgmap v220: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1004 KiB/s wr, 123 op/s 2024-09-05T12:34:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:56 smithi123 ceph-mon[34321]: pgmap v220: 129 pgs: 129 active+clean; 86 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1004 KiB/s wr, 123 op/s 2024-09-05T12:34:57.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 261110 3.81 MB/sec execute 233 sec latency 1377.636 ms 2024-09-05T12:34:58.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 261110 3.79 MB/sec execute 234 sec latency 1658.736 ms 2024-09-05T12:34:58.258 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:58 smithi071 ceph-mon[29845]: pgmap v221: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-05T12:34:58.258 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:58 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:34:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:58 smithi174 ceph-mon[34313]: pgmap v221: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-05T12:34:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:58 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:34:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:58 smithi123 ceph-mon[34321]: pgmap v221: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-09-05T12:34:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:58 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:34:59.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 262344 3.81 MB/sec execute 235 sec latency 2006.188 ms 2024-09-05T12:34:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 -- 172.21.15.71:0/1871158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 msgr2=0x7f77d810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 --2- 172.21.15.71:0/1871158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d810b8e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009860 tx=0x7f77c402a3d0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 -- 172.21.15.71:0/1871158663 shutdown_connections 2024-09-05T12:34:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 --2- 172.21.15.71:0/1871158663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d810bf00 0x7f77d810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 --2- 172.21.15.71:0/1871158663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d810b8e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.266 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 --2- 172.21.15.71:0/1871158663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77d8102ae0 0x7f77d8102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 -- 172.21.15.71:0/1871158663 >> 172.21.15.71:0/1871158663 conn(0x7f77d80777c0 msgr2=0x7f77d80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 -- 172.21.15.71:0/1871158663 shutdown_connections 2024-09-05T12:34:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.265+0000 7f77dd600640 1 -- 172.21.15.71:0/1871158663 wait complete. 2024-09-05T12:34:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.266+0000 7f77dd600640 1 Processor -- start 2024-09-05T12:34:59.267 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 -- start start 2024-09-05T12:34:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d8102ae0 0x7f77d819a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d819a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77d810bf00 0x7f77d819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.268 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8111d20 con 0x7f77d810bf00 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8111ba0 con 0x7f77d8102ae0 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dd600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77d8111ea0 con 0x7f77d8103400 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77d7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d819a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77d7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d819a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:56072/0 (socket says 172.21.15.71:56072) 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77d7400640 1 -- 172.21.15.71:0/4033166240 learned_addr learned my addr 172.21.15.71:0/4033166240 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77d7e00640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d8102ae0 0x7f77d819a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.269 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.267+0000 7f77dcc00640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77d810bf00 0x7f77d819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7400640 1 -- 172.21.15.71:0/4033166240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d8102ae0 msgr2=0x7f77d819a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7400640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d8102ae0 0x7f77d819a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7400640 1 -- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77d810bf00 msgr2=0x7f77d819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7400640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77d810bf00 0x7f77d819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.270 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7400640 1 -- 172.21.15.71:0/4033166240 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d819f420 con 0x7f77d8103400 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7e00640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d8102ae0 0x7f77d819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.268+0000 7f77d7400640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d819a970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009980 tx=0x7f77c40309a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.269+0000 7f77d4c00640 1 -- 172.21.15.71:0/4033166240 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4040070 con 0x7f77d8103400 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.269+0000 7f77d4c00640 1 -- 172.21.15.71:0/4033166240 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c4005620 con 0x7f77d8103400 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.269+0000 7f77d4c00640 1 -- 172.21.15.71:0/4033166240 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004920 con 0x7f77d8103400 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.269+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d819f6b0 con 0x7f77d8103400 2024-09-05T12:34:59.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.269+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d81a6fa0 con 0x7f77d8103400 2024-09-05T12:34:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.272+0000 7f77e19d2740 1 Processor -- start 2024-09-05T12:34:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.272+0000 7f77e19d2740 1 -- start start 2024-09-05T12:34:59.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77d4c00640 1 -- 172.21.15.71:0/4033166240 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f77c4043070 con 0x7f77d8103400 2024-09-05T12:34:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc3b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc46030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc48960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77e19d2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5615dfba1100 con 0x5615dfc3bcd0 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77e19d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615dfb491b0 con 0x5615dfc46570 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77e19d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615dfb9d0c0 con 0x5615dfc3b3b0 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc3b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77d7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc48960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc3b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:56074/0 (socket says 172.21.15.71:56074) 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77dcc00640 1 -- 172.21.15.71:0/1598169233 learned_addr learned my addr 172.21.15.71:0/1598169233 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.273+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc46030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77dcc00640 1 -- 172.21.15.71:0/1598169233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 msgr2=0x5615dfc48960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77dcc00640 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc48960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77dcc00640 1 -- 172.21.15.71:0/1598169233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 msgr2=0x5615dfc46030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77dcc00640 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc46030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77dcc00640 1 -- 172.21.15.71:0/1598169233 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615df8b4210 con 0x5615dfc3b3b0 2024-09-05T12:34:59.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77d7400640 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc48960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:34:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77dcc00640 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc3b790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009f50 tx=0x7f77d00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8af806eb177be50 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77d4c00640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f77bc0a2d70 0x7f77bc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77cd200640 1 -- 172.21.15.71:0/1598169233 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0004d20 con 0x5615dfc3b3b0 2024-09-05T12:34:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77cd200640 1 -- 172.21.15.71:0/1598169233 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0004050 con 0x5615dfc3b3b0 2024-09-05T12:34:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.274+0000 7f77cd200640 1 -- 172.21.15.71:0/1598169233 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0005280 con 0x5615dfc3b3b0 2024-09-05T12:34:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77d4c00640 1 -- 172.21.15.71:0/4033166240 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f77c40f0440 con 0x7f77d8103400 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77d7e00640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f77bc0a2d70 0x7f77bc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 -- 172.21.15.71:0/1598169233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 msgr2=0x5615dfc3b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc3b790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009f50 tx=0x7f77d00118d0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77d7e00640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f77bc0a2d70 0x7f77bc0a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005fd0 tx=0x7f77c8009260 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 -- 172.21.15.71:0/1598169233 shutdown_connections 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc48960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc46030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1598169233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc3b790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.275+0000 7f77e19d2740 1 -- 172.21.15.71:0/1598169233 >> 172.21.15.71:0/1598169233 conn(0x5615dfc37920 msgr2=0x5615dfc37d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.276+0000 7f77e19d2740 1 -- 172.21.15.71:0/1598169233 shutdown_connections 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.276+0000 7f77e19d2740 1 -- 172.21.15.71:0/1598169233 wait complete. 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.276+0000 7f77e19d2740 1 -- start start 2024-09-05T12:34:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.276+0000 7f77e19d2740 1 Processor -- start 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc436a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc43be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc44120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5615dfc48ff0 con 0x5615dfc3bcd0 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615dfb2a090 con 0x5615dfc46570 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615dfc49170 con 0x5615dfc3b3b0 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77e19d2740 10 client.0 _ll_register_callbacks cb 0x5615dfb7ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc43be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc44120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc43be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53144/0 (socket says 172.21.15.71:53144) 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77dcc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc44120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:47132/0 (socket says 172.21.15.71:47132) 2024-09-05T12:34:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 -- 172.21.15.71:0/1528887005 learned_addr learned my addr 172.21.15.71:0/1528887005 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 -- 172.21.15.71:0/1528887005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 msgr2=0x5615dfc436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 -- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 msgr2=0x5615dfc44120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc44120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.277+0000 7f77d7400640 1 -- 172.21.15.71:0/1528887005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615df980620 con 0x5615dfc3bcd0 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77d7400640 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc43be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ed800 tx=0x7f77c40b8210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a74a293d4eb24c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77c1600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:34:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77c1600640 1 -- 172.21.15.71:0/1528887005 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c404a040 con 0x5615dfc3bcd0 2024-09-05T12:34:59.283 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77c1600640 1 -- 172.21.15.71:0/1528887005 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f77c40f0160 con 0x5615dfc3bcd0 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77d0004050 con 0x5615dfc3bcd0 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77c1600640 1 -- 172.21.15.71:0/1528887005 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f77c40bc340 con 0x5615dfc3bcd0 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77e19d2740 10 client.16209 fetch_fsmap learned FSMap version 40 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5615dfc54dd0 con 0x5615dfc3bcd0 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.278+0000 7f77c1600640 1 -- 172.21.15.71:0/1528887005 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0004050 con 0x5615dfc3bcd0 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77c1600640 1 -- 172.21.15.71:0/1528887005 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7f77c4031590 con 0x5615dfc3bcd0 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77e19d2740 10 client.16209 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77e19d2740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77e19d2740 20 client.16209 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfc562e0 0x5615dfc58700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77e19d2740 4 client.16209 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:34:59.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615dfbae830 con 0x5615dfc562e0 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.279+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfc562e0 0x5615dfc58700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.280+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfc562e0 0x5615dfc58700 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb0f407380b4d1 server_cookie=4b4acedc0b7bb36c in_seq=0 out_seq=0 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.280+0000 7f77c1600640 10 client.16209 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77c1600640 1 -- 172.21.15.71:0/1528887005 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615dfbae830 con 0x5615dfc562e0 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77c1600640 10 client.16209 handle_command_reply: tid=0 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 client.16209 shutdown 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 msgr2=0x5615dfc43be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc43be0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f77c40ed800 tx=0x7f77c40b8210 comp rx=0 tx=0).stop 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 shutdown_connections 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfc562e0 0x5615dfc58700 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc46570 0x5615dfc44120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.285 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc3bcd0 0x5615dfc43be0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1528887005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3b3b0 0x5615dfc436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 >> 172.21.15.71:0/1528887005 conn(0x5615dfc37920 msgr2=0x5615dfc3d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77c1600640 0 client.16209 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.281+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 shutdown_connections 2024-09-05T12:34:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.282+0000 7f77e19d2740 1 -- 172.21.15.71:0/1528887005 wait complete. 2024-09-05T12:34:59.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.282+0000 7f77e19d2740 20 client.16209 trim_cache size 0 max 16384 2024-09-05T12:34:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.299+0000 7f77e19d2740 1 Processor -- start 2024-09-05T12:34:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.299+0000 7f77e19d2740 1 -- start start 2024-09-05T12:34:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.299+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc5dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.299+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc5e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.299+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77e19d2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81b0fd0 con 0x5615dfc46570 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77e19d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81b2bd0 con 0x5615dfc3b3b0 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77e19d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81b16a0 con 0x5615dfc3bcd0 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc5dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc5dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:47136/0 (socket says 172.21.15.71:47136) 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 -- 172.21.15.71:0/1096234626 learned_addr learned my addr 172.21.15.71:0/1096234626 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7400640 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc5e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77dcc00640 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 -- 172.21.15.71:0/1096234626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 msgr2=0x5615dfc5e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc5e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 -- 172.21.15.71:0/1096234626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 msgr2=0x5615dfc3ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77d7e00640 1 -- 172.21.15.71:0/1096234626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615dfc46150 con 0x5615dfc3b3b0 2024-09-05T12:34:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.300+0000 7f77dcc00640 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:34:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.301+0000 7f77d7e00640 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc5dfd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5615dfc58f40 tx=0x7f77c801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=431428a834a303f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.301+0000 7f77c3e00640 1 -- 172.21.15.71:0/1096234626 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c800fd20 con 0x5615dfc3b3b0 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.301+0000 7f77c3e00640 1 -- 172.21.15.71:0/1096234626 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c802a900 con 0x5615dfc3b3b0 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.301+0000 7f77c3e00640 1 -- 172.21.15.71:0/1096234626 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8003cf0 con 0x5615dfc3b3b0 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 -- 172.21.15.71:0/1096234626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 msgr2=0x5615dfc5dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc5dfd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5615dfc58f40 tx=0x7f77c801b000 comp rx=0 tx=0).stop 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 -- 172.21.15.71:0/1096234626 shutdown_connections 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc5e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 --2- 172.21.15.71:0/1096234626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc5dfd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 -- 172.21.15.71:0/1096234626 >> 172.21.15.71:0/1096234626 conn(0x5615dfc37920 msgr2=0x5615dfc41f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 -- 172.21.15.71:0/1096234626 shutdown_connections 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 -- 172.21.15.71:0/1096234626 wait complete. 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 -- start start 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.302+0000 7f77e19d2740 1 Processor -- start 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc4e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc4eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81b2440 con 0x5615dfc46570 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81b1d70 con 0x5615dfc3b3b0 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f77d81aefd0 con 0x5615dfc3bcd0 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77e19d2740 10 client.0 _ll_register_callbacks cb 0x7f77c40097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:34:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc4e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77d7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc4eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc4e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:47146/0 (socket says 172.21.15.71:47146) 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77d7e00640 1 -- 172.21.15.71:0/2226618325 learned_addr learned my addr 172.21.15.71:0/2226618325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.303+0000 7f77dcc00640 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77d7400640 1 -- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 msgr2=0x5615dfc4e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77d7400640 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc4e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77d7400640 1 -- 172.21.15.71:0/2226618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 msgr2=0x5615dfc3ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77d7400640 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77d7400640 1 -- 172.21.15.71:0/2226618325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615dfc40600 con 0x5615dfc3bcd0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77d7400640 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc4eaf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77c40b89d0 tx=0x7f77c40eda20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e98b685958b5766b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77c1600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77c1600640 1 -- 172.21.15.71:0/2226618325 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c404a040 con 0x5615dfc3bcd0 2024-09-05T12:34:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c802a900 con 0x5615dfc3bcd0 2024-09-05T12:34:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.304+0000 7f77c1600640 1 -- 172.21.15.71:0/2226618325 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9438+0+0 (secure 0 0 0) 0x7f77c40c4f60 con 0x5615dfc3bcd0 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77c1600640 1 -- 172.21.15.71:0/2226618325 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f77c40bbed0 con 0x5615dfc3bcd0 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 10 client.26269 fetch_fsmap learned FSMap version 40 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5615dfc40600 con 0x5615dfc3bcd0 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77c1600640 1 -- 172.21.15.71:0/2226618325 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c802a900 con 0x5615dfc3bcd0 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77c1600640 1 -- 172.21.15.71:0/2226618325 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7f77c40d5260 con 0x5615dfc3bcd0 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 10 client.26269 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 10 client.26269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 20 client.26269 populate_metadata read hostname 'smithi071' 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfcc0dc0 0x5615dfcc31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:34:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 4 client.26269 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:34:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615dfba8b80 con 0x5615dfcc0dc0 2024-09-05T12:34:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.305+0000 7f77d7e00640 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfcc0dc0 0x5615dfcc31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:34:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.306+0000 7f77c1600640 10 client.26269 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.306+0000 7f77d7e00640 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfcc0dc0 0x5615dfcc31e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0c894aefe2c8c8 server_cookie=eefca7ed5efada6e in_seq=0 out_seq=0 2024-09-05T12:34:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77c1600640 1 -- 172.21.15.71:0/2226618325 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615dfba8b80 con 0x5615dfcc0dc0 2024-09-05T12:34:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77c1600640 10 client.26269 handle_command_reply: tid=0 2024-09-05T12:34:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77e19d2740 1 client.26269 shutdown 2024-09-05T12:34:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 msgr2=0x5615dfc4eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77e19d2740 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc4eaf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77c40b89d0 tx=0x7f77c40eda20 comp rx=0 tx=0).stop 2024-09-05T12:34:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 shutdown_connections 2024-09-05T12:34:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.337+0000 7f77e19d2740 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5615dfcc0dc0 0x5615dfcc31e0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5615dfc46570 0x5615dfc3ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615dfc3bcd0 0x5615dfc4eaf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 1 --2- 172.21.15.71:0/2226618325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615dfc3b3b0 0x5615dfc4e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77c1600640 0 client.26269 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 >> 172.21.15.71:0/2226618325 conn(0x5615dfc37920 msgr2=0x5615dfc51240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 shutdown_connections 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 1 -- 172.21.15.71:0/2226618325 wait complete. 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.338+0000 7f77e19d2740 20 client.26269 trim_cache size 0 max 16384 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:34:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f77bc0a2d70 msgr2=0x7f77bc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f77bc0a2d70 0x7f77bc0a5230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f77c8005fd0 tx=0x7f77c8009260 comp rx=0 tx=0).stop 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 msgr2=0x7f77d819a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d819a970 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009980 tx=0x7f77c40309a0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 shutdown_connections 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f77bc0a2d70 0x7f77bc0a5230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77d810bf00 0x7f77d819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f77d8103400 0x7f77d819a970 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 --2- 172.21.15.71:0/4033166240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d8102ae0 0x7f77d819a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:34:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 >> 172.21.15.71:0/4033166240 conn(0x7f77d80777c0 msgr2=0x7f77d81006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:34:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 shutdown_connections 2024-09-05T12:34:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:34:59.339+0000 7f77dd600640 1 -- 172.21.15.71:0/4033166240 wait complete. 2024-09-05T12:34:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:59 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:34:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:59 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:34:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:34:59 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:34:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:59 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:34:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:59 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:34:59.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:34:59 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:34:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:59 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:34:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:59 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:34:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:34:59 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:35:00.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 263384 3.82 MB/sec execute 236 sec latency 75.726 ms 2024-09-05T12:35:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:00 smithi071 ceph-mon[29845]: pgmap v222: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-09-05T12:35:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:00 smithi123 ceph-mon[34321]: pgmap v222: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-09-05T12:35:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:00 smithi174 ceph-mon[34313]: pgmap v222: 129 pgs: 129 active+clean; 89 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-09-05T12:35:01.221 INFO:tasks.workunit.client.0.smithi071.stdout: 1 263384 3.80 MB/sec execute 237 sec latency 1075.872 ms 2024-09-05T12:35:02.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 263386 3.79 MB/sec execute 238 sec latency 1729.535 ms 2024-09-05T12:35:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:02 smithi071 ceph-mon[29845]: pgmap v223: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-09-05T12:35:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:02 smithi123 ceph-mon[34321]: pgmap v223: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-09-05T12:35:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:02 smithi174 ceph-mon[34313]: pgmap v223: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-09-05T12:35:03.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 263547 3.77 MB/sec execute 239 sec latency 392.280 ms 2024-09-05T12:35:04.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 264204 3.77 MB/sec execute 240 sec latency 509.821 ms 2024-09-05T12:35:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:04 smithi071 ceph-mon[29845]: pgmap v224: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-09-05T12:35:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:04 smithi123 ceph-mon[34321]: pgmap v224: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-09-05T12:35:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:04 smithi174 ceph-mon[34313]: pgmap v224: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 165 op/s 2024-09-05T12:35:05.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 264204 3.76 MB/sec execute 241 sec latency 1509.902 ms 2024-09-05T12:35:06.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 264204 3.74 MB/sec execute 242 sec latency 2510.022 ms 2024-09-05T12:35:06.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:06 smithi071 ceph-mon[29845]: pgmap v225: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-05T12:35:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:06 smithi123 ceph-mon[34321]: pgmap v225: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-05T12:35:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:06 smithi174 ceph-mon[34313]: pgmap v225: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-09-05T12:35:07.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 265010 3.74 MB/sec execute 243 sec latency 2664.354 ms 2024-09-05T12:35:08.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 265011 3.72 MB/sec execute 244 sec latency 1122.643 ms 2024-09-05T12:35:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:08 smithi071 ceph-mon[29845]: pgmap v226: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-05T12:35:08.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:08 smithi071 ceph-mon[29845]: osdmap e98: 12 total, 12 up, 12 in 2024-09-05T12:35:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:08 smithi123 ceph-mon[34321]: pgmap v226: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-05T12:35:08.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:08 smithi123 ceph-mon[34321]: osdmap e98: 12 total, 12 up, 12 in 2024-09-05T12:35:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:08 smithi174 ceph-mon[34313]: pgmap v226: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-05T12:35:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:08 smithi174 ceph-mon[34313]: osdmap e98: 12 total, 12 up, 12 in 2024-09-05T12:35:09.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 266459 3.75 MB/sec execute 245 sec latency 418.370 ms 2024-09-05T12:35:10.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 267752 3.76 MB/sec execute 246 sec latency 62.233 ms 2024-09-05T12:35:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:10 smithi071 ceph-mon[29845]: pgmap v228: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-05T12:35:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:10 smithi123 ceph-mon[34321]: pgmap v228: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-05T12:35:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:10 smithi174 ceph-mon[34313]: pgmap v228: 129 pgs: 129 active+clean; 90 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-09-05T12:35:11.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 267752 3.74 MB/sec execute 247 sec latency 1021.273 ms 2024-09-05T12:35:12.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 268627 3.74 MB/sec execute 248 sec latency 1174.576 ms 2024-09-05T12:35:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 -- 172.21.15.71:0/967980307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc10e0f0 msgr2=0x7faafc110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 --2- 172.21.15.71:0/967980307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc10e0f0 0x7faafc110540 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faaf4009ca0 tx=0x7faaf402a960 comp rx=0 tx=0).stop 2024-09-05T12:35:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 -- 172.21.15.71:0/967980307 shutdown_connections 2024-09-05T12:35:12.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 --2- 172.21.15.71:0/967980307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc10e0f0 0x7faafc110540 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 --2- 172.21.15.71:0/967980307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074f40 0x7faafc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 --2- 172.21.15.71:0/967980307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faafc074620 0x7faafc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 -- 172.21.15.71:0/967980307 >> 172.21.15.71:0/967980307 conn(0x7faafc0fd5d0 msgr2=0x7faafc0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 -- 172.21.15.71:0/967980307 shutdown_connections 2024-09-05T12:35:12.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.454+0000 7fab00c00640 1 -- 172.21.15.71:0/967980307 wait complete. 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 Processor -- start 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 -- start start 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 0x7faafc19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc074f40 0x7faafc19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faafc10e0f0 0x7faafc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faafc113f70 con 0x7faafc074f40 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faafc113df0 con 0x7faafc074620 2024-09-05T12:35:12.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.455+0000 7fab00c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1140f0 con 0x7faafc10e0f0 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc074f40 0x7faafc19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 0x7faafc19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 0x7faafc19a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33788/0 (socket says 172.21.15.71:33788) 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 -- 172.21.15.71:0/2133690059 learned_addr learned my addr 172.21.15.71:0/2133690059 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafbe00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faafc10e0f0 0x7faafc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 -- 172.21.15.71:0/2133690059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faafc10e0f0 msgr2=0x7faafc19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faafc10e0f0 0x7faafc19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 -- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc074f40 msgr2=0x7faafc19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc074f40 0x7faafc19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 -- 172.21.15.71:0/2133690059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc19f4b0 con 0x7faafc074620 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafaa00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc074f40 0x7faafc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.456+0000 7faafb400640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 0x7faafc19a4c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e9f0 tx=0x7faaec00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:12.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.457+0000 7faaf3e00640 1 -- 172.21.15.71:0/2133690059 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec00cd80 con 0x7faafc074620 2024-09-05T12:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.457+0000 7faaf3e00640 1 -- 172.21.15.71:0/2133690059 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec005500 con 0x7faafc074620 2024-09-05T12:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.457+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc19f7a0 con 0x7faafc074620 2024-09-05T12:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.457+0000 7faaf3e00640 1 -- 172.21.15.71:0/2133690059 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec010690 con 0x7faafc074620 2024-09-05T12:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.457+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1a7030 con 0x7faafc074620 2024-09-05T12:35:12.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.460+0000 7faaf3e00640 1 -- 172.21.15.71:0/2133690059 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faaec0046c0 con 0x7faafc074620 2024-09-05T12:35:12.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.460+0000 7fab050aa740 1 Processor -- start 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 -- start start 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faaf3e00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faae00a2cf0 0x7faae00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d422790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d42d570 0x55a82d42f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a82d388100 con 0x55a82d42d570 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a82d3301b0 con 0x55a82d422cd0 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7fab050aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a82d3840c0 con 0x55a82d4223b0 2024-09-05T12:35:12.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faaf3e00640 1 -- 172.21.15.71:0/2133690059 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7faaec0cfcc0 con 0x7faafc074620 2024-09-05T12:35:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faafbe00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faae00a2cf0 0x7faae00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faafaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faafbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d42d570 0x55a82d42f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faafaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33794/0 (socket says 172.21.15.71:33794) 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faafaa00640 1 -- 172.21.15.71:0/1610256918 learned_addr learned my addr 172.21.15.71:0/1610256918 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.461+0000 7faafb400640 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d422790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafaa00640 1 -- 172.21.15.71:0/1610256918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 msgr2=0x55a82d422790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafaa00640 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d422790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafaa00640 1 -- 172.21.15.71:0/1610256918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d42d570 msgr2=0x55a82d42f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafaa00640 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d42d570 0x55a82d42f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafaa00640 1 -- 172.21.15.71:0/1610256918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82d09b210 con 0x55a82d422cd0 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafbe00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faae00a2cf0 0x7faae00a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faaf4006010 tx=0x7faaf402f4a0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faafaa00640 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42d030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faae4009de0 tx=0x7faae400e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6060cabce3babb17 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:12.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faaebe00640 1 -- 172.21.15.71:0/1610256918 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4017070 con 0x55a82d422cd0 2024-09-05T12:35:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.462+0000 7faaebe00640 1 -- 172.21.15.71:0/1610256918 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4013950 con 0x55a82d422cd0 2024-09-05T12:35:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7faaebe00640 1 -- 172.21.15.71:0/1610256918 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae40129b0 con 0x55a82d422cd0 2024-09-05T12:35:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 -- 172.21.15.71:0/1610256918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 msgr2=0x55a82d42d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42d030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faae4009de0 tx=0x7faae400e5a0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 -- 172.21.15.71:0/1610256918 shutdown_connections 2024-09-05T12:35:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d42d570 0x55a82d42f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42d030 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 --2- 172.21.15.71:0/1610256918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d422790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 -- 172.21.15.71:0/1610256918 >> 172.21.15.71:0/1610256918 conn(0x55a82d41e920 msgr2=0x55a82d41ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 -- 172.21.15.71:0/1610256918 shutdown_connections 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.463+0000 7fab050aa740 1 -- 172.21.15.71:0/1610256918 wait complete. 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.464+0000 7fab050aa740 1 -- start start 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.464+0000 7fab050aa740 1 Processor -- start 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d42a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d42d570 0x55a82d42b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a82d42fff0 con 0x55a82d4223b0 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a82d311090 con 0x55a82d422cd0 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a82d430170 con 0x55a82d42d570 2024-09-05T12:35:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7fab050aa740 10 client.0 _ll_register_callbacks cb 0x55a82d365f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d42a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d42d570 0x55a82d42b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafaa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d42a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37974/0 (socket says 172.21.15.71:37974) 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafb400640 1 -- 172.21.15.71:0/1211491281 learned_addr learned my addr 172.21.15.71:0/1211491281 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafbe00640 1 -- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 msgr2=0x55a82d42ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafbe00640 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafbe00640 1 -- 172.21.15.71:0/1211491281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 msgr2=0x55a82d42a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafbe00640 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d42a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafbe00640 1 -- 172.21.15.71:0/1211491281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82d167620 con 0x55a82d42d570 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.465+0000 7faafb400640 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d42a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7faafbe00640 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d42d570 0x55a82d42b170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005e30 tx=0x7faaf4005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd18c8fa9805fcd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7faae8c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:35:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7faae8c00640 1 -- 172.21.15.71:0/1211491281 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4049460 con 0x55a82d42d570 2024-09-05T12:35:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae4013950 con 0x55a82d42d570 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7faae8c00640 1 -- 172.21.15.71:0/1211491281 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7faaf4049600 con 0x55a82d42d570 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7faae8c00640 1 -- 172.21.15.71:0/1211491281 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7faaf4049800 con 0x55a82d42d570 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7fab050aa740 10 client.26281 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.466+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a82d43be70 con 0x55a82d42d570 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7faae8c00640 1 -- 172.21.15.71:0/1211491281 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4013950 con 0x55a82d42d570 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7faae8c00640 1 -- 172.21.15.71:0/1211491281 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7faaf4050070 con 0x55a82d42d570 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7fab050aa740 10 client.26281 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7fab050aa740 10 client.26281 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7fab050aa740 20 client.26281 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:12.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7fab050aa740 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d43d300 0x55a82d43f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7fab050aa740 4 client.26281 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a82d395830 con 0x55a82d43d300 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.467+0000 7faafb400640 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d43d300 0x55a82d43f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.468+0000 7faafb400640 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d43d300 0x55a82d43f760 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33041984d83f2713 server_cookie=f84b0edbae50b06b in_seq=0 out_seq=0 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.468+0000 7faae8c00640 10 client.26281 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.468+0000 7faae8c00640 1 -- 172.21.15.71:0/1211491281 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a82d395830 con 0x55a82d43d300 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.468+0000 7faae8c00640 10 client.26281 handle_command_reply: tid=0 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 client.26281 shutdown 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d42d570 msgr2=0x55a82d42b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d42d570 0x55a82d42b170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005e30 tx=0x7faaf4005d80 comp rx=0 tx=0).stop 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 shutdown_connections 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d43d300 0x55a82d43f760 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d42d570 0x55a82d42b170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7faae8c00640 0 client.26281 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d422cd0 0x55a82d42ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 --2- 172.21.15.71:0/1211491281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d42a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 >> 172.21.15.71:0/1211491281 conn(0x55a82d41e920 msgr2=0x55a82d424630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 shutdown_connections 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 1 -- 172.21.15.71:0/1211491281 wait complete. 2024-09-05T12:35:12.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.469+0000 7fab050aa740 20 client.26281 trim_cache size 0 max 16384 2024-09-05T12:35:12.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 Processor -- start 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 -- start start 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d445030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d422cd0 0x55a82d445570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d427950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b10a0 con 0x55a82d4223b0 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b2ca0 con 0x55a82d42d570 2024-09-05T12:35:12.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7fab050aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b1770 con 0x55a82d422cd0 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7faafb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d445030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7faafb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d445030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37978/0 (socket says 172.21.15.71:37978) 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7faafb400640 1 -- 172.21.15.71:0/965643016 learned_addr learned my addr 172.21.15.71:0/965643016 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.484+0000 7faafbe00640 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d427950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafaa00640 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d422cd0 0x55a82d445570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafb400640 1 -- 172.21.15.71:0/965643016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d422cd0 msgr2=0x55a82d445570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafb400640 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d422cd0 0x55a82d445570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafb400640 1 -- 172.21.15.71:0/965643016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 msgr2=0x55a82d427950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafb400640 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d427950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafb400640 1 -- 172.21.15.71:0/965643016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82d418c20 con 0x55a82d4223b0 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faafb400640 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d445030 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faaec009380 tx=0x7faaec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53fada25a45d41d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faaeb400640 1 -- 172.21.15.71:0/965643016 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0a4a00 con 0x55a82d4223b0 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faaeb400640 1 -- 172.21.15.71:0/965643016 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec09a090 con 0x55a82d4223b0 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.485+0000 7faaeb400640 1 -- 172.21.15.71:0/965643016 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0a47d0 con 0x55a82d4223b0 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 -- 172.21.15.71:0/965643016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 msgr2=0x55a82d445030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d445030 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7faaec009380 tx=0x7faaec004a40 comp rx=0 tx=0).stop 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 -- 172.21.15.71:0/965643016 shutdown_connections 2024-09-05T12:35:12.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d427950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d422cd0 0x55a82d445570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 --2- 172.21.15.71:0/965643016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d4223b0 0x55a82d445030 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 -- 172.21.15.71:0/965643016 >> 172.21.15.71:0/965643016 conn(0x55a82d41e920 msgr2=0x55a82d41d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 -- 172.21.15.71:0/965643016 shutdown_connections 2024-09-05T12:35:12.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 -- 172.21.15.71:0/965643016 wait complete. 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 -- start start 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.486+0000 7fab050aa740 1 Processor -- start 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d435800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 0x55a82d4272a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d4277e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b2510 con 0x55a82d422cd0 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b1e40 con 0x55a82d42d570 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1af0a0 con 0x55a82d4223b0 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7fab050aa740 10 client.0 _ll_register_callbacks cb 0x7faaf402f630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7faafaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 0x55a82d4272a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7faafbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d4277e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7faafaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 0x55a82d4272a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:37986/0 (socket says 172.21.15.71:37986) 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.487+0000 7faafaa00640 1 -- 172.21.15.71:0/918925192 learned_addr learned my addr 172.21.15.71:0/918925192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafaa00640 1 -- 172.21.15.71:0/918925192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 msgr2=0x55a82d435800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafaa00640 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d435800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafaa00640 1 -- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 msgr2=0x55a82d4277e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafaa00640 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d4277e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafaa00640 1 -- 172.21.15.71:0/918925192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82d427d20 con 0x55a82d422cd0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafbe00640 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d4277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faafaa00640 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 0x55a82d4272a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faae4009de0 tx=0x7faae4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5344899625d438 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faae8c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faae8c00640 1 -- 172.21.15.71:0/918925192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4017040 con 0x55a82d422cd0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faae8c00640 1 -- 172.21.15.71:0/918925192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9526+0+0 (secure 0 0 0) 0x7faae4005d80 con 0x55a82d422cd0 2024-09-05T12:35:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec09a090 con 0x55a82d422cd0 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7faae8c00640 1 -- 172.21.15.71:0/918925192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7faae4003690 con 0x55a82d422cd0 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7fab050aa740 10 client.16233 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.488+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55a82d4a7d60 con 0x55a82d422cd0 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7faae8c00640 1 -- 172.21.15.71:0/918925192 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec09a090 con 0x55a82d422cd0 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7faae8c00640 1 -- 172.21.15.71:0/918925192 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7faae40065e0 con 0x55a82d422cd0 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7fab050aa740 10 client.16233 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7fab050aa740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7fab050aa740 20 client.16233 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7fab050aa740 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d4a8290 0x55a82d4aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7fab050aa740 4 client.16233 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:35:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a82d38fb80 con 0x55a82d4a8290 2024-09-05T12:35:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.489+0000 7faafb400640 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d4a8290 0x55a82d4aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.490+0000 7faae8c00640 10 client.16233 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.490+0000 7faafb400640 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d4a8290 0x55a82d4aa6b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1bdec3630fe147 server_cookie=d46e34e24cee9b5d in_seq=0 out_seq=0 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.538+0000 7faae8c00640 1 -- 172.21.15.71:0/918925192 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a82d38fb80 con 0x55a82d4a8290 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.538+0000 7faae8c00640 10 client.16233 handle_command_reply: tid=0 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.538+0000 7fab050aa740 1 client.16233 shutdown 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.538+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 msgr2=0x55a82d4272a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.538+0000 7fab050aa740 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 0x55a82d4272a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7faae4009de0 tx=0x7faae4004670 comp rx=0 tx=0).stop 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 shutdown_connections 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a82d4a8290 0x55a82d4aa6b0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a82d42d570 0x55a82d4277e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a82d422cd0 0x55a82d4272a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 --2- 172.21.15.71:0/918925192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a82d4223b0 0x55a82d435800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7faae8c00640 0 client.16233 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 >> 172.21.15.71:0/918925192 conn(0x55a82d41e920 msgr2=0x55a82d423e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 shutdown_connections 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 1 -- 172.21.15.71:0/918925192 wait complete. 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.539+0000 7fab050aa740 20 client.16233 trim_cache size 0 max 16384 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:35:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:35:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:35:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faae00a2cf0 msgr2=0x7faae00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faae00a2cf0 0x7faae00a51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faaf4006010 tx=0x7faaf402f4a0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 msgr2=0x7faafc19a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 0x7faafc19a4c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faaec00e9f0 tx=0x7faaec00edf0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 shutdown_connections 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faafc10e0f0 0x7faafc19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faae00a2cf0 0x7faae00a51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faafc074f40 0x7faafc19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 --2- 172.21.15.71:0/2133690059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faafc074620 0x7faafc19a4c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.540+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 >> 172.21.15.71:0/2133690059 conn(0x7faafc0fd5d0 msgr2=0x7faafc0fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.541+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 shutdown_connections 2024-09-05T12:35:12.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:12.541+0000 7fab00c00640 1 -- 172.21.15.71:0/2133690059 wait complete. 2024-09-05T12:35:12.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:12 smithi071 ceph-mon[29845]: pgmap v229: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-09-05T12:35:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:12 smithi123 ceph-mon[34321]: pgmap v229: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-09-05T12:35:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:12 smithi174 ceph-mon[34313]: pgmap v229: 129 pgs: 129 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-09-05T12:35:13.225 INFO:tasks.workunit.client.0.smithi071.stdout: 1 269711 3.76 MB/sec execute 249 sec latency 525.695 ms 2024-09-05T12:35:14.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 270718 3.76 MB/sec execute 250 sec latency 35.787 ms 2024-09-05T12:35:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:14 smithi071 ceph-mon[29845]: pgmap v230: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-05T12:35:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:14 smithi123 ceph-mon[34321]: pgmap v230: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-05T12:35:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:14 smithi174 ceph-mon[34313]: pgmap v230: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-05T12:35:15.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 271294 3.76 MB/sec execute 251 sec latency 585.308 ms 2024-09-05T12:35:16.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 271294 3.75 MB/sec execute 252 sec latency 1585.475 ms 2024-09-05T12:35:16.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:16 smithi071 ceph-mon[29845]: pgmap v231: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-05T12:35:16.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:16 smithi071 ceph-mon[29845]: osdmap e99: 12 total, 12 up, 12 in 2024-09-05T12:35:16.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:16 smithi123 ceph-mon[34321]: pgmap v231: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-05T12:35:16.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:16 smithi123 ceph-mon[34321]: osdmap e99: 12 total, 12 up, 12 in 2024-09-05T12:35:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:16 smithi174 ceph-mon[34313]: pgmap v231: 129 pgs: 129 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 190 op/s 2024-09-05T12:35:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:16 smithi174 ceph-mon[34313]: osdmap e99: 12 total, 12 up, 12 in 2024-09-05T12:35:17.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 272531 3.76 MB/sec execute 253 sec latency 1738.848 ms 2024-09-05T12:35:18.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 273604 3.77 MB/sec execute 254 sec latency 348.078 ms 2024-09-05T12:35:18.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:18 smithi123 ceph-mon[34321]: pgmap v233: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-09-05T12:35:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:18 smithi174 ceph-mon[34313]: pgmap v233: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-09-05T12:35:18.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:18 smithi071 ceph-mon[29845]: pgmap v233: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 2.1 MiB/s wr, 217 op/s 2024-09-05T12:35:19.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 274681 3.78 MB/sec execute 255 sec latency 129.659 ms 2024-09-05T12:35:20.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 274843 3.76 MB/sec execute 256 sec latency 867.555 ms 2024-09-05T12:35:20.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:20 smithi123 ceph-mon[34321]: pgmap v234: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-09-05T12:35:20.463 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:20 smithi071 ceph-mon[29845]: pgmap v234: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-09-05T12:35:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:20 smithi174 ceph-mon[34313]: pgmap v234: 129 pgs: 129 active+clean; 92 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.0 MiB/s wr, 203 op/s 2024-09-05T12:35:21.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 274843 3.75 MB/sec execute 257 sec latency 1867.658 ms 2024-09-05T12:35:22.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 276489 3.77 MB/sec execute 258 sec latency 2021.609 ms 2024-09-05T12:35:22.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:22 smithi071 ceph-mon[29845]: pgmap v235: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-09-05T12:35:22.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:22 smithi123 ceph-mon[34321]: pgmap v235: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-09-05T12:35:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:22 smithi174 ceph-mon[34313]: pgmap v235: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.8 MiB/s wr, 196 op/s 2024-09-05T12:35:23.226 INFO:tasks.workunit.client.0.smithi071.stdout: 1 277282 3.77 MB/sec execute 259 sec latency 308.780 ms 2024-09-05T12:35:24.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.78 MB/sec execute 260 sec latency 60.516 ms 2024-09-05T12:35:24.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:24 smithi123 ceph-mon[34321]: pgmap v236: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-09-05T12:35:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:24 smithi174 ceph-mon[34313]: pgmap v236: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-09-05T12:35:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:24 smithi071 ceph-mon[29845]: pgmap v236: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-09-05T12:35:25.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.76 MB/sec execute 261 sec latency 1041.822 ms 2024-09-05T12:35:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 -- 172.21.15.71:0/3424082268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 msgr2=0x7fc84810b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 --2- 172.21.15.71:0/3424082268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84810b8e0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc834009900 tx=0x7fc83402a480 comp rx=0 tx=0).stop 2024-09-05T12:35:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 -- 172.21.15.71:0/3424082268 shutdown_connections 2024-09-05T12:35:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 --2- 172.21.15.71:0/3424082268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 0x7fc84810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 --2- 172.21.15.71:0/3424082268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84810b8e0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 --2- 172.21.15.71:0/3424082268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 0x7fc848102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 -- 172.21.15.71:0/3424082268 >> 172.21.15.71:0/3424082268 conn(0x7fc8480777c0 msgr2=0x7fc8480feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 -- 172.21.15.71:0/3424082268 shutdown_connections 2024-09-05T12:35:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.665+0000 7fc84f000640 1 -- 172.21.15.71:0/3424082268 wait complete. 2024-09-05T12:35:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 Processor -- start 2024-09-05T12:35:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 -- start start 2024-09-05T12:35:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 0x7fc84819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 0x7fc84819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111cd0 con 0x7fc848103400 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111b50 con 0x7fc848102ae0 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.666+0000 7fc84f000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111e50 con 0x7fc84810bf00 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84819a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58638/0 (socket says 172.21.15.71:58638) 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84d200640 1 -- 172.21.15.71:0/1857539379 learned_addr learned my addr 172.21.15.71:0/1857539379 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84dc00640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 0x7fc84819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84e600640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 0x7fc84819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84dc00640 1 -- 172.21.15.71:0/1857539379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 msgr2=0x7fc84819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84dc00640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 0x7fc84819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84dc00640 1 -- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 msgr2=0x7fc84819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84dc00640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84dc00640 1 -- 172.21.15.71:0/1857539379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84819f400 con 0x7fc848102ae0 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84d200640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.667+0000 7fc84e600640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 0x7fc84819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.668+0000 7fc84dc00640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 0x7fc84819a410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00e9f0 tx=0x7fc83c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.668+0000 7fc846000640 1 -- 172.21.15.71:0/1857539379 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c00ccc0 con 0x7fc848102ae0 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.668+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84819f6f0 con 0x7fc848102ae0 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.668+0000 7fc846000640 1 -- 172.21.15.71:0/1857539379 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc83c010910 con 0x7fc848102ae0 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.668+0000 7fc846000640 1 -- 172.21.15.71:0/1857539379 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c018960 con 0x7fc848102ae0 2024-09-05T12:35:25.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.668+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a6f80 con 0x7fc848102ae0 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.671+0000 7fc8534ce740 1 Processor -- start 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.671+0000 7fc846000640 1 -- 172.21.15.71:0/1857539379 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc83c010ac0 con 0x7fc848102ae0 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.671+0000 7fc8534ce740 1 -- start start 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.671+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703e8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc8534ce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56457034e100 con 0x5645703f3570 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc8534ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645702f61b0 con 0x5645703e8cd0 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc8534ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56457034a0c0 con 0x5645703e83b0 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58654/0 (socket says 172.21.15.71:58654) 2024-09-05T12:35:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 -- 172.21.15.71:0/292133406 learned_addr learned my addr 172.21.15.71:0/292133406 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 -- 172.21.15.71:0/292133406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 msgr2=0x5645703e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 -- 172.21.15.71:0/292133406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 msgr2=0x5645703f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.672+0000 7fc84d200640 1 -- 172.21.15.71:0/292133406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564570061210 con 0x5645703f3570 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc84d200640 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f5960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc834000c00 tx=0x7fc834005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7122455d404713 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc846000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8300a2cf0 0x7fc8300a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc83aa00640 1 -- 172.21.15.71:0/292133406 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834040070 con 0x5645703f3570 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc83aa00640 1 -- 172.21.15.71:0/292133406 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834031970 con 0x5645703f3570 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc83aa00640 1 -- 172.21.15.71:0/292133406 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834035040 con 0x5645703f3570 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc846000640 1 -- 172.21.15.71:0/1857539379 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc83c0cfbb0 con 0x7fc848102ae0 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc84e600640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8300a2cf0 0x7fc8300a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc8534ce740 1 -- 172.21.15.71:0/292133406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 msgr2=0x5645703f5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.673+0000 7fc8534ce740 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f5960 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc834000c00 tx=0x7fc834005dd0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 -- 172.21.15.71:0/292133406 shutdown_connections 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f5960 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 --2- 172.21.15.71:0/292133406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 -- 172.21.15.71:0/292133406 >> 172.21.15.71:0/292133406 conn(0x5645703e4920 msgr2=0x5645703e4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 -- 172.21.15.71:0/292133406 shutdown_connections 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc84e600640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8300a2cf0 0x7fc8300a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc840009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 -- 172.21.15.71:0/292133406 wait complete. 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 -- start start 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.674+0000 7fc8534ce740 1 Processor -- start 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703f0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5645703f60e0 con 0x5645703f3570 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5645702d7090 con 0x5645703e8cd0 2024-09-05T12:35:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645703f6260 con 0x5645703e83b0 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc8534ce740 10 client.0 _ll_register_callbacks cb 0x56457032bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc84d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc84d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33772/0 (socket says 172.21.15.71:33772) 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc84d200640 1 -- 172.21.15.71:0/209908986 learned_addr learned my addr 172.21.15.71:0/209908986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc84e600640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.675+0000 7fc84dc00640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703f0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84e600640 1 -- 172.21.15.71:0/209908986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 msgr2=0x5645703f0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84e600640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703f0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84e600640 1 -- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 msgr2=0x5645703f0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84e600640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84e600640 1 -- 172.21.15.71:0/209908986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56457012d620 con 0x5645703f3570 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84dc00640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703f0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc84e600640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f0fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc840003f50 tx=0x7fc840004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb89b92a0f93c7e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc82f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc82f400640 1 -- 172.21.15.71:0/209908986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84000fe60 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc82f400640 1 -- 172.21.15.71:0/209908986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc840004740 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc834031970 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc82f400640 1 -- 172.21.15.71:0/209908986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc840012020 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc8534ce740 10 client.16245 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.676+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x564570401e40 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc82f400640 1 -- 172.21.15.71:0/209908986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834031970 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc82f400640 1 -- 172.21.15.71:0/209908986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fc840027030 con 0x5645703f3570 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc8534ce740 10 client.16245 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc8534ce740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc8534ce740 20 client.16245 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc8534ce740 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564570403350 0x5645704057a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc8534ce740 4 client.16245 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56457035b830 con 0x564570403350 2024-09-05T12:35:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.677+0000 7fc84dc00640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564570403350 0x5645704057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.678+0000 7fc84dc00640 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564570403350 0x5645704057a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa04ab8b892a171 server_cookie=1ef65e55057d66a1 in_seq=0 out_seq=0 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.678+0000 7fc82f400640 10 client.16245 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc82f400640 1 -- 172.21.15.71:0/209908986 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56457035b830 con 0x564570403350 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc82f400640 10 client.16245 handle_command_reply: tid=0 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 client.16245 shutdown 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 msgr2=0x5645703f0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f0fc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc840003f50 tx=0x7fc840004ab0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 shutdown_connections 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564570403350 0x5645704057a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703f3570 0x5645703f0fc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e8cd0 0x5645703f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 --2- 172.21.15.71:0/209908986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703f0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc82f400640 0 client.16245 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 >> 172.21.15.71:0/209908986 conn(0x5645703e4920 msgr2=0x5645703ea630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 shutdown_connections 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 1 -- 172.21.15.71:0/209908986 wait complete. 2024-09-05T12:35:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.679+0000 7fc8534ce740 20 client.16245 trim_cache size 0 max 16384 2024-09-05T12:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.694+0000 7fc8534ce740 1 Processor -- start 2024-09-05T12:35:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.694+0000 7fc8534ce740 1 -- start start 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e83b0 0x5645703eae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703f3570 0x56457040b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc8534ce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b0fe0 con 0x5645703e8cd0 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc8534ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b2be0 con 0x5645703e83b0 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc8534ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b16b0 con 0x5645703f3570 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58676/0 (socket says 172.21.15.71:58676) 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc84d200640 1 -- 172.21.15.71:0/472738701 learned_addr learned my addr 172.21.15.71:0/472738701 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc84dc00640 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e83b0 0x5645703eae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.695+0000 7fc84e600640 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703f3570 0x56457040b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc84d200640 1 -- 172.21.15.71:0/472738701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703f3570 msgr2=0x56457040b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc84d200640 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703f3570 0x56457040b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc84d200640 1 -- 172.21.15.71:0/472738701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e83b0 msgr2=0x5645703eae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc84d200640 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e83b0 0x5645703eae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc84d200640 1 -- 172.21.15.71:0/472738701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645703f6080 con 0x5645703e8cd0 2024-09-05T12:35:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc84d200640 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703f23a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc834000c00 tx=0x7fc834005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53adff1f39da3a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc83a000640 1 -- 172.21.15.71:0/472738701 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834035040 con 0x5645703e8cd0 2024-09-05T12:35:25.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc83a000640 1 -- 172.21.15.71:0/472738701 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834031970 con 0x5645703e8cd0 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.696+0000 7fc83a000640 1 -- 172.21.15.71:0/472738701 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834040040 con 0x5645703e8cd0 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 -- 172.21.15.71:0/472738701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 msgr2=0x5645703f23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703f23a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc834000c00 tx=0x7fc834005520 comp rx=0 tx=0).stop 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 -- 172.21.15.71:0/472738701 shutdown_connections 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703f3570 0x56457040b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703f23a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 --2- 172.21.15.71:0/472738701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703e83b0 0x5645703eae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 -- 172.21.15.71:0/472738701 >> 172.21.15.71:0/472738701 conn(0x5645703e4920 msgr2=0x5645703f1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 -- 172.21.15.71:0/472738701 shutdown_connections 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 -- 172.21.15.71:0/472738701 wait complete. 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 -- start start 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.697+0000 7fc8534ce740 1 Processor -- start 2024-09-05T12:35:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703fb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703f3570 0x5645703ecb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b2450 con 0x5645703e8cd0 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b1d80 con 0x5645703f3570 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481aefe0 con 0x5645703e83b0 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc8534ce740 10 client.0 _ll_register_callbacks cb 0x7fc8400125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc84d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703fb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58684/0 (socket says 172.21.15.71:58684) 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc84d200640 1 -- 172.21.15.71:0/3613610638 learned_addr learned my addr 172.21.15.71:0/3613610638 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc84e600640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703f3570 0x5645703ecb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc84dc00640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703fb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.698+0000 7fc84d200640 1 -- 172.21.15.71:0/3613610638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 msgr2=0x5645703fb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc84d200640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703fb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc84d200640 1 -- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703f3570 msgr2=0x5645703ecb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc84d200640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703f3570 0x5645703ecb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc84d200640 1 -- 172.21.15.71:0/3613610638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645703ed140 con 0x5645703e8cd0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc84e600640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703f3570 0x5645703ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc84d200640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703fb9f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc834009a20 tx=0x7fc834009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d642a44a0b627 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc82f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc82f400640 1 -- 172.21.15.71:0/3613610638 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc834040070 con 0x5645703e8cd0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc82f400640 1 -- 172.21.15.71:0/3613610638 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc834003ae0 con 0x5645703e8cd0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc834031970 con 0x5645703e8cd0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc82f400640 1 -- 172.21.15.71:0/3613610638 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc83405f050 con 0x5645703e8cd0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc8534ce740 10 client.16251 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.699+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56457046d930 con 0x5645703e8cd0 2024-09-05T12:35:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc82f400640 1 -- 172.21.15.71:0/3613610638 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc834031970 con 0x5645703e8cd0 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc82f400640 1 -- 172.21.15.71:0/3613610638 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fc8340568e0 con 0x5645703e8cd0 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc8534ce740 10 client.16251 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc8534ce740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc8534ce740 20 client.16251 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc8534ce740 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56457046dc20 0x564570470010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc8534ce740 4 client.16251 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564570355b80 con 0x56457046dc20 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.700+0000 7fc84dc00640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56457046dc20 0x564570470010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.701+0000 7fc84dc00640 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56457046dc20 0x564570470010 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8ad80954af6c867 server_cookie=8d804a790d33ceaa in_seq=0 out_seq=0 2024-09-05T12:35:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:25.701+0000 7fc82f400640 10 client.16251 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:26.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.75 MB/sec execute 262 sec latency 2041.946 ms 2024-09-05T12:35:26.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:26 smithi123 ceph-mon[34321]: pgmap v237: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-09-05T12:35:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:26 smithi174 ceph-mon[34313]: pgmap v237: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-09-05T12:35:26.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:26 smithi071 ceph-mon[29845]: pgmap v237: 129 pgs: 129 active+clean; 95 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-09-05T12:35:27.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.73 MB/sec execute 263 sec latency 3042.085 ms 2024-09-05T12:35:28.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.72 MB/sec execute 264 sec latency 4042.220 ms 2024-09-05T12:35:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:28 smithi174 ceph-mon[34313]: pgmap v238: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-05T12:35:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:28 smithi071 ceph-mon[29845]: pgmap v238: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-05T12:35:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:28 smithi123 ceph-mon[34321]: pgmap v238: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-05T12:35:29.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.71 MB/sec execute 265 sec latency 5042.362 ms 2024-09-05T12:35:30.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.69 MB/sec execute 266 sec latency 6042.495 ms 2024-09-05T12:35:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:30 smithi174 ceph-mon[34313]: pgmap v239: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 992 KiB/s wr, 134 op/s 2024-09-05T12:35:30.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:30 smithi071 ceph-mon[29845]: pgmap v239: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 992 KiB/s wr, 134 op/s 2024-09-05T12:35:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:30 smithi123 ceph-mon[34321]: pgmap v239: 129 pgs: 129 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 992 KiB/s wr, 134 op/s 2024-09-05T12:35:31.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 278539 3.68 MB/sec execute 267 sec latency 7042.595 ms 2024-09-05T12:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.980+0000 7fc82f400640 1 -- 172.21.15.71:0/3613610638 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564570355b80 con 0x56457046dc20 2024-09-05T12:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.980+0000 7fc82f400640 10 client.16251 handle_command_reply: tid=0 2024-09-05T12:35:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.980+0000 7fc8534ce740 1 client.16251 shutdown 2024-09-05T12:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 msgr2=0x5645703fb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703fb9f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc834009a20 tx=0x7fc834009520 comp rx=0 tx=0).stop 2024-09-05T12:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 shutdown_connections 2024-09-05T12:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56457046dc20 0x564570470010 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5645703f3570 0x5645703ecb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5645703e8cd0 0x5645703fb9f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 --2- 172.21.15.71:0/3613610638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645703e83b0 0x5645703fb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 >> 172.21.15.71:0/3613610638 conn(0x5645703e4920 msgr2=0x5645703f0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.981+0000 7fc82f400640 0 client.16251 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.982+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 shutdown_connections 2024-09-05T12:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.982+0000 7fc8534ce740 1 -- 172.21.15.71:0/3613610638 wait complete. 2024-09-05T12:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.982+0000 7fc8534ce740 20 client.16251 trim_cache size 0 max 16384 2024-09-05T12:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:35:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8300a2cf0 msgr2=0x7fc8300a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8300a2cf0 0x7fc8300a51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc840009500 comp rx=0 tx=0).stop 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 msgr2=0x7fc84819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 0x7fc84819a410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc83c00e9f0 tx=0x7fc83c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 shutdown_connections 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc84810bf00 0x7fc84819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc8300a2cf0 0x7fc8300a51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc848103400 0x7fc84819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 --2- 172.21.15.71:0/1857539379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc848102ae0 0x7fc84819a410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 >> 172.21.15.71:0/1857539379 conn(0x7fc8480777c0 msgr2=0x7fc8480fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 shutdown_connections 2024-09-05T12:35:31.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:31.984+0000 7fc84f000640 1 -- 172.21.15.71:0/1857539379 wait complete. 2024-09-05T12:35:32.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 279123 3.67 MB/sec execute 268 sec latency 7795.589 ms 2024-09-05T12:35:32.437 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:32 smithi071 ceph-mon[29845]: pgmap v240: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-09-05T12:35:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:32 smithi174 ceph-mon[34313]: pgmap v240: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-09-05T12:35:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:32 smithi123 ceph-mon[34321]: pgmap v240: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-09-05T12:35:33.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 280462 3.70 MB/sec execute 269 sec latency 247.062 ms 2024-09-05T12:35:34.227 INFO:tasks.workunit.client.0.smithi071.stdout: 1 281566 3.70 MB/sec execute 270 sec latency 25.638 ms 2024-09-05T12:35:34.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:34 smithi071 ceph-mon[29845]: pgmap v241: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 717 KiB/s wr, 100 op/s 2024-09-05T12:35:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:34 smithi174 ceph-mon[34313]: pgmap v241: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 717 KiB/s wr, 100 op/s 2024-09-05T12:35:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:34 smithi123 ceph-mon[34321]: pgmap v241: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 717 KiB/s wr, 100 op/s 2024-09-05T12:35:35.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 281934 3.69 MB/sec execute 271 sec latency 560.157 ms 2024-09-05T12:35:36.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 281934 3.68 MB/sec execute 272 sec latency 1560.291 ms 2024-09-05T12:35:36.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:36 smithi174 ceph-mon[34313]: pgmap v242: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 685 KiB/s wr, 91 op/s 2024-09-05T12:35:36.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:36 smithi071 ceph-mon[29845]: pgmap v242: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 685 KiB/s wr, 91 op/s 2024-09-05T12:35:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:36 smithi123 ceph-mon[34321]: pgmap v242: 129 pgs: 129 active+clean; 101 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 685 KiB/s wr, 91 op/s 2024-09-05T12:35:37.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 283019 3.69 MB/sec execute 273 sec latency 1712.950 ms 2024-09-05T12:35:38.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 284147 3.70 MB/sec execute 274 sec latency 237.295 ms 2024-09-05T12:35:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:38 smithi174 ceph-mon[34313]: pgmap v243: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-05T12:35:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:38 smithi071 ceph-mon[29845]: pgmap v243: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-05T12:35:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:38 smithi123 ceph-mon[34321]: pgmap v243: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-05T12:35:39.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 285369 3.71 MB/sec execute 275 sec latency 15.350 ms 2024-09-05T12:35:40.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 285476 3.70 MB/sec execute 276 sec latency 933.973 ms 2024-09-05T12:35:40.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:40 smithi174 ceph-mon[34313]: pgmap v244: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 861 KiB/s wr, 94 op/s 2024-09-05T12:35:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:40 smithi071 ceph-mon[29845]: pgmap v244: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 861 KiB/s wr, 94 op/s 2024-09-05T12:35:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:40 smithi123 ceph-mon[34321]: pgmap v244: 129 pgs: 129 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 861 KiB/s wr, 94 op/s 2024-09-05T12:35:41.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 285476 3.68 MB/sec execute 277 sec latency 1934.099 ms 2024-09-05T12:35:42.228 INFO:tasks.workunit.client.0.smithi071.stdout: 1 287297 3.71 MB/sec execute 278 sec latency 2086.340 ms 2024-09-05T12:35:42.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:42 smithi174 ceph-mon[34313]: pgmap v245: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-05T12:35:42.639 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:42 smithi071 ceph-mon[29845]: pgmap v245: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-05T12:35:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:42 smithi123 ceph-mon[34321]: pgmap v245: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-05T12:35:43.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 287858 3.71 MB/sec execute 279 sec latency 310.647 ms 2024-09-05T12:35:44.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 289024 3.71 MB/sec execute 280 sec latency 311.721 ms 2024-09-05T12:35:44.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:44 smithi174 ceph-mon[34313]: pgmap v246: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 971 KiB/s wr, 132 op/s 2024-09-05T12:35:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:44 smithi071 ceph-mon[29845]: pgmap v246: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 971 KiB/s wr, 132 op/s 2024-09-05T12:35:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:44 smithi123 ceph-mon[34321]: pgmap v246: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 971 KiB/s wr, 132 op/s 2024-09-05T12:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 -- 172.21.15.71:0/2501808952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 msgr2=0x7fdaac104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 --2- 172.21.15.71:0/2501808952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac104330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009900 tx=0x7fda9c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 -- 172.21.15.71:0/2501808952 shutdown_connections 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 --2- 172.21.15.71:0/2501808952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac1090a0 0x7fdaac110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 --2- 172.21.15.71:0/2501808952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaac104900 0x7fdaac108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 --2- 172.21.15.71:0/2501808952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac104330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 -- 172.21.15.71:0/2501808952 >> 172.21.15.71:0/2501808952 conn(0x7fdaac0ff9e0 msgr2=0x7fdaac101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.100+0000 7fdab1800640 1 -- 172.21.15.71:0/2501808952 shutdown_connections 2024-09-05T12:35:45.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.101+0000 7fdab1800640 1 -- 172.21.15.71:0/2501808952 wait complete. 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.101+0000 7fdab1800640 1 Processor -- start 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.101+0000 7fdab1800640 1 -- start start 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdab1800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdab1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac1090a0 0x7fdaac19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdab1800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaac19ee80 0x7fdaac19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdab1800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1140f0 con 0x7fdaac103f50 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdab1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac113f70 con 0x7fdaac1090a0 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdab1800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac114270 con 0x7fdaac19ee80 2024-09-05T12:35:45.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45786/0 (socket says 172.21.15.71:45786) 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 -- 172.21.15.71:0/1207748881 learned_addr learned my addr 172.21.15.71:0/1207748881 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaa000640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac1090a0 0x7fdaac19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaab400640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaac19ee80 0x7fdaac19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 -- 172.21.15.71:0/1207748881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaac19ee80 msgr2=0x7fdaac19f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaac19ee80 0x7fdaac19f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 -- 172.21.15.71:0/1207748881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac1090a0 msgr2=0x7fdaac19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac1090a0 0x7fdaac19ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.102+0000 7fdaaaa00640 1 -- 172.21.15.71:0/1207748881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac19f970 con 0x7fdaac103f50 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.103+0000 7fdaaaa00640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac19a5c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.103+0000 7fdaa3400640 1 -- 172.21.15.71:0/1207748881 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c03f030 con 0x7fdaac103f50 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.103+0000 7fdaa3400640 1 -- 172.21.15.71:0/1207748881 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9c003ce0 con 0x7fdaac103f50 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.103+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac19fc00 con 0x7fdaac103f50 2024-09-05T12:35:45.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.103+0000 7fdaa3400640 1 -- 172.21.15.71:0/1207748881 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c035070 con 0x7fdaac103f50 2024-09-05T12:35:45.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.103+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a7420 con 0x7fdaac103f50 2024-09-05T12:35:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 Processor -- start 2024-09-05T12:35:45.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdaa3400640 1 -- 172.21.15.71:0/1207748881 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fda9c042030 con 0x7fdaac103f50 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 -- start start 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c4b00 0x562b2b9cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562b2b92a100 con 0x562b2b9c4b00 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b2b8d21b0 con 0x562b2b9c41e0 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.106+0000 7fdab4684740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b2b9260c0 con 0x562b2b9cf3a0 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c4b00 0x562b2b9cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaaaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c4b00 0x562b2b9cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45802/0 (socket says 172.21.15.71:45802) 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaaaa00640 1 -- 172.21.15.71:0/2910589904 learned_addr learned my addr 172.21.15.71:0/2910589904 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42452/0 (socket says 172.21.15.71:42452) 2024-09-05T12:35:45.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaaa000640 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 -- 172.21.15.71:0/2910589904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 msgr2=0x562b2b9d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 -- 172.21.15.71:0/2910589904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c4b00 msgr2=0x562b2b9cee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c4b00 0x562b2b9cee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 -- 172.21.15.71:0/2910589904 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b2b63d210 con 0x562b2b9c41e0 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaaa000640 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaa3400640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fda900a2e20 0x7fda900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.107+0000 7fdaab400640 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c45c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009e40 tx=0x7fdaa400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1833337732236185 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.108+0000 7fdaa3400640 1 -- 172.21.15.71:0/1207748881 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fda9c0efd80 con 0x7fdaac103f50 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.108+0000 7fdaaaa00640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fda900a2e20 0x7fda900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.108+0000 7fda97e00640 1 -- 172.21.15.71:0/2910589904 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4015070 con 0x562b2b9c41e0 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.108+0000 7fda97e00640 1 -- 172.21.15.71:0/2910589904 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa401ab40 con 0x562b2b9c41e0 2024-09-05T12:35:45.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.108+0000 7fda97e00640 1 -- 172.21.15.71:0/2910589904 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4011b40 con 0x562b2b9c41e0 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 -- 172.21.15.71:0/2910589904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 msgr2=0x562b2b9c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdaaaa00640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fda900a2e20 0x7fda900a52e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0c46b0 tx=0x7fda9c0f3830 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c45c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009e40 tx=0x7fdaa400f730 comp rx=0 tx=0).stop 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 -- 172.21.15.71:0/2910589904 shutdown_connections 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c4b00 0x562b2b9cee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 --2- 172.21.15.71:0/2910589904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c45c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 -- 172.21.15.71:0/2910589904 >> 172.21.15.71:0/2910589904 conn(0x562b2b9c0750 msgr2=0x562b2b9c0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 -- 172.21.15.71:0/2910589904 shutdown_connections 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 -- 172.21.15.71:0/2910589904 wait complete. 2024-09-05T12:35:45.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 -- start start 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.109+0000 7fdab4684740 1 Processor -- start 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 0x562b2b9cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9cce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562b2b9d1fd0 con 0x562b2b9c41e0 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b2b8b3090 con 0x562b2b9c4b00 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b2b9d2150 con 0x562b2b9cf3a0 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdab4684740 10 client.0 _ll_register_callbacks cb 0x562b2b907f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 0x562b2b9cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 0x562b2b9cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45816/0 (socket says 172.21.15.71:45816) 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 -- 172.21.15.71:0/1933365731 learned_addr learned my addr 172.21.15.71:0/1933365731 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaaa000640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 -- 172.21.15.71:0/1933365731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 msgr2=0x562b2b9cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 -- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 msgr2=0x562b2b9cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaab400640 1 -- 172.21.15.71:0/1933365731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2b709620 con 0x562b2b9c41e0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.110+0000 7fdaaa000640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fdaab400640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 0x562b2b9cc3a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011620 tx=0x7fdaa4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf0e4bed8b34e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fda94c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fda94c00640 1 -- 172.21.15.71:0/1933365731 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4015070 con 0x562b2b9c41e0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa401ab40 con 0x562b2b9c41e0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fda94c00640 1 -- 172.21.15.71:0/1933365731 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fdaa401a460 con 0x562b2b9c41e0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fda94c00640 1 -- 172.21.15.71:0/1933365731 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdaa401a660 con 0x562b2b9c41e0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fdab4684740 10 client.16263 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562b2b9ddbf0 con 0x562b2b9c41e0 2024-09-05T12:35:45.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.111+0000 7fda94c00640 1 -- 172.21.15.71:0/1933365731 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaa401ab40 con 0x562b2b9c41e0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fda94c00640 1 -- 172.21.15.71:0/1933365731 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fdaa40052a0 con 0x562b2b9c41e0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdab4684740 10 client.16263 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdab4684740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdab4684740 20 client.16263 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdab4684740 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2b9defa0 0x562b2b9e1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdab4684740 4 client.16263 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b2b937830 con 0x562b2b9defa0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.112+0000 7fdaaa000640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2b9defa0 0x562b2b9e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.113+0000 7fdaaa000640 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2b9defa0 0x562b2b9e1390 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4259c34a9ce2ddb1 server_cookie=bdb58ddfbe177289 in_seq=0 out_seq=0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.113+0000 7fda94c00640 10 client.16263 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.113+0000 7fda94c00640 1 -- 172.21.15.71:0/1933365731 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b2b937830 con 0x562b2b9defa0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.113+0000 7fda94c00640 10 client.16263 handle_command_reply: tid=0 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 client.16263 shutdown 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 msgr2=0x562b2b9cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 0x562b2b9cc3a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4011620 tx=0x7fdaa4009f90 comp rx=0 tx=0).stop 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 shutdown_connections 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2b9defa0 0x562b2b9e1390 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9cf3a0 0x562b2b9cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fda94c00640 0 client.16263 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 --2- 172.21.15.71:0/1933365731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9c41e0 0x562b2b9cc3a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 >> 172.21.15.71:0/1933365731 conn(0x562b2b9c0750 msgr2=0x562b2b9c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 shutdown_connections 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 1 -- 172.21.15.71:0/1933365731 wait complete. 2024-09-05T12:35:45.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.114+0000 7fdab4684740 20 client.16263 trim_cache size 0 max 16384 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 Processor -- start 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 -- start start 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c41e0 0x562b2b9ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9e6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b1160 con 0x562b2b9cf3a0 2024-09-05T12:35:45.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b2d60 con 0x562b2b9c4b00 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdab4684740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b1830 con 0x562b2b9c41e0 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9e6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45832/0 (socket says 172.21.15.71:45832) 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.129+0000 7fdaab400640 1 -- 172.21.15.71:0/3397062070 learned_addr learned my addr 172.21.15.71:0/3397062070 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaab400640 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c41e0 0x562b2b9ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9e6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 -- 172.21.15.71:0/3397062070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c41e0 msgr2=0x562b2b9ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c41e0 0x562b2b9ceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 -- 172.21.15.71:0/3397062070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 msgr2=0x562b2b9e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 -- 172.21.15.71:0/3397062070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b2b9cb2c0 con 0x562b2b9c4b00 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaab400640 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:45.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fdaaa000640 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9e6ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x562b2b9cd9e0 tx=0x7fda98005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6c1e9142e55f825 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fda97400640 1 -- 172.21.15.71:0/3397062070 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98004030 con 0x562b2b9c4b00 2024-09-05T12:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fda97400640 1 -- 172.21.15.71:0/3397062070 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98005500 con 0x562b2b9c4b00 2024-09-05T12:35:45.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.130+0000 7fda97400640 1 -- 172.21.15.71:0/3397062070 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9800fd30 con 0x562b2b9c4b00 2024-09-05T12:35:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 -- 172.21.15.71:0/3397062070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 msgr2=0x562b2b9e6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9e6ab0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x562b2b9cd9e0 tx=0x7fda98005f90 comp rx=0 tx=0).stop 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 -- 172.21.15.71:0/3397062070 shutdown_connections 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c4b00 0x562b2b9e6ab0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 --2- 172.21.15.71:0/3397062070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c41e0 0x562b2b9ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 -- 172.21.15.71:0/3397062070 >> 172.21.15.71:0/3397062070 conn(0x562b2b9c0750 msgr2=0x562b2b9c6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 -- 172.21.15.71:0/3397062070 shutdown_connections 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 -- 172.21.15.71:0/3397062070 wait complete. 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.131+0000 7fdab4684740 1 -- start start 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 Processor -- start 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c4b00 0x562b2b9c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b25d0 con 0x562b2b9cf3a0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1b1f00 con 0x562b2b9c41e0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac1af160 con 0x562b2b9c4b00 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdab4684740 10 client.0 _ll_register_callbacks cb 0x7fda9c0f3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdaab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45840/0 (socket says 172.21.15.71:45840) 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdaab400640 1 -- 172.21.15.71:0/2473384346 learned_addr learned my addr 172.21.15.71:0/2473384346 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdaaa000640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c4b00 0x562b2b9c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.132+0000 7fdaab400640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdaab400640 1 -- 172.21.15.71:0/2473384346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c4b00 msgr2=0x562b2b9c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdaab400640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c4b00 0x562b2b9c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdaab400640 1 -- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 msgr2=0x562b2b9c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdaab400640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdaab400640 1 -- 172.21.15.71:0/2473384346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b2ba514e0 con 0x562b2b9cf3a0 2024-09-05T12:35:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdaab400640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9ca010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401c010 tx=0x7fdaa4009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded13037f4280d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fda94c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fda94c00640 1 -- 172.21.15.71:0/2473384346 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4015070 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fda94c00640 1 -- 172.21.15.71:0/2473384346 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fdaa4003970 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda98005500 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fda94c00640 1 -- 172.21.15.71:0/2473384346 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fdaa4003b70 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdab4684740 10 client.16269 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.133+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x562b2ba527b0 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fda94c00640 1 -- 172.21.15.71:0/2473384346 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda98005500 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fda94c00640 1 -- 172.21.15.71:0/2473384346 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fdaa4004cd0 con 0x562b2b9cf3a0 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdab4684740 10 client.16269 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdab4684740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdab4684740 20 client.16269 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdab4684740 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2ba52dc0 0x562b2ba551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdab4684740 4 client.16269 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b2b931b80 con 0x562b2ba52dc0 2024-09-05T12:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.134+0000 7fdaaa000640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2ba52dc0 0x562b2ba551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.135+0000 7fdaaa000640 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2ba52dc0 0x562b2ba551e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e216d3450ca1fd server_cookie=c78eda684911b73f in_seq=0 out_seq=0 2024-09-05T12:35:45.139 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.135+0000 7fda94c00640 10 client.16269 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fda94c00640 1 -- 172.21.15.71:0/2473384346 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b2b931b80 con 0x562b2ba52dc0 2024-09-05T12:35:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fda94c00640 10 client.16269 handle_command_reply: tid=0 2024-09-05T12:35:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 client.16269 shutdown 2024-09-05T12:35:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 msgr2=0x562b2b9ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9ca010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401c010 tx=0x7fdaa4009980 comp rx=0 tx=0).stop 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 shutdown_connections 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562b2ba52dc0 0x562b2ba551e0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b2b9cf3a0 0x562b2b9ca010 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b2b9c4b00 0x562b2b9c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 --2- 172.21.15.71:0/2473384346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b2b9c41e0 0x562b2b9c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fda94c00640 0 client.16269 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 >> 172.21.15.71:0/2473384346 conn(0x562b2b9c0750 msgr2=0x562b2b9c5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.163+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 shutdown_connections 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab4684740 1 -- 172.21.15.71:0/2473384346 wait complete. 2024-09-05T12:35:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab4684740 20 client.16269 trim_cache size 0 max 16384 2024-09-05T12:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:35:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fda900a2e20 msgr2=0x7fda900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fda900a2e20 0x7fda900a52e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0c46b0 tx=0x7fda9c0f3830 comp rx=0 tx=0).stop 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 msgr2=0x7fdaac19a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac19a5c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fda9c006010 tx=0x7fda9c0042f0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 shutdown_connections 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdaac19ee80 0x7fdaac19f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac1090a0 0x7fdaac19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fda900a2e20 0x7fda900a52e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 --2- 172.21.15.71:0/1207748881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdaac103f50 0x7fdaac19a5c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.164+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 >> 172.21.15.71:0/1207748881 conn(0x7fdaac0ff9e0 msgr2=0x7fdaac100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.165+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 shutdown_connections 2024-09-05T12:35:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:45.165+0000 7fdab1800640 1 -- 172.21.15.71:0/1207748881 wait complete. 2024-09-05T12:35:45.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 289024 3.70 MB/sec execute 281 sec latency 1200.353 ms 2024-09-05T12:35:46.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 289024 3.69 MB/sec execute 282 sec latency 2200.456 ms 2024-09-05T12:35:46.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:46 smithi174 ceph-mon[34313]: pgmap v247: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 963 KiB/s wr, 129 op/s 2024-09-05T12:35:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:46 smithi071 ceph-mon[29845]: pgmap v247: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 963 KiB/s wr, 129 op/s 2024-09-05T12:35:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:46 smithi123 ceph-mon[34321]: pgmap v247: 129 pgs: 129 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 963 KiB/s wr, 129 op/s 2024-09-05T12:35:47.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 290353 3.70 MB/sec execute 283 sec latency 2349.619 ms 2024-09-05T12:35:48.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 291337 3.71 MB/sec execute 284 sec latency 58.782 ms 2024-09-05T12:35:48.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:48 smithi071 ceph-mon[29845]: pgmap v248: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-09-05T12:35:48.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:48 smithi174 ceph-mon[34313]: pgmap v248: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-09-05T12:35:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:48 smithi123 ceph-mon[34321]: pgmap v248: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 201 op/s 2024-09-05T12:35:49.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 292462 3.71 MB/sec execute 285 sec latency 211.775 ms 2024-09-05T12:35:50.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 292567 3.70 MB/sec execute 286 sec latency 931.331 ms 2024-09-05T12:35:50.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:50 smithi123 ceph-mon[34321]: pgmap v249: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-09-05T12:35:50.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:50 smithi174 ceph-mon[34313]: pgmap v249: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-09-05T12:35:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:50 smithi071 ceph-mon[29845]: pgmap v249: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 136 op/s 2024-09-05T12:35:51.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 292567 3.69 MB/sec execute 287 sec latency 1931.409 ms 2024-09-05T12:35:52.231 INFO:tasks.workunit.client.0.smithi071.stdout: 1 293887 3.70 MB/sec execute 288 sec latency 2081.738 ms 2024-09-05T12:35:52.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:52 smithi174 ceph-mon[34313]: pgmap v250: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-05T12:35:52.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:52 smithi071 ceph-mon[29845]: pgmap v250: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-05T12:35:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:52 smithi123 ceph-mon[34321]: pgmap v250: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-05T12:35:53.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 294948 3.71 MB/sec execute 289 sec latency 129.219 ms 2024-09-05T12:35:54.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 295880 3.72 MB/sec execute 290 sec latency 329.166 ms 2024-09-05T12:35:54.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:54 smithi174 ceph-mon[34313]: pgmap v251: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-09-05T12:35:54.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:54 smithi071 ceph-mon[29845]: pgmap v251: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-09-05T12:35:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:54 smithi123 ceph-mon[34321]: pgmap v251: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-09-05T12:35:55.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 296117 3.70 MB/sec execute 291 sec latency 732.623 ms 2024-09-05T12:35:56.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 296117 3.69 MB/sec execute 292 sec latency 1732.709 ms 2024-09-05T12:35:56.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:56 smithi174 ceph-mon[34313]: pgmap v252: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-09-05T12:35:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:56 smithi071 ceph-mon[29845]: pgmap v252: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-09-05T12:35:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:56 smithi123 ceph-mon[34321]: pgmap v252: 129 pgs: 129 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-09-05T12:35:57.224 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:35:57.224 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.15 2024-09-05T12:35:57.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:35:57.239 INFO:tasks.workunit.client.0.smithi071.stdout: 1 297641 3.71 MB/sec execute 293 sec latency 1877.619 ms 2024-09-05T12:35:57.651 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:35:58.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 298526 3.71 MB/sec execute 294 sec latency 203.021 ms 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 -- 172.21.15.71:0/4043492349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c103b00 msgr2=0x7fa65c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 --2- 172.21.15.71:0/4043492349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c103b00 0x7fa65c103f60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa648009930 tx=0x7fa64802a420 comp rx=0 tx=0).stop 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 -- 172.21.15.71:0/4043492349 shutdown_connections 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 --2- 172.21.15.71:0/4043492349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c1044a0 0x7fa65c1109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 --2- 172.21.15.71:0/4043492349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c103b00 0x7fa65c103f60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 --2- 172.21.15.71:0/4043492349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa65c109ba0 0x7fa65c109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.294+0000 7fa662e00640 1 -- 172.21.15.71:0/4043492349 >> 172.21.15.71:0/4043492349 conn(0x7fa65c0ff7c0 msgr2=0x7fa65c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.295+0000 7fa662e00640 1 -- 172.21.15.71:0/4043492349 shutdown_connections 2024-09-05T12:35:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.295+0000 7fa662e00640 1 -- 172.21.15.71:0/4043492349 wait complete. 2024-09-05T12:35:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 Processor -- start 2024-09-05T12:35:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 -- start start 2024-09-05T12:35:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c103b00 0x7fa65c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa65c1044a0 0x7fa65c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c109ba0 0x7fa65c19edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c113f60 con 0x7fa65c1044a0 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c113de0 con 0x7fa65c103b00 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1140e0 con 0x7fa65c109ba0 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa661a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c103b00 0x7fa65c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa662400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c109ba0 0x7fa65c19edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa661a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c103b00 0x7fa65c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60190/0 (socket says 172.21.15.71:60190) 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa661a00640 1 -- 172.21.15.71:0/245788455 learned_addr learned my addr 172.21.15.71:0/245788455 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.296+0000 7fa661000640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa65c1044a0 0x7fa65c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662400640 1 -- 172.21.15.71:0/245788455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c103b00 msgr2=0x7fa65c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662400640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c103b00 0x7fa65c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662400640 1 -- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa65c1044a0 msgr2=0x7fa65c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662400640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa65c1044a0 0x7fa65c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662400640 1 -- 172.21.15.71:0/245788455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65c19f4e0 con 0x7fa65c109ba0 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662400640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c109ba0 0x7fa65c19edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa65400ec50 tx=0x7fa65400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa65a000640 1 -- 172.21.15.71:0/245788455 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654003b70 con 0x7fa65c109ba0 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa65a000640 1 -- 172.21.15.71:0/245788455 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa654003d10 con 0x7fa65c109ba0 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa65a000640 1 -- 172.21.15.71:0/245788455 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654010890 con 0x7fa65c109ba0 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65c19f7d0 con 0x7fa65c109ba0 2024-09-05T12:35:58.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.297+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65c1a7060 con 0x7fa65c109ba0 2024-09-05T12:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.300+0000 7fa65a000640 1 -- 172.21.15.71:0/245788455 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa654003fc0 con 0x7fa65c109ba0 2024-09-05T12:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 Processor -- start 2024-09-05T12:35:58.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 -- start start 2024-09-05T12:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790eb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bd79044100 con 0x55bd790e9570 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bd78fec1b0 con 0x55bd790decd0 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa667244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bd790400c0 con 0x55bd790de3b0 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa661a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa661000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa661a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60200/0 (socket says 172.21.15.71:60200) 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.301+0000 7fa661a00640 1 -- 172.21.15.71:0/1516218273 learned_addr learned my addr 172.21.15.71:0/1516218273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661000640 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790eb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661a00640 1 -- 172.21.15.71:0/1516218273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 msgr2=0x55bd790de790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661a00640 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790de790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661a00640 1 -- 172.21.15.71:0/1516218273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 msgr2=0x55bd790eb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661a00640 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790eb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661a00640 1 -- 172.21.15.71:0/1516218273 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd78d57210 con 0x55bd790decd0 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661000640 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa661a00640 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e9030 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa650009d40 tx=0x7fa650011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a79e6274002c8aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa65a000640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa6440a2d70 0x7fa6440a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa64ea00640 1 -- 172.21.15.71:0/1516218273 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650004220 con 0x55bd790decd0 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa64ea00640 1 -- 172.21.15.71:0/1516218273 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6500043c0 con 0x55bd790decd0 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa64ea00640 1 -- 172.21.15.71:0/1516218273 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650004e20 con 0x55bd790decd0 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.302+0000 7fa65a000640 1 -- 172.21.15.71:0/245788455 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fa6540cfd70 con 0x7fa65c109ba0 2024-09-05T12:35:58.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa661000640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa6440a2d70 0x7fa6440a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 -- 172.21.15.71:0/1516218273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 msgr2=0x55bd790e9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e9030 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa650009d40 tx=0x7fa650011810 comp rx=0 tx=0).stop 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa661000640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa6440a2d70 0x7fa6440a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa648006010 tx=0x7fa64802a420 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 -- 172.21.15.71:0/1516218273 shutdown_connections 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e9030 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 --2- 172.21.15.71:0/1516218273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 -- 172.21.15.71:0/1516218273 >> 172.21.15.71:0/1516218273 conn(0x55bd790da920 msgr2=0x55bd790dad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 -- 172.21.15.71:0/1516218273 shutdown_connections 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.303+0000 7fa667244740 1 -- 172.21.15.71:0/1516218273 wait complete. 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.304+0000 7fa667244740 1 -- start start 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.304+0000 7fa667244740 1 Processor -- start 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.304+0000 7fa667244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790e6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.304+0000 7fa667244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.304+0000 7fa667244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa667244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55bd790ec110 con 0x55bd790e9570 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa667244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bd78fcd090 con 0x55bd790decd0 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa667244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bd790ec290 con 0x55bd790de3b0 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa667244740 10 client.0 _ll_register_callbacks cb 0x55bd79021f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790e6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790e6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53308/0 (socket says 172.21.15.71:53308) 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 -- 172.21.15.71:0/2527483472 learned_addr learned my addr 172.21.15.71:0/2527483472 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa662400640 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 -- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 msgr2=0x55bd790e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 -- 172.21.15.71:0/2527483472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 msgr2=0x55bd790e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 -- 172.21.15.71:0/2527483472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd78e23620 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa661a00640 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790e6520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa650000c00 tx=0x7fa650019bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e71abd2f7c8dbd6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.305+0000 7fa643400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa643400640 1 -- 172.21.15.71:0/2527483472 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650028020 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6500043c0 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa643400640 1 -- 172.21.15.71:0/2527483472 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fa650019d70 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa643400640 1 -- 172.21.15.71:0/2527483472 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa650033050 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa667244740 10 client.26332 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bd790f7d90 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa643400640 1 -- 172.21.15.71:0/2527483472 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6500043c0 con 0x55bd790de3b0 2024-09-05T12:35:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.306+0000 7fa643400640 1 -- 172.21.15.71:0/2527483472 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fa650023070 con 0x55bd790de3b0 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa667244740 10 client.26332 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa667244740 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa667244740 20 client.26332 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa667244740 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd790f9200 0x55bd790fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa667244740 4 client.26332 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd79051830 con 0x55bd790f9200 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa661000640 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd790f9200 0x55bd790fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa643400640 10 client.26332 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.307+0000 7fa661000640 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd790f9200 0x55bd790fb5f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4ac93e76995b278 server_cookie=e1dd3d260041d8cd in_seq=0 out_seq=0 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa643400640 1 -- 172.21.15.71:0/2527483472 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd79051830 con 0x55bd790f9200 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa643400640 10 client.26332 handle_command_reply: tid=0 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 client.26332 shutdown 2024-09-05T12:35:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 msgr2=0x55bd790e6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790e6520 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa650000c00 tx=0x7fa650019bd0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 shutdown_connections 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd790f9200 0x55bd790fb5f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790decd0 0x55bd790e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 --2- 172.21.15.71:0/2527483472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790de3b0 0x55bd790e6520 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa643400640 0 client.26332 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.308+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 >> 172.21.15.71:0/2527483472 conn(0x55bd790da920 msgr2=0x55bd790e0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.309+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 shutdown_connections 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.309+0000 7fa667244740 1 -- 172.21.15.71:0/2527483472 wait complete. 2024-09-05T12:35:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.309+0000 7fa667244740 20 client.26332 trim_cache size 0 max 16384 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.325+0000 7fa667244740 1 Processor -- start 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.325+0000 7fa667244740 1 -- start start 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.325+0000 7fa667244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790d98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa667244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd79100e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa667244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd79101380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa667244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1b1120 con 0x55bd790e9570 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa667244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1b2d20 con 0x55bd790de3b0 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa667244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1b17f0 con 0x55bd790decd0 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790d98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790d98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60218/0 (socket says 172.21.15.71:60218) 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 -- 172.21.15.71:0/1962679769 learned_addr learned my addr 172.21.15.71:0/1962679769 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa662400640 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd79101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661000640 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd79100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 -- 172.21.15.71:0/1962679769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 msgr2=0x55bd79100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd79100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 -- 172.21.15.71:0/1962679769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 msgr2=0x55bd79101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd79101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 -- 172.21.15.71:0/1962679769 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd790e54b0 con 0x55bd790de3b0 2024-09-05T12:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.326+0000 7fa661a00640 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790d98e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa65000a010 tx=0x7fa650005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=687b1694b02437d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.327+0000 7fa64e000640 1 -- 172.21.15.71:0/1962679769 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650015020 con 0x55bd790de3b0 2024-09-05T12:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.327+0000 7fa64e000640 1 -- 172.21.15.71:0/1962679769 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa650023eb0 con 0x55bd790de3b0 2024-09-05T12:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.327+0000 7fa64e000640 1 -- 172.21.15.71:0/1962679769 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650028040 con 0x55bd790de3b0 2024-09-05T12:35:58.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 -- 172.21.15.71:0/1962679769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 msgr2=0x55bd790d98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790d98e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa65000a010 tx=0x7fa650005d30 comp rx=0 tx=0).stop 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 -- 172.21.15.71:0/1962679769 shutdown_connections 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd79101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd79100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 --2- 172.21.15.71:0/1962679769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790d98e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 -- 172.21.15.71:0/1962679769 >> 172.21.15.71:0/1962679769 conn(0x55bd790da920 msgr2=0x55bd790d56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 -- 172.21.15.71:0/1962679769 shutdown_connections 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 -- 172.21.15.71:0/1962679769 wait complete. 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 -- start start 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.328+0000 7fa667244740 1 Processor -- start 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790ed310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd790ed850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1b2590 con 0x55bd790e9570 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1b1ec0 con 0x55bd790de3b0 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa65c1af120 con 0x55bd790decd0 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa667244740 10 client.0 _ll_register_callbacks cb 0x7fa650009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd790ed850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa662400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd790ed850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53328/0 (socket says 172.21.15.71:53328) 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 -- 172.21.15.71:0/2048919349 learned_addr learned my addr 172.21.15.71:0/2048919349 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661a00640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790ed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 -- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 msgr2=0x55bd790ed310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790ed310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 -- 172.21.15.71:0/2048919349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 msgr2=0x55bd790edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:35:58.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790edd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661000640 1 -- 172.21.15.71:0/2048919349 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd79162f70 con 0x55bd790decd0 2024-09-05T12:35:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.329+0000 7fa661a00640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790ed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:35:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa661000640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd790ed850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa64802aa10 tx=0x7fa648000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=402709d70553258d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa643400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:35:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa643400640 1 -- 172.21.15.71:0/2048919349 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480317b0 con 0x55bd790decd0 2024-09-05T12:35:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bd7904bb80 con 0x55bd790decd0 2024-09-05T12:35:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa643400640 1 -- 172.21.15.71:0/2048919349 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fa648031950 con 0x55bd790decd0 2024-09-05T12:35:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa643400640 1 -- 172.21.15.71:0/2048919349 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fa648031b50 con 0x55bd790decd0 2024-09-05T12:35:58.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa667244740 10 client.26338 fetch_fsmap learned FSMap version 40 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.330+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55bd791642d0 con 0x55bd790decd0 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa643400640 1 -- 172.21.15.71:0/2048919349 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55bd7904bb80 con 0x55bd790decd0 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa643400640 1 -- 172.21.15.71:0/2048919349 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7fa648044070 con 0x55bd790decd0 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa667244740 10 client.26338 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa667244740 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa667244740 20 client.26338 populate_metadata read hostname 'smithi071' 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa667244740 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd79164af0 0x55bd79166f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa667244740 4 client.26338 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd78fc87c0 con 0x55bd79164af0 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa661a00640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd79164af0 0x55bd79166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa661a00640 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd79164af0 0x55bd79166f10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d0c13c8d3d3abf server_cookie=d4bbeb91a7e0f742 in_seq=0 out_seq=0 2024-09-05T12:35:58.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.331+0000 7fa643400640 10 client.26338 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa643400640 1 -- 172.21.15.71:0/2048919349 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd78fc87c0 con 0x55bd79164af0 2024-09-05T12:35:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa643400640 10 client.26338 handle_command_reply: tid=0 2024-09-05T12:35:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 client.26338 shutdown 2024-09-05T12:35:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 msgr2=0x55bd790ed850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd790ed850 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa64802aa10 tx=0x7fa648000bc0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 shutdown_connections 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55bd79164af0 0x55bd79166f10 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd790e9570 0x55bd790edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa643400640 0 client.26338 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd790decd0 0x55bd790ed850 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 --2- 172.21.15.71:0/2048919349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bd790de3b0 0x55bd790ed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 >> 172.21.15.71:0/2048919349 conn(0x55bd790da920 msgr2=0x55bd790e0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 shutdown_connections 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 1 -- 172.21.15.71:0/2048919349 wait complete. 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.357+0000 7fa667244740 20 client.26338 trim_cache size 0 max 16384 2024-09-05T12:35:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa6440a2d70 msgr2=0x7fa6440a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa6440a2d70 0x7fa6440a5230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa648006010 tx=0x7fa64802a420 comp rx=0 tx=0).stop 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c109ba0 msgr2=0x7fa65c19edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c109ba0 0x7fa65c19edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa65400ec50 tx=0x7fa65400c6a0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 shutdown_connections 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa65c109ba0 0x7fa65c19edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa6440a2d70 0x7fa6440a5230 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa65c1044a0 0x7fa65c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 --2- 172.21.15.71:0/245788455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa65c103b00 0x7fa65c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 >> 172.21.15.71:0/245788455 conn(0x7fa65c0ff7c0 msgr2=0x7fa65c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 shutdown_connections 2024-09-05T12:35:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:35:58.358+0000 7fa662e00640 1 -- 172.21.15.71:0/245788455 wait complete. 2024-09-05T12:35:58.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:58 smithi174 ceph-mon[34313]: pgmap v253: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-05T12:35:58.534 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:58 smithi071 ceph-mon[29845]: pgmap v253: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 -- 172.21.15.71:0/550774949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480101280 msgr2=0x7fc48010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 --2- 172.21.15.71:0/550774949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480101280 0x7fc48010df40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009860 tx=0x7fc46c02a340 comp rx=0 tx=0).stop 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 -- 172.21.15.71:0/550774949 shutdown_connections 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 --2- 172.21.15.71:0/550774949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc48010e480 0x7fc480110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 --2- 172.21.15.71:0/550774949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc480101280 0x7fc48010df40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 --2- 172.21.15.71:0/550774949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4801008d0 0x7fc480100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.552 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 -- 172.21.15.71:0/550774949 >> 172.21.15.71:0/550774949 conn(0x7fc4800fc7a0 msgr2=0x7fc4800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 -- 172.21.15.71:0/550774949 shutdown_connections 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.533+0000 7fc488400640 1 -- 172.21.15.71:0/550774949 wait complete. 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.534+0000 7fc488400640 1 Processor -- start 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.534+0000 7fc488400640 1 -- start start 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc488400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 0x7fc48019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc488400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc480101280 0x7fc48019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc488400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc48010e480 0x7fc48019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc488400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc480114190 con 0x7fc48010e480 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc488400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc480114010 con 0x7fc4801008d0 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc488400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc480114310 con 0x7fc480101280 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc485a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 0x7fc48019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc485a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 0x7fc48019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60236/0 (socket says 172.21.15.71:60236) 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc485a00640 1 -- 172.21.15.71:0/3904028584 learned_addr learned my addr 172.21.15.71:0/3904028584 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc486400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc48010e480 0x7fc48019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.535+0000 7fc485000640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc480101280 0x7fc48019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.536+0000 7fc485a00640 1 -- 172.21.15.71:0/3904028584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc480101280 msgr2=0x7fc48019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.536+0000 7fc485a00640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc480101280 0x7fc48019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.536+0000 7fc485a00640 1 -- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc48010e480 msgr2=0x7fc48019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.536+0000 7fc485a00640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc48010e480 0x7fc48019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.536+0000 7fc485a00640 1 -- 172.21.15.71:0/3904028584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48019f6e0 con 0x7fc4801008d0 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.536+0000 7fc485a00640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 0x7fc48019a730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e9c0 tx=0x7fc47400edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.537+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47400ccc0 con 0x7fc4801008d0 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.537+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc474005500 con 0x7fc4801008d0 2024-09-05T12:35:58.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.537+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474010430 con 0x7fc4801008d0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.537+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48019f9d0 con 0x7fc4801008d0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.537+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc480102ae0 con 0x7fc4801008d0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.539+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc480100cb0 con 0x7fc4801008d0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.540+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc4740105d0 con 0x7fc4801008d0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.542+0000 7fc47e000640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4680a2d70 0x7fc4680a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.542+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7fc4740cfab0 con 0x7fc4801008d0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.542+0000 7fc485000640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4680a2d70 0x7fc4680a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.543+0000 7fc485000640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4680a2d70 0x7fc4680a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc46c02a930 tx=0x7fc46c0058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:35:58.555 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.550+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc47409c2e0 con 0x7fc4801008d0 2024-09-05T12:35:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:58 smithi123 ceph-mon[34321]: pgmap v253: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-09-05T12:35:58.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.821+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc480109610 con 0x7fc4801008d0 2024-09-05T12:35:58.823 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.822+0000 7fc47e000640 1 -- 172.21.15.71:0/3904028584 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11975 (secure 0 0 0) 0x7fc47409bbe0 con 0x7fc4801008d0 2024-09-05T12:35:58.823 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:35:58.823 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":40,"btime":"2024-09-05T12:34:37:175171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:34:36.948618+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:35:58.824 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 40 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4680a2d70 msgr2=0x7fc4680a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4680a2d70 0x7fc4680a5230 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc46c02a930 tx=0x7fc46c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 msgr2=0x7fc48019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 0x7fc48019a730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc47400e9c0 tx=0x7fc47400edc0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 shutdown_connections 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4680a2d70 0x7fc4680a5230 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc48010e480 0x7fc48019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc480101280 0x7fc48019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 --2- 172.21.15.71:0/3904028584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4801008d0 0x7fc48019a730 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.826+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 >> 172.21.15.71:0/3904028584 conn(0x7fc4800fc7a0 msgr2=0x7fc4800feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:35:58.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.827+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 shutdown_connections 2024-09-05T12:35:58.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:35:58.827+0000 7fc488400640 1 -- 172.21.15.71:0/3904028584 wait complete. 2024-09-05T12:35:59.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299665 3.72 MB/sec execute 295 sec latency 120.115 ms 2024-09-05T12:35:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:59 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:35:59.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:35:59 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3904028584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:35:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:59 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:35:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:35:59 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3904028584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:35:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:59 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:35:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:35:59 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3904028584' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:35:59.847 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0' 2024-09-05T12:35:59.847 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0 2024-09-05T12:36:00.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299665 3.71 MB/sec execute 296 sec latency 1120.173 ms 2024-09-05T12:36:00.260 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:00 smithi174 ceph-mon[34313]: pgmap v254: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-09-05T12:36:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:36:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:36:00.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:36:00.599 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:00 smithi071 ceph-mon[29845]: pgmap v254: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-09-05T12:36:00.599 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:36:00.599 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:36:00.600 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:36:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:00 smithi123 ceph-mon[34321]: pgmap v254: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-09-05T12:36:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:36:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:36:00.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:36:00.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 -- 172.21.15.71:0/2952723196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c103ff0 msgr2=0x7f2a0c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.906 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 --2- 172.21.15.71:0/2952723196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c1043d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f29fc009900 tx=0x7f29fc02a410 comp rx=0 tx=0).stop 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 -- 172.21.15.71:0/2952723196 shutdown_connections 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 --2- 172.21.15.71:0/2952723196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c109120 0x7f2a0c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 --2- 172.21.15.71:0/2952723196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c1049a0 0x7f2a0c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 --2- 172.21.15.71:0/2952723196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c1043d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 -- 172.21.15.71:0/2952723196 >> 172.21.15.71:0/2952723196 conn(0x7f2a0c0ffa90 msgr2=0x7f2a0c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 -- 172.21.15.71:0/2952723196 shutdown_connections 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.869+0000 7f2a12000640 1 -- 172.21.15.71:0/2952723196 wait complete. 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.870+0000 7f2a12000640 1 Processor -- start 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.870+0000 7f2a12000640 1 -- start start 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a12000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a12000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c1049a0 0x7f2a0c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a12000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c109120 0x7f2a0c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a12000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c114140 con 0x7f2a0c109120 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a12000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c113fc0 con 0x7f2a0c1049a0 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a12000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1142c0 con 0x7f2a0c103ff0 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a0b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a0b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53364/0 (socket says 172.21.15.71:53364) 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a0b400640 1 -- 172.21.15.71:0/1239293480 learned_addr learned my addr 172.21.15.71:0/1239293480 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c109120 0x7f2a0c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c1049a0 0x7f2a0c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.871+0000 7f2a0b400640 1 -- 172.21.15.71:0/1239293480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c1049a0 msgr2=0x7f2a0c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a0b400640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c1049a0 0x7f2a0c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a0b400640 1 -- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c109120 msgr2=0x7f2a0c19eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a0b400640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c109120 0x7f2a0c19eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a0b400640 1 -- 172.21.15.71:0/1239293480 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a0c19f5d0 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a0b400640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c19a620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006010 tx=0x7f29fc0042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a03e00640 1 -- 172.21.15.71:0/1239293480 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc03f030 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a0c19f860 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a03e00640 1 -- 172.21.15.71:0/1239293480 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29fc003ce0 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a03e00640 1 -- 172.21.15.71:0/1239293480 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29fc035070 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.872+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a0c1a71a0 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.875+0000 7f2a149f0740 1 Processor -- start 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.875+0000 7f2a03e00640 1 -- 172.21.15.71:0/1239293480 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f29fc042030 con 0x7f2a0c103ff0 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.875+0000 7f2a149f0740 1 -- start start 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 0x5604c83ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83b66c0 0x5604c83b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a149f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5604c8310260 con 0x5604c83b66c0 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a149f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604c82b7500 con 0x5604c83abe00 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a149f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604c830c220 con 0x5604c83ab4e0 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 0x5604c83ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 0x5604c83ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53370/0 (socket says 172.21.15.71:53370) 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 -- 172.21.15.71:0/847590114 learned_addr learned my addr 172.21.15.71:0/847590114 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83b66c0 0x5604c83b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0b400640 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 -- 172.21.15.71:0/847590114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 msgr2=0x5604c83b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 -- 172.21.15.71:0/847590114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83b66c0 msgr2=0x5604c83b8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83b66c0 0x5604c83b8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0be00640 1 -- 172.21.15.71:0/847590114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c8081ba0 con 0x5604c83ab4e0 2024-09-05T12:36:00.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.876+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83b66c0 0x5604c83b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f2a0be00640 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 0x5604c83ab8c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009f90 tx=0x7f2a04010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd31165cced0756 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f2a03e00640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29f00a2cf0 0x7f29f00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f29f7e00640 1 -- 172.21.15.71:0/847590114 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04011700 con 0x5604c83ab4e0 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f29f7e00640 1 -- 172.21.15.71:0/847590114 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04011d80 con 0x5604c83ab4e0 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f29f7e00640 1 -- 172.21.15.71:0/847590114 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04018380 con 0x5604c83ab4e0 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f2a03e00640 1 -- 172.21.15.71:0/1239293480 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f29fc0efd70 con 0x7f2a0c103ff0 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.877+0000 7f2a0b400640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29f00a2cf0 0x7f29f00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 -- 172.21.15.71:0/847590114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 msgr2=0x5604c83ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 0x5604c83ab8c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009f90 tx=0x7f2a04010860 comp rx=0 tx=0).stop 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 -- 172.21.15.71:0/847590114 shutdown_connections 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83b66c0 0x5604c83b8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 --2- 172.21.15.71:0/847590114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83ab4e0 0x5604c83ab8c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 -- 172.21.15.71:0/847590114 >> 172.21.15.71:0/847590114 conn(0x5604c83a7a70 msgr2=0x5604c83a7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a0b400640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29f00a2cf0 0x7f29f00a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005ab0 tx=0x7f29fc0f3820 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 -- 172.21.15.71:0/847590114 shutdown_connections 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 -- 172.21.15.71:0/847590114 wait complete. 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.878+0000 7f2a149f0740 1 -- start start 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 Processor -- start 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5604c83b9290 con 0x5604c83ab4e0 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604c82cabd0 con 0x5604c83abe00 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604c83b9410 con 0x5604c83b66c0 2024-09-05T12:36:00.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a149f0740 10 client.0 _ll_register_callbacks cb 0x5604c801f0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40264/0 (socket says 172.21.15.71:40264) 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a0be00640 1 -- 172.21.15.71:0/1432552638 learned_addr learned my addr 172.21.15.71:0/1432552638 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.879+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0aa00640 1 -- 172.21.15.71:0/1432552638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 msgr2=0x5604c83b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0aa00640 1 -- 172.21.15.71:0/1432552638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 msgr2=0x5604c83b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0aa00640 1 -- 172.21.15.71:0/1432552638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c83c2730 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b3b20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0fff20 tx=0x7f29f8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962bf7dfe88fac55 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f29f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f29f4c00640 1 -- 172.21.15.71:0/1432552638 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f8004030 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f29f4c00640 1 -- 172.21.15.71:0/1432552638 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f29f80041d0 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.880+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a04011d80 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f29f4c00640 1 -- 172.21.15.71:0/1432552638 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f29f80043d0 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 10 client.26330 fetch_fsmap learned FSMap version 40 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5604c83c5100 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f29f4c00640 1 -- 172.21.15.71:0/1432552638 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04011d80 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f29f4c00640 1 -- 172.21.15.71:0/1432552638 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7f29f8003c30 con 0x5604c83abe00 2024-09-05T12:36:00.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 10 client.26330 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 10 client.26330 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 20 client.26330 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c83c65c0 0x5604c83c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 4 client.26330 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.881+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604c831d990 con 0x5604c83c65c0 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.882+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c83c65c0 0x5604c83c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.882+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c83c65c0 0x5604c83c8a10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8cd3bc4d0ec953b server_cookie=ee78533c8166df9 in_seq=0 out_seq=0 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.882+0000 7f29f4c00640 10 client.26330 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f29f4c00640 1 -- 172.21.15.71:0/1432552638 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604c831d990 con 0x5604c83c65c0 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f29f4c00640 10 client.26330 handle_command_reply: tid=0 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 client.26330 shutdown 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 msgr2=0x5604c83b3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b3b20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0fff20 tx=0x7f29f8012b60 comp rx=0 tx=0).stop 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 shutdown_connections 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c83c65c0 0x5604c83c8a10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83abe00 0x5604c83b3b20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1432552638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f29f4c00640 0 client.26330 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 >> 172.21.15.71:0/1432552638 conn(0x5604c83a7a70 msgr2=0x5604c83ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.883+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 shutdown_connections 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.884+0000 7f2a149f0740 1 -- 172.21.15.71:0/1432552638 wait complete. 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.884+0000 7f2a149f0740 20 client.26330 trim_cache size 0 max 16384 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.904+0000 7f2a149f0740 1 Processor -- start 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.904+0000 7f2a149f0740 1 -- start start 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.905+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 0x5604c83ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.905+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83abe00 0x5604c83ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.905+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.905+0000 7f2a149f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1b11e0 con 0x5604c83abe00 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.905+0000 7f2a149f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1b2de0 con 0x5604c83ab4e0 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.905+0000 7f2a149f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1b18b0 con 0x5604c83b66c0 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 0x5604c83ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 0x5604c83ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60276/0 (socket says 172.21.15.71:60276) 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 -- 172.21.15.71:0/2712819399 learned_addr learned my addr 172.21.15.71:0/2712819399 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83abe00 0x5604c83ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 -- 172.21.15.71:0/2712819399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 msgr2=0x5604c83b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 -- 172.21.15.71:0/2712819399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83abe00 msgr2=0x5604c83ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83abe00 0x5604c83ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 -- 172.21.15.71:0/2712819399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604c83a1fe0 con 0x5604c83ab4e0 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.906+0000 7f2a0be00640 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 0x5604c83ce320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a040196b0 tx=0x7f2a04019ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16f6166c109c186c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.907+0000 7f29f7400640 1 -- 172.21.15.71:0/2712819399 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04019b40 con 0x5604c83ab4e0 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.907+0000 7f29f7400640 1 -- 172.21.15.71:0/2712819399 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04003e50 con 0x5604c83ab4e0 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.907+0000 7f29f7400640 1 -- 172.21.15.71:0/2712819399 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a040117a0 con 0x5604c83ab4e0 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.907+0000 7f2a149f0740 1 -- 172.21.15.71:0/2712819399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 msgr2=0x5604c83ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.907+0000 7f2a149f0740 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 0x5604c83ce320 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2a040196b0 tx=0x7f2a04019ad0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 -- 172.21.15.71:0/2712819399 shutdown_connections 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83b66c0 0x5604c83b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83abe00 0x5604c83ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 --2- 172.21.15.71:0/2712819399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83ab4e0 0x5604c83ce320 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 -- 172.21.15.71:0/2712819399 >> 172.21.15.71:0/2712819399 conn(0x5604c83a7a70 msgr2=0x5604c83a6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 -- 172.21.15.71:0/2712819399 shutdown_connections 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 -- 172.21.15.71:0/2712819399 wait complete. 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 -- start start 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.908+0000 7f2a149f0740 1 Processor -- start 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83ba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83abe00 0x5604c83bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83b66c0 0x5604c83bb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1b2650 con 0x5604c83ab4e0 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1b1f80 con 0x5604c83b66c0 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a0c1af1e0 con 0x5604c83abe00 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a149f0740 10 client.0 _ll_register_callbacks cb 0x7f29f8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83ba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a0aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83abe00 0x5604c83bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a0be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83ba730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40296/0 (socket says 172.21.15.71:40296) 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a0be00640 1 -- 172.21.15.71:0/1673960635 learned_addr learned my addr 172.21.15.71:0/1673960635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.909+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83b66c0 0x5604c83bb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0be00640 1 -- 172.21.15.71:0/1673960635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83abe00 msgr2=0x5604c83bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83abe00 0x5604c83bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0be00640 1 -- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83b66c0 msgr2=0x5604c83bb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83b66c0 0x5604c83bb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0be00640 1 -- 172.21.15.71:0/1673960635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604c83a8b20 con 0x5604c83ab4e0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83abe00 0x5604c83bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f2a0be00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83ba730 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009d50 tx=0x7f2a04011640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0efea774a81ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.910+0000 7f29f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f29f4c00640 1 -- 172.21.15.71:0/1673960635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04003a60 con 0x5604c83ab4e0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f29f4c00640 1 -- 172.21.15.71:0/1673960635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9438+0+0 (secure 0 0 0) 0x7f2a0401f070 con 0x5604c83ab4e0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a04003e50 con 0x5604c83ab4e0 2024-09-05T12:36:00.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f29f4c00640 1 -- 172.21.15.71:0/1673960635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f2a04003c00 con 0x5604c83ab4e0 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f2a149f0740 10 client.16308 fetch_fsmap learned FSMap version 40 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5604c84315e0 con 0x5604c83ab4e0 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f29f4c00640 1 -- 172.21.15.71:0/1673960635 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a04003e50 con 0x5604c83ab4e0 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f29f4c00640 1 -- 172.21.15.71:0/1673960635 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) ==== 8386+0+0 (secure 0 0 0) 0x7f2a040050b0 con 0x5604c83ab4e0 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f2a149f0740 10 client.16308 fetch_fsmap finished waiting for FSMap version 40 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.911+0000 7f2a149f0740 10 client.16308 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f2a149f0740 20 client.16308 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c8431ad0 0x5604c8433ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f2a149f0740 4 client.16308 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}] 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}) -- 0x5604c8317ce0 con 0x5604c8431ad0 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c8431ad0 0x5604c8433ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f29f4c00640 10 client.16308 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:00.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.912+0000 7f2a0aa00640 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c8431ad0 0x5604c8433ef0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a80dae4196a4ff server_cookie=a389b683be36c173 in_seq=0 out_seq=0 2024-09-05T12:36:00.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.962+0000 7f29f4c00640 1 -- 172.21.15.71:0/1673960635 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5604c8317ce0 con 0x5604c8431ad0 2024-09-05T12:36:00.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.962+0000 7f29f4c00640 10 client.16308 handle_command_reply: tid=0 2024-09-05T12:36:00.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 client.16308 shutdown 2024-09-05T12:36:00.964 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 msgr2=0x5604c83ba730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83ba730 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2a04009d50 tx=0x7f2a04011640 comp rx=0 tx=0).stop 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 shutdown_connections 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5604c8431ad0 0x5604c8433ef0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604c83b66c0 0x5604c83bb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604c83abe00 0x5604c83bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 --2- 172.21.15.71:0/1673960635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604c83ab4e0 0x5604c83ba730 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f29f4c00640 0 client.16308 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.963+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 >> 172.21.15.71:0/1673960635 conn(0x5604c83a7a70 msgr2=0x5604c83acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 shutdown_connections 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a149f0740 1 -- 172.21.15.71:0/1673960635 wait complete. 2024-09-05T12:36:00.965 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a149f0740 20 client.16308 trim_cache size 0 max 16384 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 40, 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 26, 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "917b42cc": { 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 26, 2024-09-05T12:36:00.966 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 146.2, 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:00.967 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:00.968 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:00.968 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29f00a2cf0 msgr2=0x7f29f00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a12000640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29f00a2cf0 0x7f29f00a51b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f29fc005ab0 tx=0x7f29fc0f3820 comp rx=0 tx=0).stop 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 msgr2=0x7f2a0c19a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.964+0000 7f2a12000640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c19a620 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f29fc006010 tx=0x7f29fc0042f0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 shutdown_connections 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29f00a2cf0 0x7f29f00a51b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a0c109120 0x7f2a0c19eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a0c1049a0 0x7f2a0c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 --2- 172.21.15.71:0/1239293480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a0c103ff0 0x7f2a0c19a620 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 >> 172.21.15.71:0/1239293480 conn(0x7f2a0c0ffa90 msgr2=0x7f2a0c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 shutdown_connections 2024-09-05T12:36:00.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:00.965+0000 7f2a12000640 1 -- 172.21.15.71:0/1239293480 wait complete. 2024-09-05T12:36:01.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.69 MB/sec execute 297 sec latency 1819.682 ms 2024-09-05T12:36:01.848 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '917b42cc', quiesce took 4.6 seconds. Will release after: 21.549999999999997 2024-09-05T12:36:02.237 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.68 MB/sec execute 298 sec latency 1284.876 ms 2024-09-05T12:36:02.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:02 smithi174 ceph-mon[34313]: pgmap v255: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-05T12:36:02.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:02 smithi071 ceph-mon[29845]: pgmap v255: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-05T12:36:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:02 smithi123 ceph-mon[34321]: pgmap v255: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-05T12:36:03.248 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.67 MB/sec execute 299 sec latency 2295.165 ms 2024-09-05T12:36:03.380 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:03 smithi071 ceph-mon[29845]: osdmap e100: 12 total, 12 up, 12 in 2024-09-05T12:36:03.380 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:03 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:36:03.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:03 smithi123 ceph-mon[34321]: osdmap e100: 12 total, 12 up, 12 in 2024-09-05T12:36:03.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:03 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:36:03.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:03 smithi174 ceph-mon[34313]: osdmap e100: 12 total, 12 up, 12 in 2024-09-05T12:36:03.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:03 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:36:04.248 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.66 MB/sec execute 300 sec latency 3295.271 ms 2024-09-05T12:36:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:04 smithi071 ceph-mon[29845]: pgmap v257: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-05T12:36:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:04 smithi123 ceph-mon[34321]: pgmap v257: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-05T12:36:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:04 smithi174 ceph-mon[34313]: pgmap v257: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-05T12:36:05.248 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.64 MB/sec execute 301 sec latency 4295.410 ms 2024-09-05T12:36:06.248 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.63 MB/sec execute 302 sec latency 5295.548 ms 2024-09-05T12:36:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:06 smithi071 ceph-mon[29845]: pgmap v258: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-05T12:36:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:06 smithi123 ceph-mon[34321]: pgmap v258: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-05T12:36:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:06 smithi174 ceph-mon[34313]: pgmap v258: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 178 op/s 2024-09-05T12:36:07.248 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.62 MB/sec execute 303 sec latency 6295.696 ms 2024-09-05T12:36:08.248 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.61 MB/sec execute 304 sec latency 7295.825 ms 2024-09-05T12:36:08.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:08 smithi071 ceph-mon[29845]: pgmap v259: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 768 KiB/s wr, 125 op/s 2024-09-05T12:36:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:08 smithi123 ceph-mon[34321]: pgmap v259: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 768 KiB/s wr, 125 op/s 2024-09-05T12:36:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:08 smithi174 ceph-mon[34313]: pgmap v259: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 768 KiB/s wr, 125 op/s 2024-09-05T12:36:09.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.60 MB/sec execute 305 sec latency 8295.943 ms 2024-09-05T12:36:10.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.58 MB/sec execute 306 sec latency 9295.985 ms 2024-09-05T12:36:10.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:10 smithi071 ceph-mon[29845]: pgmap v260: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 768 KiB/s wr, 125 op/s 2024-09-05T12:36:10.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:10 smithi071 ceph-mon[29845]: osdmap e101: 12 total, 12 up, 12 in 2024-09-05T12:36:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:10 smithi123 ceph-mon[34321]: pgmap v260: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 768 KiB/s wr, 125 op/s 2024-09-05T12:36:10.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:10 smithi123 ceph-mon[34321]: osdmap e101: 12 total, 12 up, 12 in 2024-09-05T12:36:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:10 smithi174 ceph-mon[34313]: pgmap v260: 129 pgs: 129 active+clean; 98 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 768 KiB/s wr, 125 op/s 2024-09-05T12:36:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:10 smithi174 ceph-mon[34313]: osdmap e101: 12 total, 12 up, 12 in 2024-09-05T12:36:11.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.57 MB/sec execute 307 sec latency 10296.064 ms 2024-09-05T12:36:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 -- 172.21.15.71:0/6665219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f8102ae0 msgr2=0x7f50f8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 --2- 172.21.15.71:0/6665219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f8102ae0 0x7f50f8102ec0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f50e8009900 tx=0x7f50e802a480 comp rx=0 tx=0).stop 2024-09-05T12:36:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 -- 172.21.15.71:0/6665219 shutdown_connections 2024-09-05T12:36:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 --2- 172.21.15.71:0/6665219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50f810bf00 0x7f50f810e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 --2- 172.21.15.71:0/6665219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 0x7f50f810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 --2- 172.21.15.71:0/6665219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f8102ae0 0x7f50f8102ec0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 -- 172.21.15.71:0/6665219 >> 172.21.15.71:0/6665219 conn(0x7f50f80777c0 msgr2=0x7f50f80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.469+0000 7f50fe600640 1 -- 172.21.15.71:0/6665219 shutdown_connections 2024-09-05T12:36:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.470+0000 7f50fe600640 1 -- 172.21.15.71:0/6665219 wait complete. 2024-09-05T12:36:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.470+0000 7f50fe600640 1 Processor -- start 2024-09-05T12:36:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 -- start start 2024-09-05T12:36:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50f8102ae0 0x7f50f819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 0x7f50f819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 0x7f50f819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111d20 con 0x7f50f810bf00 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111ba0 con 0x7f50f8102ae0 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fe600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f8111ea0 con 0x7f50f8103400 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 0x7f50f819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fdc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 0x7f50f819ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51964/0 (socket says 172.21.15.71:51964) 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fdc00640 1 -- 172.21.15.71:0/369300295 learned_addr learned my addr 172.21.15.71:0/369300295 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50efe00640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 0x7f50f819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.471+0000 7f50fd200640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50f8102ae0 0x7f50f819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fdc00640 1 -- 172.21.15.71:0/369300295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 msgr2=0x7f50f819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fdc00640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 0x7f50f819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fdc00640 1 -- 172.21.15.71:0/369300295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50f8102ae0 msgr2=0x7f50f819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fdc00640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50f8102ae0 0x7f50f819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fdc00640 1 -- 172.21.15.71:0/369300295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f819f430 con 0x7f50f810bf00 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50efe00640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 0x7f50f819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fdc00640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 0x7f50f819ed00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f50f000eb10 tx=0x7f50f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50f6000640 1 -- 172.21.15.71:0/369300295 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0004b20 con 0x7f50f810bf00 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50f6000640 1 -- 172.21.15.71:0/369300295 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50f0004cc0 con 0x7f50f810bf00 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50f6000640 1 -- 172.21.15.71:0/369300295 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0010760 con 0x7f50f810bf00 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f819f720 con 0x7f50f810bf00 2024-09-05T12:36:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.472+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f81a6fb0 con 0x7f50f810bf00 2024-09-05T12:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.475+0000 7f50f6000640 1 -- 172.21.15.71:0/369300295 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f50f0005020 con 0x7f50f810bf00 2024-09-05T12:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.475+0000 7f5102c07740 1 Processor -- start 2024-09-05T12:36:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.475+0000 7f5102c07740 1 -- start start 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f5102c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 0x563b6f18d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f5102c07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18dcd0 0x563b6f198030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f5102c07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f19a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f5102c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563b6f0f3100 con 0x563b6f18d3b0 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f5102c07740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b6f09b1b0 con 0x563b6f18dcd0 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f5102c07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b6f0ef0c0 con 0x563b6f198570 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 0x563b6f18d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 0x563b6f18d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51970/0 (socket says 172.21.15.71:51970) 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 -- 172.21.15.71:0/2582957810 learned_addr learned my addr 172.21.15.71:0/2582957810 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50fd200640 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18dcd0 0x563b6f198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 -- 172.21.15.71:0/2582957810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 msgr2=0x563b6f19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 -- 172.21.15.71:0/2582957810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18dcd0 msgr2=0x563b6f198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18dcd0 0x563b6f198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.476+0000 7f50efe00640 1 -- 172.21.15.71:0/2582957810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6ee06210 con 0x563b6f18d3b0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50efe00640 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 0x563b6f18d790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009710 tx=0x7f50e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da44efef5eb85a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50ee000640 1 -- 172.21.15.71:0/2582957810 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0004050 con 0x563b6f18d3b0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50f6000640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f50dc0a2cf0 0x7f50dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50ee000640 1 -- 172.21.15.71:0/2582957810 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e000fe20 con 0x563b6f18d3b0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50ee000640 1 -- 172.21.15.71:0/2582957810 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0003c20 con 0x563b6f18d3b0 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50f6000640 1 -- 172.21.15.71:0/369300295 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f50f00cfd30 con 0x7f50f810bf00 2024-09-05T12:36:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.477+0000 7f50fd200640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f50dc0a2cf0 0x7f50dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 -- 172.21.15.71:0/2582957810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 msgr2=0x563b6f18d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 0x563b6f18d790 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009710 tx=0x7f50e0009c00 comp rx=0 tx=0).stop 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 -- 172.21.15.71:0/2582957810 shutdown_connections 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18dcd0 0x563b6f198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 --2- 172.21.15.71:0/2582957810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18d3b0 0x563b6f18d790 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 -- 172.21.15.71:0/2582957810 >> 172.21.15.71:0/2582957810 conn(0x563b6f189920 msgr2=0x563b6f189d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 -- 172.21.15.71:0/2582957810 shutdown_connections 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f50fd200640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f50dc0a2cf0 0x7f50dc0a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50e8000c00 tx=0x7f50e8005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 -- 172.21.15.71:0/2582957810 wait complete. 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 -- start start 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.478+0000 7f5102c07740 1 Processor -- start 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f195680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f196100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563b6f19aff0 con 0x563b6f18dcd0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b6f07c090 con 0x563b6f18d3b0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b6f19b170 con 0x563b6f198570 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f5102c07740 10 client.0 _ll_register_callbacks cb 0x563b6f0d0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51978/0 (socket says 172.21.15.71:51978) 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 -- 172.21.15.71:0/1423251039 learned_addr learned my addr 172.21.15.71:0/1423251039 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50fd200640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50fdc00640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 -- 172.21.15.71:0/1423251039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 msgr2=0x563b6f196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 -- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 msgr2=0x563b6f195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.479+0000 7f50efe00640 1 -- 172.21.15.71:0/1423251039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6eed2620 con 0x563b6f18dcd0 2024-09-05T12:36:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f50efe00640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f195bc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f50e0005df0 tx=0x7f50e000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b770c919c0d273 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f50e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f50e6a00640 1 -- 172.21.15.71:0/1423251039 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0005440 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e000fe20 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f50e6a00640 1 -- 172.21.15.71:0/1423251039 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f50e00055e0 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f50e6a00640 1 -- 172.21.15.71:0/1423251039 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f50e0031050 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f5102c07740 10 client.16320 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.480+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563b6f1a6de0 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f50e6a00640 1 -- 172.21.15.71:0/1423251039 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e000fe20 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f50e6a00640 1 -- 172.21.15.71:0/1423251039 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f50e001e070 con 0x563b6f18dcd0 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f5102c07740 10 client.16320 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f5102c07740 10 client.16320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f5102c07740 20 client.16320 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f5102c07740 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f1a8130 0x563b6f1aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f5102c07740 4 client.16320 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b6f100830 con 0x563b6f1a8130 2024-09-05T12:36:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.481+0000 7f50fd200640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f1a8130 0x563b6f1aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.482+0000 7f50fd200640 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f1a8130 0x563b6f1aa540 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa27647758089fb server_cookie=2ccd3779f6b62885 in_seq=0 out_seq=0 2024-09-05T12:36:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.482+0000 7f50e6a00640 10 client.16320 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f50e6a00640 1 -- 172.21.15.71:0/1423251039 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b6f100830 con 0x563b6f1a8130 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f50e6a00640 10 client.16320 handle_command_reply: tid=0 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 client.16320 shutdown 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 msgr2=0x563b6f195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f195bc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f50e0005df0 tx=0x7f50e000fd40 comp rx=0 tx=0).stop 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 shutdown_connections 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f1a8130 0x563b6f1aa540 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f195bc0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f50e6a00640 0 client.16320 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 --2- 172.21.15.71:0/1423251039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 >> 172.21.15.71:0/1423251039 conn(0x563b6f189920 msgr2=0x563b6f18f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 shutdown_connections 2024-09-05T12:36:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 1 -- 172.21.15.71:0/1423251039 wait complete. 2024-09-05T12:36:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.483+0000 7f5102c07740 20 client.16320 trim_cache size 0 max 16384 2024-09-05T12:36:11.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 Processor -- start 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 -- start start 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f18ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f1afe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b0fe0 con 0x563b6f18dcd0 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b2be0 con 0x563b6f18d3b0 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f5102c07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b16b0 con 0x563b6f198570 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51992/0 (socket says 172.21.15.71:51992) 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.498+0000 7f50efe00640 1 -- 172.21.15.71:0/2860357126 learned_addr learned my addr 172.21.15.71:0/2860357126 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50fd200640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f18ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50fdc00640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f1afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50efe00640 1 -- 172.21.15.71:0/2860357126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 msgr2=0x563b6f1afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50efe00640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f1afe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50efe00640 1 -- 172.21.15.71:0/2860357126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 msgr2=0x563b6f18ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50efe00640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f18ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50efe00640 1 -- 172.21.15.71:0/2860357126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6f1891b0 con 0x563b6f18dcd0 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50fdc00640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f1afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:36:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50fd200640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f18ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50efe00640 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1974e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f50e0010830 tx=0x7f50e0003870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ee5f9251cfefff server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50ed600640 1 -- 172.21.15.71:0/2860357126 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e002e020 con 0x563b6f18dcd0 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50ed600640 1 -- 172.21.15.71:0/2860357126 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e00137e0 con 0x563b6f18dcd0 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.499+0000 7f50ed600640 1 -- 172.21.15.71:0/2860357126 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0010a30 con 0x563b6f18dcd0 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 -- 172.21.15.71:0/2860357126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 msgr2=0x563b6f1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1974e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f50e0010830 tx=0x7f50e0003870 comp rx=0 tx=0).stop 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 -- 172.21.15.71:0/2860357126 shutdown_connections 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f1afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1974e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 --2- 172.21.15.71:0/2860357126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f18ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 -- 172.21.15.71:0/2860357126 >> 172.21.15.71:0/2860357126 conn(0x563b6f189920 msgr2=0x563b6f196dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 -- 172.21.15.71:0/2860357126 shutdown_connections 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 -- 172.21.15.71:0/2860357126 wait complete. 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 -- start start 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.500+0000 7f5102c07740 1 Processor -- start 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1924f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f192c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b2450 con 0x563b6f18dcd0 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81b1d80 con 0x563b6f18d3b0 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50f81aefe0 con 0x563b6f198570 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f5102c07740 10 client.0 _ll_register_callbacks cb 0x7f50e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1924f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1924f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52006/0 (socket says 172.21.15.71:52006) 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 -- 172.21.15.71:0/537853000 learned_addr learned my addr 172.21.15.71:0/537853000 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50fd200640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50fdc00640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f192c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 -- 172.21.15.71:0/537853000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 msgr2=0x563b6f192c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f192c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 -- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 msgr2=0x563b6f1a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f1a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.501+0000 7f50efe00640 1 -- 172.21.15.71:0/537853000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6f211de0 con 0x563b6f18dcd0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f50efe00640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1924f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50e00096e0 tx=0x7f50e0003800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1341653d87b611ae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f50e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f50e6a00640 1 -- 172.21.15.71:0/537853000 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0010430 con 0x563b6f18dcd0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f50e6a00640 1 -- 172.21.15.71:0/537853000 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f50e0005240 con 0x563b6f18dcd0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e00137e0 con 0x563b6f18dcd0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f50e6a00640 1 -- 172.21.15.71:0/537853000 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f50e0005a70 con 0x563b6f18dcd0 2024-09-05T12:36:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f5102c07740 10 client.16326 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.502+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563b6f212f50 con 0x563b6f18dcd0 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f50e6a00640 1 -- 172.21.15.71:0/537853000 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50e00137e0 con 0x563b6f18dcd0 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f50e6a00640 1 -- 172.21.15.71:0/537853000 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f50e001e030 con 0x563b6f18dcd0 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f5102c07740 10 client.16326 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f5102c07740 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f5102c07740 20 client.16326 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f5102c07740 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f2136a0 0x563b6f215ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f5102c07740 4 client.16326 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b6f0fab80 con 0x563b6f2136a0 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.503+0000 7f50fd200640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f2136a0 0x563b6f215ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.504+0000 7f50fd200640 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f2136a0 0x563b6f215ac0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b707b085d8d79a server_cookie=601dec43188f878 in_seq=0 out_seq=0 2024-09-05T12:36:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.504+0000 7f50e6a00640 10 client.16326 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f50e6a00640 1 -- 172.21.15.71:0/537853000 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b6f0fab80 con 0x563b6f2136a0 2024-09-05T12:36:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f50e6a00640 10 client.16326 handle_command_reply: tid=0 2024-09-05T12:36:11.529 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 client.16326 shutdown 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 msgr2=0x563b6f1924f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1924f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f50e00096e0 tx=0x7f50e0003800 comp rx=0 tx=0).stop 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 shutdown_connections 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563b6f2136a0 0x563b6f215ac0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6f198570 0x563b6f192c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b6f18dcd0 0x563b6f1924f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f50e6a00640 0 client.16326 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 --2- 172.21.15.71:0/537853000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b6f18d3b0 0x563b6f1a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 >> 172.21.15.71:0/537853000 conn(0x563b6f189920 msgr2=0x563b6f1889c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.527+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 shutdown_connections 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f5102c07740 1 -- 172.21.15.71:0/537853000 wait complete. 2024-09-05T12:36:11.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f5102c07740 20 client.16326 trim_cache size 0 max 16384 2024-09-05T12:36:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:36:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:36:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:36:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:36:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f50dc0a2cf0 msgr2=0x7f50dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f50dc0a2cf0 0x7f50dc0a51b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50e8000c00 tx=0x7f50e8005b50 comp rx=0 tx=0).stop 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 msgr2=0x7f50f819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 0x7f50f819ed00 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f50f000eb10 tx=0x7f50f000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 shutdown_connections 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f50dc0a2cf0 0x7f50dc0a51b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50f810bf00 0x7f50f819ed00 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50f8103400 0x7f50f819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 --2- 172.21.15.71:0/369300295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50f8102ae0 0x7f50f819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.528+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 >> 172.21.15.71:0/369300295 conn(0x7f50f80777c0 msgr2=0x7f50f8100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.529+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 shutdown_connections 2024-09-05T12:36:11.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:11.529+0000 7f50fe600640 1 -- 172.21.15.71:0/369300295 wait complete. 2024-09-05T12:36:12.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.56 MB/sec execute 308 sec latency 11296.163 ms 2024-09-05T12:36:12.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:12 smithi071 ceph-mon[29845]: pgmap v262: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 772 KiB/s wr, 65 op/s 2024-09-05T12:36:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:12 smithi123 ceph-mon[34321]: pgmap v262: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 772 KiB/s wr, 65 op/s 2024-09-05T12:36:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:12 smithi174 ceph-mon[34313]: pgmap v262: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 772 KiB/s wr, 65 op/s 2024-09-05T12:36:13.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.55 MB/sec execute 309 sec latency 12296.252 ms 2024-09-05T12:36:14.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.54 MB/sec execute 310 sec latency 13296.391 ms 2024-09-05T12:36:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:14 smithi071 ceph-mon[29845]: pgmap v263: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 663 KiB/s wr, 56 op/s 2024-09-05T12:36:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:14 smithi123 ceph-mon[34321]: pgmap v263: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 663 KiB/s wr, 56 op/s 2024-09-05T12:36:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:14 smithi174 ceph-mon[34313]: pgmap v263: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 663 KiB/s wr, 56 op/s 2024-09-05T12:36:15.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.53 MB/sec execute 311 sec latency 14296.536 ms 2024-09-05T12:36:16.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.52 MB/sec execute 312 sec latency 15296.676 ms 2024-09-05T12:36:16.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:16 smithi071 ceph-mon[29845]: pgmap v264: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 663 KiB/s wr, 56 op/s 2024-09-05T12:36:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:16 smithi123 ceph-mon[34321]: pgmap v264: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 663 KiB/s wr, 56 op/s 2024-09-05T12:36:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:16 smithi174 ceph-mon[34313]: pgmap v264: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 663 KiB/s wr, 56 op/s 2024-09-05T12:36:17.249 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.50 MB/sec execute 313 sec latency 16296.808 ms 2024-09-05T12:36:18.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.49 MB/sec execute 314 sec latency 17296.934 ms 2024-09-05T12:36:18.559 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:18 smithi071 ceph-mon[29845]: pgmap v265: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 689 KiB/s wr, 47 op/s 2024-09-05T12:36:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:18 smithi123 ceph-mon[34321]: pgmap v265: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 689 KiB/s wr, 47 op/s 2024-09-05T12:36:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:18 smithi174 ceph-mon[34313]: pgmap v265: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 689 KiB/s wr, 47 op/s 2024-09-05T12:36:19.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.48 MB/sec execute 315 sec latency 18297.039 ms 2024-09-05T12:36:20.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.47 MB/sec execute 316 sec latency 19297.149 ms 2024-09-05T12:36:20.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:20 smithi174 ceph-mon[34313]: pgmap v266: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 689 KiB/s wr, 47 op/s 2024-09-05T12:36:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:20 smithi071 ceph-mon[29845]: pgmap v266: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 689 KiB/s wr, 47 op/s 2024-09-05T12:36:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:20 smithi123 ceph-mon[34321]: pgmap v266: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 689 KiB/s wr, 47 op/s 2024-09-05T12:36:21.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.46 MB/sec execute 317 sec latency 20297.258 ms 2024-09-05T12:36:22.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.45 MB/sec execute 318 sec latency 21297.399 ms 2024-09-05T12:36:22.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:22 smithi071 ceph-mon[29845]: pgmap v267: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1011 KiB/s wr, 63 op/s 2024-09-05T12:36:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:22 smithi123 ceph-mon[34321]: pgmap v267: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1011 KiB/s wr, 63 op/s 2024-09-05T12:36:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:22 smithi174 ceph-mon[34313]: pgmap v267: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1011 KiB/s wr, 63 op/s 2024-09-05T12:36:23.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.44 MB/sec execute 319 sec latency 22297.580 ms 2024-09-05T12:36:23.399 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:36:23.841 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:24.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.43 MB/sec execute 320 sec latency 23297.667 ms 2024-09-05T12:36:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 -- 172.21.15.71:0/852217124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c106f10 msgr2=0x7f895c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 --2- 172.21.15.71:0/852217124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c106f10 0x7f895c10ea90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8954009be0 tx=0x7f895402a6c0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 -- 172.21.15.71:0/852217124 shutdown_connections 2024-09-05T12:36:24.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 --2- 172.21.15.71:0/852217124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c106f10 0x7f895c10ea90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 --2- 172.21.15.71:0/852217124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c102790 0x7f895c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 --2- 172.21.15.71:0/852217124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c101de0 0x7f895c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 -- 172.21.15.71:0/852217124 >> 172.21.15.71:0/852217124 conn(0x7f895c0fd8c0 msgr2=0x7f895c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 -- 172.21.15.71:0/852217124 shutdown_connections 2024-09-05T12:36:24.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.465+0000 7f8961c00640 1 -- 172.21.15.71:0/852217124 wait complete. 2024-09-05T12:36:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.466+0000 7f8961c00640 1 Processor -- start 2024-09-05T12:36:24.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.466+0000 7f8961c00640 1 -- start start 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.466+0000 7f8961c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 0x7f895c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.466+0000 7f8961c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c102790 0x7f895c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.466+0000 7f8961c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c106f10 0x7f895c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f8961c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f895c111f70 con 0x7f895c101de0 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f8961c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f895c111df0 con 0x7f895c106f10 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f8961c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f895c1120f0 con 0x7f895c102790 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 0x7f895c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 0x7f895c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38794/0 (socket says 172.21.15.71:38794) 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 -- 172.21.15.71:0/581769071 learned_addr learned my addr 172.21.15.71:0/581769071 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895a000640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c102790 0x7f895c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895b400640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c106f10 0x7f895c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 -- 172.21.15.71:0/581769071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c102790 msgr2=0x7f895c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c102790 0x7f895c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 -- 172.21.15.71:0/581769071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c106f10 msgr2=0x7f895c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c106f10 0x7f895c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.467+0000 7f895aa00640 1 -- 172.21.15.71:0/581769071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c19f560 con 0x7f895c101de0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.468+0000 7f895aa00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 0x7f895c19a5b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f894c00e9f0 tx=0x7f894c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.468+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c00cd80 con 0x7f895c101de0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.468+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c005500 con 0x7f895c101de0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.468+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c19f850 con 0x7f895c101de0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.468+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c010690 con 0x7f895c101de0 2024-09-05T12:36:24.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.468+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c1a7130 con 0x7f895c101de0 2024-09-05T12:36:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.469+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c1021c0 con 0x7f895c101de0 2024-09-05T12:36:24.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.471+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f894c0046c0 con 0x7f895c101de0 2024-09-05T12:36:24.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.472+0000 7f8953400640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f89400a2cf0 0x7f89400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.473+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f894c0cfc20 con 0x7f895c101de0 2024-09-05T12:36:24.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.473+0000 7f895a000640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f89400a2cf0 0x7f89400a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.473+0000 7f895a000640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f89400a2cf0 0x7f89400a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f89480096c0 tx=0x7f8948009260 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.477+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f894c09c450 con 0x7f895c101de0 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.647+0000 7f6b8a600640 1 -- 172.21.15.71:0/3776120228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b84103f50 msgr2=0x7f6b84104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.647+0000 7f6b8a600640 1 --2- 172.21.15.71:0/3776120228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b84103f50 0x7f6b84104330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b7402a410 comp rx=0 tx=0).stop 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 -- 172.21.15.71:0/3776120228 shutdown_connections 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 --2- 172.21.15.71:0/3776120228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b841090a0 0x7f6b84110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 --2- 172.21.15.71:0/3776120228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b84104900 0x7f6b84108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 --2- 172.21.15.71:0/3776120228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b84103f50 0x7f6b84104330 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 -- 172.21.15.71:0/3776120228 >> 172.21.15.71:0/3776120228 conn(0x7f6b840ff9e0 msgr2=0x7f6b84101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 -- 172.21.15.71:0/3776120228 shutdown_connections 2024-09-05T12:36:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.648+0000 7f6b8a600640 1 -- 172.21.15.71:0/3776120228 wait complete. 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 Processor -- start 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 -- start start 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84103f50 0x7f6b8419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b841090a0 0x7f6b8419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 0x7f6b8419f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84114130 con 0x7f6b8419eec0 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84113fb0 con 0x7f6b84103f50 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b8a600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841142b0 con 0x7f6b841090a0 2024-09-05T12:36:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84103f50 0x7f6b8419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b83e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 0x7f6b8419f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b82a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b841090a0 0x7f6b8419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b83e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 0x7f6b8419f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38812/0 (socket says 172.21.15.71:38812) 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b83e00640 1 -- 172.21.15.71:0/1079542268 learned_addr learned my addr 172.21.15.71:0/1079542268 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.649+0000 7f6b83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84103f50 0x7f6b8419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34736/0 (socket says 172.21.15.71:34736) 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b83e00640 1 -- 172.21.15.71:0/1079542268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b841090a0 msgr2=0x7f6b8419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b83e00640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b841090a0 0x7f6b8419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b83e00640 1 -- 172.21.15.71:0/1079542268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84103f50 msgr2=0x7f6b8419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b83e00640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84103f50 0x7f6b8419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b83e00640 1 -- 172.21.15.71:0/1079542268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8419f9b0 con 0x7f6b8419eec0 2024-09-05T12:36:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b83e00640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 0x7f6b8419f370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00eb10 tx=0x7f6b7c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b7be00640 1 -- 172.21.15.71:0/1079542268 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c004b20 con 0x7f6b8419eec0 2024-09-05T12:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b7be00640 1 -- 172.21.15.71:0/1079542268 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b7c004cc0 con 0x7f6b8419eec0 2024-09-05T12:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b7be00640 1 -- 172.21.15.71:0/1079542268 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c010760 con 0x7f6b8419eec0 2024-09-05T12:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8419fca0 con 0x7f6b8419eec0 2024-09-05T12:36:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.650+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b841a7440 con 0x7f6b8419eec0 2024-09-05T12:36:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.653+0000 7f6b8d4a7740 1 Processor -- start 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.653+0000 7f6b7be00640 1 -- 172.21.15.71:0/1079542268 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6b7c005020 con 0x7f6b8419eec0 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 -- start start 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33d45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33e1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559fb333a100 con 0x559fb33d4b00 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559fb32e21b0 con 0x559fb33df3a0 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559fb33360c0 con 0x559fb33d41e0 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b82a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33d45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b83400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b82a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33d45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52934/0 (socket says 172.21.15.71:52934) 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b82a00640 1 -- 172.21.15.71:0/941964143 learned_addr learned my addr 172.21.15.71:0/941964143 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.654+0000 7f6b82a00640 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b83400640 1 -- 172.21.15.71:0/941964143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 msgr2=0x559fb33d45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b83400640 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33d45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b83400640 1 -- 172.21.15.71:0/941964143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 msgr2=0x559fb33e1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b83400640 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33e1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b83400640 1 -- 172.21.15.71:0/941964143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb304d210 con 0x559fb33d4b00 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b7be00640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b680a2da0 0x7f6b680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b82a00640 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b7be00640 1 -- 172.21.15.71:0/1079542268 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f6b7c0cfd30 con 0x7f6b8419eec0 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b83400640 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dee60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402aa00 tx=0x7f6b74005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e89c8f7f5c0b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.655+0000 7f6b82a00640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b680a2da0 0x7f6b680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b78c00640 1 -- 172.21.15.71:0/941964143 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74035040 con 0x559fb33d4b00 2024-09-05T12:36:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b78c00640 1 -- 172.21.15.71:0/941964143 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b74004400 con 0x559fb33d4b00 2024-09-05T12:36:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b78c00640 1 -- 172.21.15.71:0/941964143 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b740035f0 con 0x559fb33d4b00 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/941964143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 msgr2=0x559fb33dee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b82a00640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b680a2da0 0x7f6b680a5260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x559fb33d1ad0 tx=0x7f6b70005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dee60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402aa00 tx=0x7f6b74005de0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/941964143 shutdown_connections 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33e1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dee60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/941964143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33d45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.656+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/941964143 >> 172.21.15.71:0/941964143 conn(0x559fb33d0750 msgr2=0x559fb33d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.657+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/941964143 shutdown_connections 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.657+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/941964143 wait complete. 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.657+0000 7f6b8d4a7740 1 -- start start 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.657+0000 7f6b8d4a7740 1 Processor -- start 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559fb33e1f70 con 0x559fb33d4b00 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559fb32c3090 con 0x559fb33df3a0 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559fb33e20f0 con 0x559fb33d41e0 2024-09-05T12:36:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b8d4a7740 10 client.0 _ll_register_callbacks cb 0x559fb3317f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b82a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33dc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52936/0 (socket says 172.21.15.71:52936) 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 -- 172.21.15.71:0/2553806544 learned_addr learned my addr 172.21.15.71:0/2553806544 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83e00640 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 -- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 msgr2=0x559fb33dcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33dcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 -- 172.21.15.71:0/2553806544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 msgr2=0x559fb33dc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.658+0000 7f6b83400640 1 -- 172.21.15.71:0/2553806544 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb3119620 con 0x559fb33d41e0 2024-09-05T12:36:24.664 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:24 smithi071 ceph-mon[29845]: pgmap v268: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 632 KiB/s wr, 42 op/s 2024-09-05T12:36:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b83400640 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33dc420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b74004930 tx=0x7f6b74004960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c29e8b82e33b7357 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b6d600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b6d600640 1 -- 172.21.15.71:0/2553806544 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74052060 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b74004400 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b6d600640 1 -- 172.21.15.71:0/2553806544 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f6b74004200 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b6d600640 1 -- 172.21.15.71:0/2553806544 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6b74035070 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b8d4a7740 10 client.26398 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.659+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559fb33e1f70 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b6d600640 1 -- 172.21.15.71:0/2553806544 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b74004400 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b6d600640 1 -- 172.21.15.71:0/2553806544 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f6b74004fb0 con 0x559fb33d41e0 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b8d4a7740 10 client.26398 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:24.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b8d4a7740 10 client.26398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b8d4a7740 20 client.26398 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb33eed90 0x559fb33f1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b8d4a7740 4 client.26398 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.660+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fb3347830 con 0x559fb33eed90 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.661+0000 7f6b82a00640 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb33eed90 0x559fb33f1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.661+0000 7f6b82a00640 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb33eed90 0x559fb33f1180 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c158bbd64bc824ac server_cookie=4c43606a85d1b31c in_seq=0 out_seq=0 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.661+0000 7f6b6d600640 10 client.26398 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b6d600640 1 -- 172.21.15.71:0/2553806544 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fb3347830 con 0x559fb33eed90 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b6d600640 10 client.26398 handle_command_reply: tid=0 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 client.26398 shutdown 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 msgr2=0x559fb33dc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33dc420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b74004930 tx=0x7f6b74004960 comp rx=0 tx=0).stop 2024-09-05T12:36:24.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 shutdown_connections 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb33eed90 0x559fb33f1180 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33df3a0 0x559fb33dcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33dc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b6d600640 0 client.26398 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2553806544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d41e0 0x559fb33dc420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 >> 172.21.15.71:0/2553806544 conn(0x559fb33d0750 msgr2=0x559fb33d6460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 shutdown_connections 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2553806544 wait complete. 2024-09-05T12:36:24.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.662+0000 7f6b8d4a7740 20 client.26398 trim_cache size 0 max 16384 2024-09-05T12:36:24.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:24 smithi123 ceph-mon[34321]: pgmap v268: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 632 KiB/s wr, 42 op/s 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 Processor -- start 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 -- start start 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33f6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33df3a0 0x559fb33d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841b1110 con 0x559fb33d4b00 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841b2d10 con 0x559fb33d41e0 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841b17e0 con 0x559fb33df3a0 2024-09-05T12:36:24.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34778/0 (socket says 172.21.15.71:34778) 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b83400640 1 -- 172.21.15.71:0/2654479486 learned_addr learned my addr 172.21.15.71:0/2654479486 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.678+0000 7f6b82a00640 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33f6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83e00640 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33df3a0 0x559fb33d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83400640 1 -- 172.21.15.71:0/2654479486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33df3a0 msgr2=0x559fb33d9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83400640 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33df3a0 0x559fb33d9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83400640 1 -- 172.21.15.71:0/2654479486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 msgr2=0x559fb33f6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83400640 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33f6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83400640 1 -- 172.21.15.71:0/2654479486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fb33d4930 con 0x559fb33d41e0 2024-09-05T12:36:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b83400640 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33f6a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b74005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10787f75ec0e8ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b6fe00640 1 -- 172.21.15.71:0/2654479486 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74052070 con 0x559fb33d41e0 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b6fe00640 1 -- 172.21.15.71:0/2654479486 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b74009520 con 0x559fb33d41e0 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.679+0000 7f6b6fe00640 1 -- 172.21.15.71:0/2654479486 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74003b30 con 0x559fb33d41e0 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2654479486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 msgr2=0x559fb33f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33f6a00 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b74005de0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2654479486 shutdown_connections 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33df3a0 0x559fb33d9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33d4b00 0x559fb33f6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/2654479486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33f6a00 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2654479486 >> 172.21.15.71:0/2654479486 conn(0x559fb33d0750 msgr2=0x559fb33db4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2654479486 shutdown_connections 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/2654479486 wait complete. 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.680+0000 7f6b8d4a7740 1 -- start start 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 Processor -- start 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33e2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d4b00 0x559fb33e2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33df3a0 0x559fb3451a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841b2580 con 0x559fb33df3a0 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841b1eb0 con 0x559fb33d41e0 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841af110 con 0x559fb33d4b00 2024-09-05T12:36:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b8d4a7740 10 client.0 _ll_register_callbacks cb 0x7f6b70009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33e2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b83400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33e2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34792/0 (socket says 172.21.15.71:34792) 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b83400640 1 -- 172.21.15.71:0/551291064 learned_addr learned my addr 172.21.15.71:0/551291064 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b82a00640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d4b00 0x559fb33e2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.681+0000 7f6b83e00640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33df3a0 0x559fb3451a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b83400640 1 -- 172.21.15.71:0/551291064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d4b00 msgr2=0x559fb33e2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b83400640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d4b00 0x559fb33e2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b83400640 1 -- 172.21.15.71:0/551291064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33df3a0 msgr2=0x559fb3451a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b83400640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33df3a0 0x559fb3451a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b83400640 1 -- 172.21.15.71:0/551291064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fb3452040 con 0x559fb33d41e0 2024-09-05T12:36:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b82a00640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d4b00 0x559fb33e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b83400640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33e2910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402a450 tx=0x7f6b7402ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f110bb3b3ea95b1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b6d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b6d600640 1 -- 172.21.15.71:0/551291064 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74052070 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b74009520 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.682+0000 7f6b6d600640 1 -- 172.21.15.71:0/551291064 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f6b74004eb0 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b6d600640 1 -- 172.21.15.71:0/551291064 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6b740050b0 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 10 client.26378 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559fb34593c0 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b6d600640 1 -- 172.21.15.71:0/551291064 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b74009520 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b6d600640 1 -- 172.21.15.71:0/551291064 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f6b74031b60 con 0x559fb33d41e0 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 10 client.26378 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 10 client.26378 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 20 client.26378 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb3459970 0x559fb345bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 4 client.26378 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559fb3341b80 con 0x559fb3459970 2024-09-05T12:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.683+0000 7f6b82a00640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb3459970 0x559fb345bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.684+0000 7f6b82a00640 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb3459970 0x559fb345bd90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe135f745b760bee server_cookie=538a70f0fdd154f4 in_seq=0 out_seq=0 2024-09-05T12:36:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.684+0000 7f6b6d600640 10 client.26378 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b6d600640 1 -- 172.21.15.71:0/551291064 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559fb3341b80 con 0x559fb3459970 2024-09-05T12:36:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b6d600640 10 client.26378 handle_command_reply: tid=0 2024-09-05T12:36:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 client.26378 shutdown 2024-09-05T12:36:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 msgr2=0x559fb33e2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33e2910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6b7402a450 tx=0x7f6b7402ab90 comp rx=0 tx=0).stop 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 shutdown_connections 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559fb3459970 0x559fb345bd90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fb33df3a0 0x559fb3451a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fb33d4b00 0x559fb33e2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 --2- 172.21.15.71:0/551291064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fb33d41e0 0x559fb33e2910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b6d600640 0 client.26378 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 >> 172.21.15.71:0/551291064 conn(0x559fb33d0750 msgr2=0x559fb33dc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 shutdown_connections 2024-09-05T12:36:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.704+0000 7f6b8d4a7740 1 -- 172.21.15.71:0/551291064 wait complete. 2024-09-05T12:36:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8d4a7740 20 client.26378 trim_cache size 0 max 16384 2024-09-05T12:36:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:36:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:36:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:36:24.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:36:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b680a2da0 msgr2=0x7f6b680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b680a2da0 0x7f6b680a5260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x559fb33d1ad0 tx=0x7f6b70005f90 comp rx=0 tx=0).stop 2024-09-05T12:36:24.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 msgr2=0x7f6b8419f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 0x7f6b8419f370 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00eb10 tx=0x7f6b7c00c2d0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 shutdown_connections 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b680a2da0 0x7f6b680a5260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b8419eec0 0x7f6b8419f370 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b841090a0 0x7f6b8419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 --2- 172.21.15.71:0/1079542268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84103f50 0x7f6b8419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.705+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 >> 172.21.15.71:0/1079542268 conn(0x7f6b840ff9e0 msgr2=0x7f6b84100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.706+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 shutdown_connections 2024-09-05T12:36:24.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:24.706+0000 7f6b8a600640 1 -- 172.21.15.71:0/1079542268 wait complete. 2024-09-05T12:36:24.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.714+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f895c19b910 con 0x7f895c101de0 2024-09-05T12:36:24.717 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.715+0000 7f8953400640 1 -- 172.21.15.71:0/581769071 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11976 (secure 0 0 0) 0x7f894c09bd50 con 0x7f895c101de0 2024-09-05T12:36:24.717 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:36:24.718 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":41,"btime":"2024-09-05T12:36:02:247256+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:36:01.371812+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:36:24.718 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 41 2024-09-05T12:36:24.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f89400a2cf0 msgr2=0x7f89400a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f89400a2cf0 0x7f89400a51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f89480096c0 tx=0x7f8948009260 comp rx=0 tx=0).stop 2024-09-05T12:36:24.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 msgr2=0x7f895c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:24.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 0x7f895c19a5b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f894c00e9f0 tx=0x7f894c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 shutdown_connections 2024-09-05T12:36:24.720 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c106f10 0x7f895c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f895c102790 0x7f895c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f89400a2cf0 0x7f89400a51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 --2- 172.21.15.71:0/581769071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f895c101de0 0x7f895c19a5b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:24.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 >> 172.21.15.71:0/581769071 conn(0x7f895c0fd8c0 msgr2=0x7f895c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:24.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 shutdown_connections 2024-09-05T12:36:24.721 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:24.718+0000 7f8961c00640 1 -- 172.21.15.71:0/581769071 wait complete. 2024-09-05T12:36:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:24 smithi174 ceph-mon[34313]: pgmap v268: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 632 KiB/s wr, 42 op/s 2024-09-05T12:36:25.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.42 MB/sec execute 321 sec latency 24297.796 ms 2024-09-05T12:36:25.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:25 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/581769071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:25.651 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 917b42cc --release --await' 2024-09-05T12:36:25.651 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 917b42cc --release --await 2024-09-05T12:36:25.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:25 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/581769071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:25.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:25 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/581769071' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:26.021 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:26.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 299668 3.41 MB/sec execute 322 sec latency 25297.886 ms 2024-09-05T12:36:26.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:26 smithi071 ceph-mon[29845]: pgmap v269: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 621 KiB/s wr, 40 op/s 2024-09-05T12:36:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:26 smithi123 ceph-mon[34321]: pgmap v269: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 621 KiB/s wr, 40 op/s 2024-09-05T12:36:26.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 -- 172.21.15.71:0/2993684692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0076720 msgr2=0x7f4da0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 --2- 172.21.15.71:0/2993684692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0076720 0x7f4da0076b00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4d94009900 tx=0x7f4d9402a410 comp rx=0 tx=0).stop 2024-09-05T12:36:26.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 -- 172.21.15.71:0/2993684692 shutdown_connections 2024-09-05T12:36:26.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 --2- 172.21.15.71:0/2993684692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4da0112870 0x7f4da0114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 --2- 172.21.15.71:0/2993684692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0077040 0x7f4da0112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 --2- 172.21.15.71:0/2993684692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0076720 0x7f4da0076b00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 -- 172.21.15.71:0/2993684692 >> 172.21.15.71:0/2993684692 conn(0x7f4da00ff8d0 msgr2=0x7f4da0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:26.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 -- 172.21.15.71:0/2993684692 shutdown_connections 2024-09-05T12:36:26.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.689+0000 7f4da7200640 1 -- 172.21.15.71:0/2993684692 wait complete. 2024-09-05T12:36:26.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.690+0000 7f4da7200640 1 Processor -- start 2024-09-05T12:36:26.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.690+0000 7f4da7200640 1 -- start start 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da7200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 0x7f4da01a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4da0077040 0x7f4da01a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0112870 0x7f4da019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4da0118430 con 0x7f4da0112870 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01182b0 con 0x7f4da0077040 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da7200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01185b0 con 0x7f4da0076720 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 0x7f4da01a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 0x7f4da01a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52992/0 (socket says 172.21.15.71:52992) 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 -- 172.21.15.71:0/2251490867 learned_addr learned my addr 172.21.15.71:0/2251490867 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da6800640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0112870 0x7f4da019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5400640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4da0077040 0x7f4da01a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 -- 172.21.15.71:0/2251490867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4da0077040 msgr2=0x7f4da01a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4da0077040 0x7f4da01a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 -- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0112870 msgr2=0x7f4da019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0112870 0x7f4da019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.691+0000 7f4da5e00640 1 -- 172.21.15.71:0/2251490867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da019efd0 con 0x7f4da0076720 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.692+0000 7f4da5e00640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 0x7f4da01a4690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d94004770 tx=0x7f4d94003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.692+0000 7f4d9aa00640 1 -- 172.21.15.71:0/2251490867 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9403f070 con 0x7f4da0076720 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.692+0000 7f4d9aa00640 1 -- 172.21.15.71:0/2251490867 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d94031ca0 con 0x7f4da0076720 2024-09-05T12:36:26.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.692+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da019f260 con 0x7f4da0076720 2024-09-05T12:36:26.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.692+0000 7f4d9aa00640 1 -- 172.21.15.71:0/2251490867 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94035030 con 0x7f4da0076720 2024-09-05T12:36:26.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.692+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da01ab440 con 0x7f4da0076720 2024-09-05T12:36:26.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.695+0000 7f4d9aa00640 1 -- 172.21.15.71:0/2251490867 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4d94042070 con 0x7f4da0076720 2024-09-05T12:36:26.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.695+0000 7f4dab212740 1 Processor -- start 2024-09-05T12:36:26.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.695+0000 7f4dab212740 1 -- start start 2024-09-05T12:36:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4dab212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcc2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4dab212740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2e40 0x55b10dccd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4dab212740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dccd700 0x55b10dccfaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4dab212740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b10dc28650 con 0x55b10dccd700 2024-09-05T12:36:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4dab212740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b10dbcff10 con 0x55b10dcc2e40 2024-09-05T12:36:26.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4dab212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b10dc24610 con 0x55b10dcc2520 2024-09-05T12:36:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4da6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcc2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4da6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcc2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53006/0 (socket says 172.21.15.71:53006) 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dccd700 0x55b10dccfaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4da6800640 1 -- 172.21.15.71:0/3620229572 learned_addr learned my addr 172.21.15.71:0/3620229572 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.696+0000 7f4da5e00640 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2e40 0x55b10dccd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da6800640 1 -- 172.21.15.71:0/3620229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2e40 msgr2=0x55b10dccd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da6800640 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2e40 0x55b10dccd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da6800640 1 -- 172.21.15.71:0/3620229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dccd700 msgr2=0x55b10dccfaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da6800640 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dccd700 0x55b10dccfaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da6800640 1 -- 172.21.15.71:0/3620229572 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10d937580 con 0x55b10dcc2520 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da5e00640 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2e40 0x55b10dccd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da6800640 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcc2900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009f10 tx=0x7f4d9c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a89802cc857d9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4d9aa00640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4d880a2cf0 0x7f4d880a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4d9aa00640 1 -- 172.21.15.71:0/2251490867 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4d940effb0 con 0x7f4da0076720 2024-09-05T12:36:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4d8f000640 1 -- 172.21.15.71:0/3620229572 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c016070 con 0x55b10dcc2520 2024-09-05T12:36:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4da5e00640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4d880a2cf0 0x7f4d880a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4d8f000640 1 -- 172.21.15.71:0/3620229572 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9c012930 con 0x55b10dcc2520 2024-09-05T12:36:26.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.697+0000 7f4d8f000640 1 -- 172.21.15.71:0/3620229572 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c011980 con 0x55b10dcc2520 2024-09-05T12:36:26.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 -- 172.21.15.71:0/3620229572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 msgr2=0x55b10dcc2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4da5e00640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4d880a2cf0 0x7f4d880a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000c00 tx=0x7f4d940f3a60 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcc2900 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c009f10 tx=0x7f4d9c010740 comp rx=0 tx=0).stop 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 -- 172.21.15.71:0/3620229572 shutdown_connections 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dccd700 0x55b10dccfaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2e40 0x55b10dccd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 --2- 172.21.15.71:0/3620229572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcc2900 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 -- 172.21.15.71:0/3620229572 >> 172.21.15.71:0/3620229572 conn(0x55b10dcbeab0 msgr2=0x55b10dcbee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 -- 172.21.15.71:0/3620229572 shutdown_connections 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.698+0000 7f4dab212740 1 -- 172.21.15.71:0/3620229572 wait complete. 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 -- start start 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 Processor -- start 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dccac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dccb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b10dcd02d0 con 0x55b10dcc2e40 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b10dbe3350 con 0x55b10dccd700 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.699+0000 7f4dab212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b10dcd0450 con 0x55b10dcc2520 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4dab212740 10 client.0 _ll_register_callbacks cb 0x55b10dc06290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dccac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dccac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38880/0 (socket says 172.21.15.71:38880) 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 -- 172.21.15.71:0/1310269693 learned_addr learned my addr 172.21.15.71:0/1310269693 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da6800640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dccb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da6800640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 -- 172.21.15.71:0/1310269693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 msgr2=0x55b10dcca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 -- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 msgr2=0x55b10dccb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dccb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 -- 172.21.15.71:0/1310269693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10da01960 con 0x55b10dcc2e40 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da6800640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4da5400640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dccac40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000bf00 tx=0x7f4d90011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6049e722a6a5227c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.700+0000 7f4d83400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4d83400640 1 -- 172.21.15.71:0/1310269693 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d90004020 con 0x55b10dcc2e40 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4d83400640 1 -- 172.21.15.71:0/1310269693 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4d900041c0 con 0x55b10dcc2e40 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d9c012930 con 0x55b10dcc2e40 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4d83400640 1 -- 172.21.15.71:0/1310269693 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4d90024050 con 0x55b10dcc2e40 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4dab212740 10 client.16362 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b10dcdc000 con 0x55b10dcc2e40 2024-09-05T12:36:26.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.701+0000 7f4d83400640 1 -- 172.21.15.71:0/1310269693 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d9c012930 con 0x55b10dcc2e40 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4d83400640 1 -- 172.21.15.71:0/1310269693 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f4d900038f0 con 0x55b10dcc2e40 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4dab212740 10 client.16362 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4dab212740 10 client.16362 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4dab212740 20 client.16362 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4dab212740 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dcdd580 0x55b10dcdf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4dab212740 4 client.16362 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:26.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b10dc35bc0 con 0x55b10dcdd580 2024-09-05T12:36:26.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.702+0000 7f4da6800640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dcdd580 0x55b10dcdf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.703+0000 7f4da6800640 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dcdd580 0x55b10dcdf9a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0863c0317869a4f server_cookie=f111b7673ffcc3d5 in_seq=0 out_seq=0 2024-09-05T12:36:26.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.703+0000 7f4d83400640 10 client.16362 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:26.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.703+0000 7f4d83400640 1 -- 172.21.15.71:0/1310269693 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b10dc35bc0 con 0x55b10dcdd580 2024-09-05T12:36:26.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.703+0000 7f4d83400640 10 client.16362 handle_command_reply: tid=0 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.703+0000 7f4dab212740 1 client.16362 shutdown 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 msgr2=0x55b10dccac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dccac40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000bf00 tx=0x7f4d90011810 comp rx=0 tx=0).stop 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 shutdown_connections 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dcdd580 0x55b10dcdf9a0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dccb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dccac40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 --2- 172.21.15.71:0/1310269693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dcca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4d83400640 0 client.16362 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 >> 172.21.15.71:0/1310269693 conn(0x55b10dcbeab0 msgr2=0x55b10dcbdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 shutdown_connections 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 1 -- 172.21.15.71:0/1310269693 wait complete. 2024-09-05T12:36:26.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.704+0000 7f4dab212740 20 client.16362 trim_cache size 0 max 16384 2024-09-05T12:36:26.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 Processor -- start 2024-09-05T12:36:26.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 -- start start 2024-09-05T12:36:26.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dce5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dce5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dcc7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01b53c0 con 0x55b10dcc2e40 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01b6fc0 con 0x55b10dccd700 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.722+0000 7f4dab212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01b5a90 con 0x55b10dcc2520 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dce5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dce5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38894/0 (socket says 172.21.15.71:38894) 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 -- 172.21.15.71:0/4215892151 learned_addr learned my addr 172.21.15.71:0/4215892151 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da6800640 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dce5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da6800640 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dcc7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 -- 172.21.15.71:0/4215892151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 msgr2=0x55b10dce5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dce5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 -- 172.21.15.71:0/4215892151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 msgr2=0x55b10dcc7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dcc7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 -- 172.21.15.71:0/4215892151 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b10dcbfb60 con 0x55b10dcc2e40 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da6800640 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dce5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.723+0000 7f4da5400640 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dce5790 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4d90003f00 tx=0x7f4d90011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca5fb6d77a684d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4d8e600640 1 -- 172.21.15.71:0/4215892151 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9001a3f0 con 0x55b10dcc2e40 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4d8e600640 1 -- 172.21.15.71:0/4215892151 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d90004460 con 0x55b10dcc2e40 2024-09-05T12:36:26.727 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4d8e600640 1 -- 172.21.15.71:0/4215892151 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d900052d0 con 0x55b10dcc2e40 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 -- 172.21.15.71:0/4215892151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 msgr2=0x55b10dce5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dce5790 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4d90003f00 tx=0x7f4d90011d10 comp rx=0 tx=0).stop 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 -- 172.21.15.71:0/4215892151 shutdown_connections 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dccd700 0x55b10dcc7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dce5790 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 --2- 172.21.15.71:0/4215892151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dcc2520 0x55b10dce5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.724+0000 7f4dab212740 1 -- 172.21.15.71:0/4215892151 >> 172.21.15.71:0/4215892151 conn(0x55b10dcbeab0 msgr2=0x55b10dcc4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:26.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 -- 172.21.15.71:0/4215892151 shutdown_connections 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 -- 172.21.15.71:0/4215892151 wait complete. 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 -- start start 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 Processor -- start 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 0x55b10dcd1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dcd21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dccd700 0x55b10dd411f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01b6830 con 0x55b10dcc2e40 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01b6160 con 0x55b10dcc2520 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.725+0000 7f4dab212740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4da01b33c0 con 0x55b10dccd700 2024-09-05T12:36:26.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4dab212740 10 client.0 _ll_register_callbacks cb 0x7f4d9000bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dcd21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 0x55b10dcd1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dcd21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38900/0 (socket says 172.21.15.71:38900) 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 0x55b10dcd1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34852/0 (socket says 172.21.15.71:34852) 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da5400640 1 -- 172.21.15.71:0/1684074148 learned_addr learned my addr 172.21.15.71:0/1684074148 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dccd700 0x55b10dd411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 -- 172.21.15.71:0/1684074148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dccd700 msgr2=0x55b10dd411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dccd700 0x55b10dd411f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 -- 172.21.15.71:0/1684074148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 msgr2=0x55b10dcd21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:26.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dcd21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 -- 172.21.15.71:0/1684074148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b10dd41830 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4da6800640 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 0x55b10dcd1c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c010c40 tx=0x7f4d9000ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc610fa01f107451 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.726+0000 7f4d83400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4d83400640 1 -- 172.21.15.71:0/1684074148 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c030070 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4d83400640 1 -- 172.21.15.71:0/1684074148 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4d9c02be80 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d90004460 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4d83400640 1 -- 172.21.15.71:0/1684074148 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4d9c03c050 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4dab212740 10 client.26396 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b10dd48c00 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4d83400640 1 -- 172.21.15.71:0/1684074148 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d90004460 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4d83400640 1 -- 172.21.15.71:0/1684074148 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f4d9c00f040 con 0x55b10dcc2520 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4dab212740 10 client.26396 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:26.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.727+0000 7f4dab212740 10 client.26396 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4dab212740 20 client.26396 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4dab212740 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dd49140 0x55b10dd4b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4dab212740 4 client.26396 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "917b42cc", "release": true, "await": true}] 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "917b42cc", "release": true, "await": true}) -- 0x55b10dcbe380 con 0x55b10dd49140 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4da5400640 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dd49140 0x55b10dd4b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4da5400640 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dd49140 0x55b10dd4b530 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c280e0ea2f2791 server_cookie=e56a1c2a1d195f99 in_seq=0 out_seq=0 2024-09-05T12:36:26.735 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.728+0000 7f4d83400640 10 client.26396 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4d83400640 1 -- 172.21.15.71:0/1684074148 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b10dcbe380 con 0x55b10dd49140 2024-09-05T12:36:26.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4d83400640 10 client.26396 handle_command_reply: tid=0 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 client.26396 shutdown 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 msgr2=0x55b10dcd1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 0x55b10dcd1c90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c010c40 tx=0x7f4d9000ba80 comp rx=0 tx=0).stop 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 shutdown_connections 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b10dd49140 0x55b10dd4b530 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b10dccd700 0x55b10dd411f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b10dcc2e40 0x55b10dcd21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 --2- 172.21.15.71:0/1684074148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b10dcc2520 0x55b10dcd1c90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4d83400640 0 client.26396 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:26.737 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 >> 172.21.15.71:0/1684074148 conn(0x55b10dcbeab0 msgr2=0x55b10dcc8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 shutdown_connections 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 1 -- 172.21.15.71:0/1684074148 wait complete. 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.734+0000 7f4dab212740 20 client.26396 trim_cache size 0 max 16384 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 41, 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 28, 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:36:26.738 INFO:teuthology.orchestra.run.smithi071.stdout: "917b42cc": { 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 28, 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 146.2, 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:36:26.739 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:26.740 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4d880a2cf0 msgr2=0x7f4d880a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4d880a2cf0 0x7f4d880a51b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d94000c00 tx=0x7f4d940f3a60 comp rx=0 tx=0).stop 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 msgr2=0x7f4da01a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 0x7f4da01a4690 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4d94004770 tx=0x7f4d94003c10 comp rx=0 tx=0).stop 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 shutdown_connections 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4d880a2cf0 0x7f4d880a51b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4da0112870 0x7f4da019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.741 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4da0077040 0x7f4da01a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 --2- 172.21.15.71:0/2251490867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4da0076720 0x7f4da01a4690 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 >> 172.21.15.71:0/2251490867 conn(0x7f4da00ff8d0 msgr2=0x7f4da0101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 shutdown_connections 2024-09-05T12:36:26.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:26.735+0000 7f4da7200640 1 -- 172.21.15.71:0/2251490867 wait complete. 2024-09-05T12:36:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:26 smithi174 ceph-mon[34313]: pgmap v269: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 621 KiB/s wr, 40 op/s 2024-09-05T12:36:27.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 300743 3.41 MB/sec execute 323 sec latency 25789.592 ms 2024-09-05T12:36:27.554 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '917b42cc', total seconds elapsed: 30.3 2024-09-05T12:36:27.554 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.15 2024-09-05T12:36:27.555 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:36:27.993 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:28.250 INFO:tasks.workunit.client.0.smithi071.stdout: 1 302066 3.43 MB/sec execute 324 sec latency 17.947 ms 2024-09-05T12:36:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:28 smithi071 ceph-mon[29845]: pgmap v270: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 901 KiB/s wr, 64 op/s 2024-09-05T12:36:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:28 smithi123 ceph-mon[34321]: pgmap v270: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 901 KiB/s wr, 64 op/s 2024-09-05T12:36:28.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 -- 172.21.15.71:0/40493019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230103e30 msgr2=0x7f9230104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:28.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 --2- 172.21.15.71:0/40493019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230103e30 0x7f9230104210 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9224009900 tx=0x7f922402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 -- 172.21.15.71:0/40493019 shutdown_connections 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 --2- 172.21.15.71:0/40493019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9230108f60 0x7f9230110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 --2- 172.21.15.71:0/40493019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92301047e0 0x7f9230108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 --2- 172.21.15.71:0/40493019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230103e30 0x7f9230104210 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 -- 172.21.15.71:0/40493019 >> 172.21.15.71:0/40493019 conn(0x7f92300ff8d0 msgr2=0x7f9230101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 -- 172.21.15.71:0/40493019 shutdown_connections 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.674+0000 7f9236000640 1 -- 172.21.15.71:0/40493019 wait complete. 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.675+0000 7f9236000640 1 Processor -- start 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.675+0000 7f9236000640 1 -- start start 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9236000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 0x7f923019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9236000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92301047e0 0x7f923019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9236000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230108f60 0x7f923019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9236000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9230114010 con 0x7f9230103e30 2024-09-05T12:36:28.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9236000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9230113e90 con 0x7f9230108f60 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9236000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9230114190 con 0x7f92301047e0 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9234c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 0x7f923019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f922fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92301047e0 0x7f923019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9234c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 0x7f923019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38916/0 (socket says 172.21.15.71:38916) 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9234c00640 1 -- 172.21.15.71:0/1911513833 learned_addr learned my addr 172.21.15.71:0/1911513833 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.676+0000 7f9235600640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230108f60 0x7f923019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9234c00640 1 -- 172.21.15.71:0/1911513833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92301047e0 msgr2=0x7f923019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9234c00640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92301047e0 0x7f923019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9234c00640 1 -- 172.21.15.71:0/1911513833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230108f60 msgr2=0x7f923019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9234c00640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230108f60 0x7f923019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9234c00640 1 -- 172.21.15.71:0/1911513833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923019f440 con 0x7f9230103e30 2024-09-05T12:36:28.703 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9234c00640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 0x7f923019a490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9224004770 tx=0x7f9224003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922403f070 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9224031ca0 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224035030 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.677+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f923019f6d0 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.678+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f923019fb60 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.679+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9230104210 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.681+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9224042070 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.683+0000 7f922d600640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92180a2cf0 0x7f92180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.683+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f92240efe00 con 0x7f9230103e30 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.683+0000 7f922fe00640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92180a2cf0 0x7f92180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.683+0000 7f922fe00640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92180a2cf0 0x7f92180a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f921c0095f0 tx=0x7f921c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:28.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.687+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92240bc0a0 con 0x7f9230103e30 2024-09-05T12:36:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:28 smithi174 ceph-mon[34313]: pgmap v270: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 901 KiB/s wr, 64 op/s 2024-09-05T12:36:28.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.952+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f923002d7c0 con 0x7f9230103e30 2024-09-05T12:36:28.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.953+0000 7f922d600640 1 -- 172.21.15.71:0/1911513833 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11976 (secure 0 0 0) 0x7f92240c03a0 con 0x7f9230103e30 2024-09-05T12:36:28.964 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:36:28.964 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":41,"btime":"2024-09-05T12:36:02:247256+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:36:01.371812+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:36:28.965 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 41 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92180a2cf0 msgr2=0x7f92180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92180a2cf0 0x7f92180a51b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f921c0095f0 tx=0x7f921c005f90 comp rx=0 tx=0).stop 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 msgr2=0x7f923019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 0x7f923019a490 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f9224004770 tx=0x7f9224003c10 comp rx=0 tx=0).stop 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 shutdown_connections 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9230108f60 0x7f923019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92301047e0 0x7f923019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92180a2cf0 0x7f92180a51b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 --2- 172.21.15.71:0/1911513833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9230103e30 0x7f923019a490 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:28.968 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 >> 172.21.15.71:0/1911513833 conn(0x7f92300ff8d0 msgr2=0x7f9230100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:28.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.958+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 shutdown_connections 2024-09-05T12:36:28.969 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:28.959+0000 7f9236000640 1 -- 172.21.15.71:0/1911513833 wait complete. 2024-09-05T12:36:29.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 302677 3.42 MB/sec execute 325 sec latency 509.673 ms 2024-09-05T12:36:29.615 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:29 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1911513833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:29.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:29 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1911513833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:29.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:29 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1911513833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:30.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0' 2024-09-05T12:36:30.081 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 146.15 --await-for 45.0 2024-09-05T12:36:30.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303214 3.42 MB/sec execute 326 sec latency 632.043 ms 2024-09-05T12:36:30.452 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:30.591 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:30 smithi071 ceph-mon[29845]: pgmap v271: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 621 KiB/s wr, 41 op/s 2024-09-05T12:36:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:30 smithi123 ceph-mon[34321]: pgmap v271: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 621 KiB/s wr, 41 op/s 2024-09-05T12:36:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:30 smithi174 ceph-mon[34313]: pgmap v271: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 621 KiB/s wr, 41 op/s 2024-09-05T12:36:31.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 -- 172.21.15.71:0/691342094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c10c170 msgr2=0x7f1c6c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 --2- 172.21.15.71:0/691342094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c10c170 0x7f1c6c10e610 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009ca0 tx=0x7f1c6802a950 comp rx=0 tx=0).stop 2024-09-05T12:36:31.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 -- 172.21.15.71:0/691342094 shutdown_connections 2024-09-05T12:36:31.126 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 --2- 172.21.15.71:0/691342094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c10c170 0x7f1c6c10e610 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 --2- 172.21.15.71:0/691342094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c103660 0x7f1c6c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 --2- 172.21.15.71:0/691342094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c102d40 0x7f1c6c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 -- 172.21.15.71:0/691342094 >> 172.21.15.71:0/691342094 conn(0x7f1c6c0fc7a0 msgr2=0x7f1c6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:31.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 -- 172.21.15.71:0/691342094 shutdown_connections 2024-09-05T12:36:31.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.125+0000 7f1c74800640 1 -- 172.21.15.71:0/691342094 wait complete. 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.126+0000 7f1c74800640 1 Processor -- start 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.126+0000 7f1c74800640 1 -- start start 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c74800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c102d40 0x7f1c6c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c74800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 0x7f1c6c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c74800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c10c170 0x7f1c6c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c74800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c111f70 con 0x7f1c6c103660 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c74800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c111df0 con 0x7f1c6c102d40 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c74800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1120f0 con 0x7f1c6c10c170 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 0x7f1c6c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.129 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 0x7f1c6c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38936/0 (socket says 172.21.15.71:38936) 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 -- 172.21.15.71:0/1993409492 learned_addr learned my addr 172.21.15.71:0/1993409492 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71e00640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c102d40 0x7f1c6c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c72800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c10c170 0x7f1c6c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 -- 172.21.15.71:0/1993409492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c10c170 msgr2=0x7f1c6c19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c10c170 0x7f1c6c19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 -- 172.21.15.71:0/1993409492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c102d40 msgr2=0x7f1c6c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c102d40 0x7f1c6c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c71400640 1 -- 172.21.15.71:0/1993409492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19f5b0 con 0x7f1c6c103660 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.127+0000 7f1c72800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c10c170 0x7f1c6c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.128+0000 7f1c71400640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 0x7f1c6c19ab40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9f0 tx=0x7f1c5c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.128+0000 7f1c66a00640 1 -- 172.21.15.71:0/1993409492 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00ccc0 con 0x7f1c6c103660 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.128+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19f8a0 con 0x7f1c6c103660 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.128+0000 7f1c66a00640 1 -- 172.21.15.71:0/1993409492 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c5c010910 con 0x7f1c6c103660 2024-09-05T12:36:31.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.128+0000 7f1c66a00640 1 -- 172.21.15.71:0/1993409492 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c018920 con 0x7f1c6c103660 2024-09-05T12:36:31.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.128+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a7180 con 0x7f1c6c103660 2024-09-05T12:36:31.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.131+0000 7f1c66a00640 1 -- 172.21.15.71:0/1993409492 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1c5c010430 con 0x7f1c6c103660 2024-09-05T12:36:31.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.131+0000 7f1c77295740 1 Processor -- start 2024-09-05T12:36:31.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.131+0000 7f1c77295740 1 -- start start 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.131+0000 7f1c77295740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 0x55e8088c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c77295740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c8e00 0x55e8088d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c77295740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c77295740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e80882d260 con 0x55e8088c84e0 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c77295740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e8087d4500 con 0x55e8088c8e00 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c77295740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e808829220 con 0x55e8088d36c0 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 0x55e8088c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 0x55e8088c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38946/0 (socket says 172.21.15.71:38946) 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 -- 172.21.15.71:0/2929777036 learned_addr learned my addr 172.21.15.71:0/2929777036 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c71e00640 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c8e00 0x55e8088d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c71400640 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 -- 172.21.15.71:0/2929777036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 msgr2=0x55e8088d5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 -- 172.21.15.71:0/2929777036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c8e00 msgr2=0x55e8088d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c8e00 0x55e8088d3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c72800640 1 -- 172.21.15.71:0/2929777036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80859eba0 con 0x55e8088c84e0 2024-09-05T12:36:31.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.132+0000 7f1c71e00640 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c8e00 0x55e8088d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c66a00640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1c540a2d70 0x7f1c540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c72800640 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 0x55e8088c88c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009830 tx=0x7f1c68009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b759564fb7238ca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c66a00640 1 -- 172.21.15.71:0/1993409492 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f1c5c0cfa10 con 0x7f1c6c103660 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c71e00640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1c540a2d70 0x7f1c540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c5b000640 1 -- 172.21.15.71:0/2929777036 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68034040 con 0x55e8088c84e0 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c5b000640 1 -- 172.21.15.71:0/2929777036 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6802fbc0 con 0x55e8088c84e0 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.133+0000 7f1c5b000640 1 -- 172.21.15.71:0/2929777036 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68004570 con 0x55e8088c84e0 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 -- 172.21.15.71:0/2929777036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 msgr2=0x55e8088c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c71e00640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1c540a2d70 0x7f1c540a5230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009f70 tx=0x7f1c60005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 0x55e8088c88c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009830 tx=0x7f1c68009860 comp rx=0 tx=0).stop 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 -- 172.21.15.71:0/2929777036 shutdown_connections 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c8e00 0x55e8088d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 --2- 172.21.15.71:0/2929777036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c84e0 0x55e8088c88c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 -- 172.21.15.71:0/2929777036 >> 172.21.15.71:0/2929777036 conn(0x55e8088c4a70 msgr2=0x55e8088c4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:31.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 -- 172.21.15.71:0/2929777036 shutdown_connections 2024-09-05T12:36:31.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.134+0000 7f1c77295740 1 -- 172.21.15.71:0/2929777036 wait complete. 2024-09-05T12:36:31.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 -- start start 2024-09-05T12:36:31.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 Processor -- start 2024-09-05T12:36:31.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e8088d6230 con 0x55e8088c8e00 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e8087e7bd0 con 0x55e8088c84e0 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.135+0000 7f1c77295740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e8088d63b0 con 0x55e8088d36c0 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c77295740 10 client.0 _ll_register_callbacks cb 0x55e80853c0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38954/0 (socket says 172.21.15.71:38954) 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 -- 172.21.15.71:0/2055468722 learned_addr learned my addr 172.21.15.71:0/2055468722 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71e00640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c72800640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 -- 172.21.15.71:0/2055468722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 msgr2=0x55e8088d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 -- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 msgr2=0x55e8088d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 -- 172.21.15.71:0/2055468722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8088df750 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c71400640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d0ba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9c0 tx=0x7f1c5c09b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a41a0a42c60436 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.136+0000 7f1c4f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c4f400640 1 -- 172.21.15.71:0/2055468722 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c09c240 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c4f400640 1 -- 172.21.15.71:0/2055468722 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f1c5c0ccff0 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6802fbc0 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c4f400640 1 -- 172.21.15.71:0/2055468722 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1c5c0cd1f0 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c77295740 10 client.16386 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e8088e20c0 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c4f400640 1 -- 172.21.15.71:0/2055468722 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6802fbc0 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c4f400640 1 -- 172.21.15.71:0/2055468722 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f1c5c09af90 con 0x55e8088c8e00 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c77295740 10 client.16386 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c77295740 10 client.16386 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:31.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.137+0000 7f1c77295740 20 client.16386 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:31.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.138+0000 7f1c77295740 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e8088e3670 0x55e8088e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.138+0000 7f1c77295740 4 client.16386 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:31.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.138+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80883a990 con 0x55e8088e3670 2024-09-05T12:36:31.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.138+0000 7f1c71e00640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e8088e3670 0x55e8088e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.138+0000 7f1c71e00640 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e8088e3670 0x55e8088e5a80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61c58382558f02d server_cookie=e52f16e158557d42 in_seq=0 out_seq=0 2024-09-05T12:36:31.142 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.138+0000 7f1c4f400640 10 client.16386 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.139+0000 7f1c4f400640 1 -- 172.21.15.71:0/2055468722 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e80883a990 con 0x55e8088e3670 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.139+0000 7f1c4f400640 10 client.16386 handle_command_reply: tid=0 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.139+0000 7f1c77295740 1 client.16386 shutdown 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 msgr2=0x55e8088d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d0ba0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9c0 tx=0x7f1c5c09b910 comp rx=0 tx=0).stop 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 shutdown_connections 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e8088e3670 0x55e8088e5a80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.143 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e8088d10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d0ba0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 --2- 172.21.15.71:0/2055468722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c4f400640 0 client.16386 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 >> 172.21.15.71:0/2055468722 conn(0x55e8088c4a70 msgr2=0x55e8088ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 shutdown_connections 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 1 -- 172.21.15.71:0/2055468722 wait complete. 2024-09-05T12:36:31.144 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.140+0000 7f1c77295740 20 client.16386 trim_cache size 0 max 16384 2024-09-05T12:36:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.157+0000 7f1c77295740 1 Processor -- start 2024-09-05T12:36:31.158 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 -- start start 2024-09-05T12:36:31.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088c8e00 0x55e8088eb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 0x55e8088eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1b12d0 con 0x55e8088d36c0 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1b2ed0 con 0x55e8088c84e0 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c77295740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1b19a0 con 0x55e8088c8e00 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c72800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 0x55e8088eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c71400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088c8e00 0x55e8088eb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c72800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 0x55e8088eb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38956/0 (socket says 172.21.15.71:38956) 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c72800640 1 -- 172.21.15.71:0/843037576 learned_addr learned my addr 172.21.15.71:0/843037576 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.158+0000 7f1c71e00640 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c72800640 1 -- 172.21.15.71:0/843037576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088c8e00 msgr2=0x55e8088eb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.160 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c72800640 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088c8e00 0x55e8088eb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c72800640 1 -- 172.21.15.71:0/843037576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 msgr2=0x55e8088cf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c72800640 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088cf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c72800640 1 -- 172.21.15.71:0/843037576 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8088befe0 con 0x55e8088d36c0 2024-09-05T12:36:31.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c71400640 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088c8e00 0x55e8088eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:31.161 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c72800640 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 0x55e8088eb890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1c68006010 tx=0x7f1c6802afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a221beef22783195 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:31.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c5a600640 1 -- 172.21.15.71:0/843037576 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6804c070 con 0x55e8088d36c0 2024-09-05T12:36:31.163 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c5a600640 1 -- 172.21.15.71:0/843037576 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6802fbc0 con 0x55e8088d36c0 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.159+0000 7f1c5a600640 1 -- 172.21.15.71:0/843037576 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68034040 con 0x55e8088d36c0 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 -- 172.21.15.71:0/843037576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 msgr2=0x55e8088eb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 0x55e8088eb890 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1c68006010 tx=0x7f1c6802afd0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 -- 172.21.15.71:0/843037576 shutdown_connections 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088d36c0 0x55e8088eb890 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088c8e00 0x55e8088eb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 --2- 172.21.15.71:0/843037576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088cf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 -- 172.21.15.71:0/843037576 >> 172.21.15.71:0/843037576 conn(0x55e8088c4a70 msgr2=0x55e8088d25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 -- 172.21.15.71:0/843037576 shutdown_connections 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 -- 172.21.15.71:0/843037576 wait complete. 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.160+0000 7f1c77295740 1 -- start start 2024-09-05T12:36:31.164 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 Processor -- start 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e80894d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1b2740 con 0x55e8088c8e00 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1b2070 con 0x55e8088c84e0 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c1af2d0 con 0x55e8088d36c0 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c77295740 10 client.0 _ll_register_callbacks cb 0x7f1c5c0d31b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c71400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c71e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c71400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38958/0 (socket says 172.21.15.71:38958) 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.161+0000 7f1c71400640 1 -- 172.21.15.71:0/3655807185 learned_addr learned my addr 172.21.15.71:0/3655807185 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c72800640 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e80894d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.165 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c71400640 1 -- 172.21.15.71:0/3655807185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 msgr2=0x55e80894d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c71400640 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e80894d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c71400640 1 -- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 msgr2=0x55e8088d77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c71400640 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c71400640 1 -- 172.21.15.71:0/3655807185 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80894d860 con 0x55e8088c8e00 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c71400640 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d7d10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c098510 tx=0x7f1c5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9286cb745515cf05 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c4f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c4f400640 1 -- 172.21.15.71:0/3655807185 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c0cd200 con 0x55e8088c8e00 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c4f400640 1 -- 172.21.15.71:0/3655807185 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f1c5c097a90 con 0x55e8088c8e00 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.162+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c6802fbc0 con 0x55e8088c8e00 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c4f400640 1 -- 172.21.15.71:0/3655807185 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1c5c09fef0 con 0x55e8088c8e00 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 10 client.16392 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e80894ea00 con 0x55e8088c8e00 2024-09-05T12:36:31.166 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c4f400640 1 -- 172.21.15.71:0/3655807185 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6802fbc0 con 0x55e8088c8e00 2024-09-05T12:36:31.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c4f400640 1 -- 172.21.15.71:0/3655807185 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f1c5c0a10f0 con 0x55e8088c8e00 2024-09-05T12:36:31.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 10 client.16392 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:31.167 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 10 client.16392 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 20 client.16392 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e80894f2b0 0x55e8089516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 4 client.16392 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}] 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.163+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.15, "await_for": 45.0}) -- 0x55e808834ce0 con 0x55e80894f2b0 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.164+0000 7f1c71e00640 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e80894f2b0 0x55e8089516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.164+0000 7f1c71e00640 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e80894f2b0 0x55e8089516d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3703602ea167ee server_cookie=643ef75c32c943aa in_seq=0 out_seq=0 2024-09-05T12:36:31.168 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.164+0000 7f1c4f400640 10 client.16392 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:31.217 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c4f400640 1 -- 172.21.15.71:0/3655807185 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e808834ce0 con 0x55e80894f2b0 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c4f400640 10 client.16392 handle_command_reply: tid=0 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 client.16392 shutdown 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 msgr2=0x55e8088d7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d7d10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c098510 tx=0x7f1c5c009f90 comp rx=0 tx=0).stop 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 shutdown_connections 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e80894f2b0 0x55e8089516d0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e8088d36c0 0x55e80894d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e8088c8e00 0x55e8088d7d10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 --2- 172.21.15.71:0/3655807185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e8088c84e0 0x55e8088d77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 >> 172.21.15.71:0/3655807185 conn(0x55e8088c4a70 msgr2=0x55e8088d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:31.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c4f400640 0 client.16392 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.216+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 shutdown_connections 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.217+0000 7f1c77295740 1 -- 172.21.15.71:0/3655807185 wait complete. 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.217+0000 7f1c77295740 20 client.16392 trim_cache size 0 max 16384 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 41, 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 30, 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:36:31.219 INFO:teuthology.orchestra.run.smithi071.stdout: "2f1ff020": { 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 30, 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 146.2, 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:31.220 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:31.221 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.217+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1c540a2d70 msgr2=0x7f1c540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.217+0000 7f1c74800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1c540a2d70 0x7f1c540a5230 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009f70 tx=0x7f1c60005f90 comp rx=0 tx=0).stop 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.217+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 msgr2=0x7f1c6c19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.217+0000 7f1c74800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 0x7f1c6c19ab40 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e9f0 tx=0x7f1c5c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 shutdown_connections 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c6c10c170 0x7f1c6c19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1c540a2d70 0x7f1c540a5230 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1c6c103660 0x7f1c6c19ab40 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 --2- 172.21.15.71:0/1993409492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c6c102d40 0x7f1c6c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 >> 172.21.15.71:0/1993409492 conn(0x7f1c6c0fc7a0 msgr2=0x7f1c6c10d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 shutdown_connections 2024-09-05T12:36:31.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:31.218+0000 7f1c74800640 1 -- 172.21.15.71:0/1993409492 wait complete. 2024-09-05T12:36:31.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.41 MB/sec execute 327 sec latency 1425.178 ms 2024-09-05T12:36:32.200 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2f1ff020', quiesce took 4.6 seconds. Will release after: 21.549999999999997 2024-09-05T12:36:32.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.40 MB/sec execute 328 sec latency 1048.988 ms 2024-09-05T12:36:32.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:32 smithi071 ceph-mon[29845]: pgmap v272: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-09-05T12:36:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:32 smithi123 ceph-mon[34321]: pgmap v272: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-09-05T12:36:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:32 smithi174 ceph-mon[34313]: pgmap v272: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-09-05T12:36:33.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.39 MB/sec execute 329 sec latency 2049.108 ms 2024-09-05T12:36:34.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.38 MB/sec execute 330 sec latency 3049.191 ms 2024-09-05T12:36:34.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:34 smithi071 ceph-mon[29845]: pgmap v273: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 898 KiB/s wr, 91 op/s 2024-09-05T12:36:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:34 smithi123 ceph-mon[34321]: pgmap v273: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 898 KiB/s wr, 91 op/s 2024-09-05T12:36:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:34 smithi174 ceph-mon[34313]: pgmap v273: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 898 KiB/s wr, 91 op/s 2024-09-05T12:36:35.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.37 MB/sec execute 331 sec latency 4049.321 ms 2024-09-05T12:36:36.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.36 MB/sec execute 332 sec latency 5049.451 ms 2024-09-05T12:36:36.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:36 smithi071 ceph-mon[29845]: pgmap v274: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 869 KiB/s wr, 89 op/s 2024-09-05T12:36:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:36 smithi123 ceph-mon[34321]: pgmap v274: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 869 KiB/s wr, 89 op/s 2024-09-05T12:36:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:36 smithi174 ceph-mon[34313]: pgmap v274: 129 pgs: 129 active+clean; 100 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 866 KiB/s rd, 869 KiB/s wr, 89 op/s 2024-09-05T12:36:37.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.35 MB/sec execute 333 sec latency 6049.561 ms 2024-09-05T12:36:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 -- 172.21.15.71:0/2490612219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841090a0 msgr2=0x7fa784110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 --2- 172.21.15.71:0/2490612219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841090a0 0x7fa784110bf0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009ca0 tx=0x7fa77c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 -- 172.21.15.71:0/2490612219 shutdown_connections 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 --2- 172.21.15.71:0/2490612219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7841090a0 0x7fa784110bf0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 --2- 172.21.15.71:0/2490612219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa784104900 0x7fa784108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 --2- 172.21.15.71:0/2490612219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784103f50 0x7fa784104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 -- 172.21.15.71:0/2490612219 >> 172.21.15.71:0/2490612219 conn(0x7fa7840ff9e0 msgr2=0x7fa784101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 -- 172.21.15.71:0/2490612219 shutdown_connections 2024-09-05T12:36:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.830+0000 7fa78be00640 1 -- 172.21.15.71:0/2490612219 wait complete. 2024-09-05T12:36:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.831+0000 7fa78be00640 1 Processor -- start 2024-09-05T12:36:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.831+0000 7fa78be00640 1 -- start start 2024-09-05T12:36:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa78be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 0x7fa784076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa78be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7841090a0 0x7fa784076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa78be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784077180 0x7fa7841aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa78be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa784114120 con 0x7fa784103f50 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa78be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa784113fa0 con 0x7fa784077180 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa78be00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841142a0 con 0x7fa7841090a0 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 0x7fa784076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 0x7fa784076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47886/0 (socket says 172.21.15.71:47886) 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 -- 172.21.15.71:0/4007852658 learned_addr learned my addr 172.21.15.71:0/4007852658 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa788a00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7841090a0 0x7fa784076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789e00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784077180 0x7fa7841aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 -- 172.21.15.71:0/4007852658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7841090a0 msgr2=0x7fa784076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7841090a0 0x7fa784076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 -- 172.21.15.71:0/4007852658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784077180 msgr2=0x7fa7841aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784077180 0x7fa7841aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.832+0000 7fa789400640 1 -- 172.21.15.71:0/4007852658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7841aafe0 con 0x7fa784103f50 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.833+0000 7fa789400640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 0x7fa784076700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e9f0 tx=0x7fa77800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.833+0000 7fa782000640 1 -- 172.21.15.71:0/4007852658 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77800cd80 con 0x7fa784103f50 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.833+0000 7fa782000640 1 -- 172.21.15.71:0/4007852658 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa778010910 con 0x7fa784103f50 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.833+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7841ab2d0 con 0x7fa784103f50 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.833+0000 7fa782000640 1 -- 172.21.15.71:0/4007852658 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778018990 con 0x7fa784103f50 2024-09-05T12:36:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.833+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7841ab840 con 0x7fa784103f50 2024-09-05T12:36:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.836+0000 7fa782000640 1 -- 172.21.15.71:0/4007852658 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa778010ac0 con 0x7fa784103f50 2024-09-05T12:36:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.836+0000 7fa78ecc4740 1 Processor -- start 2024-09-05T12:36:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 -- start start 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd0f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd19e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd1c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bc75100 con 0x55ac3bd0fb00 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bc1d1b0 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bc710c0 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa789400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd19e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa789e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd0f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa789e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd0f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40718/0 (socket says 172.21.15.71:40718) 2024-09-05T12:36:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa788a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd1c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.837+0000 7fa789e00640 1 -- 172.21.15.71:0/3711449736 learned_addr learned my addr 172.21.15.71:0/3711449736 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789400640 1 -- 172.21.15.71:0/3711449736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0f1e0 msgr2=0x55ac3bd0f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789400640 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd0f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789400640 1 -- 172.21.15.71:0/3711449736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd1a3a0 msgr2=0x55ac3bd1c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789400640 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd1c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789400640 1 -- 172.21.15.71:0/3711449736 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3b988210 con 0x55ac3bd0fb00 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa782000640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa76c0a2e20 0x7fa76c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789400640 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd19e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cce10 tx=0x7fa7780d81a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a40324ad075039 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa782000640 1 -- 172.21.15.71:0/4007852658 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa7780cfab0 con 0x7fa784103f50 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa776600640 1 -- 172.21.15.71:0/3711449736 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa778097f30 con 0x55ac3bd0fb00 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa789e00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa76c0a2e20 0x7fa76c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa776600640 1 -- 172.21.15.71:0/3711449736 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7780d6070 con 0x55ac3bd0fb00 2024-09-05T12:36:37.840 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.838+0000 7fa776600640 1 -- 172.21.15.71:0/3711449736 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77809b5f0 con 0x55ac3bd0fb00 2024-09-05T12:36:37.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 -- 172.21.15.71:0/3711449736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0fb00 msgr2=0x55ac3bd19e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa789e00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa76c0a2e20 0x7fa76c0a52e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009a60 tx=0x7fa77c005d80 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd19e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cce10 tx=0x7fa7780d81a0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 -- 172.21.15.71:0/3711449736 shutdown_connections 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd1c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd19e60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/3711449736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd0f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 -- 172.21.15.71:0/3711449736 >> 172.21.15.71:0/3711449736 conn(0x55ac3bd0b750 msgr2=0x55ac3bd0bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 -- 172.21.15.71:0/3711449736 shutdown_connections 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.839+0000 7fa78ecc4740 1 -- 172.21.15.71:0/3711449736 wait complete. 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 -- start start 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 Processor -- start 2024-09-05T12:36:37.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd17930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd17e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bd1cf70 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bbfe090 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac3bd1d0f0 con 0x55ac3bd0fb00 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa78ecc4740 10 client.0 _ll_register_callbacks cb 0x55ac3bc52f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.840+0000 7fa789400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:41140/0 (socket says 172.21.15.71:41140) 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd17e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789400640 1 -- 172.21.15.71:0/626450522 learned_addr learned my addr 172.21.15.71:0/626450522 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa788a00640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd17930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 -- 172.21.15.71:0/626450522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 msgr2=0x55ac3bd17930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd17930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 -- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 msgr2=0x55ac3bd173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 -- 172.21.15.71:0/626450522 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3ba54620 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789400640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa789e00640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd17e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02aeb0 tx=0x7fa77c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8afb116aa6b74d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.841+0000 7fa76aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa76aa00640 1 -- 172.21.15.71:0/626450522 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77c031720 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7780d6070 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa76aa00640 1 -- 172.21.15.71:0/626450522 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa77c0318c0 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa76aa00640 1 -- 172.21.15.71:0/626450522 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa77c003fd0 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 10 client.16404 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac3bd28bd0 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa76aa00640 1 -- 172.21.15.71:0/626450522 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7780d6070 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa76aa00640 1 -- 172.21.15.71:0/626450522 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7fa77c051070 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 10 client.16404 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 20 client.16404 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.842+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd2a150 0x55ac3bd2c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.843+0000 7fa78ecc4740 4 client.16404 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.843+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac3bc82830 con 0x55ac3bd2a150 2024-09-05T12:36:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.843+0000 7fa789400640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd2a150 0x55ac3bd2c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.843+0000 7fa76aa00640 10 client.16404 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.843+0000 7fa789400640 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd2a150 0x55ac3bd2c5d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b2dd415aedd2cf server_cookie=97dfc00165f0ac43 in_seq=0 out_seq=0 2024-09-05T12:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa76aa00640 1 -- 172.21.15.71:0/626450522 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac3bc82830 con 0x55ac3bd2a150 2024-09-05T12:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa76aa00640 10 client.16404 handle_command_reply: tid=0 2024-09-05T12:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 client.16404 shutdown 2024-09-05T12:36:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 msgr2=0x55ac3bd17e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd17e70 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa77c02aeb0 tx=0x7fa77c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 shutdown_connections 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd2a150 0x55ac3bd2c5d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd17e70 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd17930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/626450522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa76aa00640 0 client.16404 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.844+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 >> 172.21.15.71:0/626450522 conn(0x55ac3bd0b750 msgr2=0x55ac3bd11460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.845+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 shutdown_connections 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.845+0000 7fa78ecc4740 1 -- 172.21.15.71:0/626450522 wait complete. 2024-09-05T12:36:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.845+0000 7fa78ecc4740 20 client.16404 trim_cache size 0 max 16384 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.859+0000 7fa78ecc4740 1 Processor -- start 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 -- start start 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd11de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd31ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd32420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b5560 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b7160 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b5c30 con 0x55ac3bd0fb00 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa789e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd32420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa788a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd31ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa788a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd31ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40734/0 (socket says 172.21.15.71:40734) 2024-09-05T12:36:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa789400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd11de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.860+0000 7fa788a00640 1 -- 172.21.15.71:0/1630909538 learned_addr learned my addr 172.21.15.71:0/1630909538 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa789400640 1 -- 172.21.15.71:0/1630909538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 msgr2=0x55ac3bd31ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa789400640 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd31ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa789400640 1 -- 172.21.15.71:0/1630909538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 msgr2=0x55ac3bd32420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa789400640 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd32420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa789400640 1 -- 172.21.15.71:0/1630909538 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac3bd06d60 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa789400640 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd11de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cce40 tx=0x7fa778009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6d4cdee5f123ee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa775c00640 1 -- 172.21.15.71:0/1630909538 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77809b7d0 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa775c00640 1 -- 172.21.15.71:0/1630909538 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7780d6040 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.861+0000 7fa775c00640 1 -- 172.21.15.71:0/1630909538 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa77809ba60 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 -- 172.21.15.71:0/1630909538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 msgr2=0x55ac3bd11de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd11de0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa7780cce40 tx=0x7fa778009310 comp rx=0 tx=0).stop 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 -- 172.21.15.71:0/1630909538 shutdown_connections 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd32420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd31ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/1630909538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd11de0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 -- 172.21.15.71:0/1630909538 >> 172.21.15.71:0/1630909538 conn(0x55ac3bd0b750 msgr2=0x55ac3bd12680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 -- 172.21.15.71:0/1630909538 shutdown_connections 2024-09-05T12:36:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 -- 172.21.15.71:0/1630909538 wait complete. 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 -- start start 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.862+0000 7fa78ecc4740 1 Processor -- start 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd1d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd1db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd92ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b69d0 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b6300 con 0x55ac3bd0fb00 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7841b3560 con 0x55ac3bd1a3a0 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa78ecc4740 10 client.0 _ll_register_callbacks cb 0x7fa77c02f4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd1d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd1d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47906/0 (socket says 172.21.15.71:47906) 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 -- 172.21.15.71:0/2905477263 learned_addr learned my addr 172.21.15.71:0/2905477263 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789e00640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd92ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa788a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd1db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 -- 172.21.15.71:0/2905477263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd1a3a0 msgr2=0x55ac3bd92ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd92ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 -- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0fb00 msgr2=0x55ac3bd1db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd1db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa789400640 1 -- 172.21.15.71:0/2905477263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac3bd933e0 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.863+0000 7fa788a00640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd1db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa789400640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd1d600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa7780d3b70 tx=0x7fa7780d3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989ebc1dd82822c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa76aa00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa76aa00640 1 -- 172.21.15.71:0/2905477263 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7780b9060 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa76aa00640 1 -- 172.21.15.71:0/2905477263 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fa778099f90 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7780d6040 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa76aa00640 1 -- 172.21.15.71:0/2905477263 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa77809b7d0 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 10 client.16410 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ac3bd933e0 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa76aa00640 1 -- 172.21.15.71:0/2905477263 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7780d6040 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa76aa00640 1 -- 172.21.15.71:0/2905477263 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7fa7780a4450 con 0x55ac3bd0f1e0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 10 client.16410 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 20 client.16410 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd946e0 0x55ac3bd96ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 4 client.16410 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.864+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac3bc7cb80 con 0x55ac3bd946e0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.865+0000 7fa788a00640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd946e0 0x55ac3bd96ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.865+0000 7fa788a00640 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd946e0 0x55ac3bd96ad0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f00e15d89c6f43 server_cookie=6801df3905c66468 in_seq=0 out_seq=0 2024-09-05T12:36:37.870 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.865+0000 7fa76aa00640 10 client.16410 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.887+0000 7fa76aa00640 1 -- 172.21.15.71:0/2905477263 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac3bc7cb80 con 0x55ac3bd946e0 2024-09-05T12:36:37.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.887+0000 7fa76aa00640 10 client.16410 handle_command_reply: tid=0 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.887+0000 7fa78ecc4740 1 client.16410 shutdown 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 msgr2=0x55ac3bd1d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd1d600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa7780d3b70 tx=0x7fa7780d3100 comp rx=0 tx=0).stop 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 shutdown_connections 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ac3bd946e0 0x55ac3bd96ad0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3bd1a3a0 0x55ac3bd92ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac3bd0fb00 0x55ac3bd1db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 --2- 172.21.15.71:0/2905477263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ac3bd0f1e0 0x55ac3bd1d600 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa76aa00640 0 client.16410 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:37.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 >> 172.21.15.71:0/2905477263 conn(0x55ac3bd0b750 msgr2=0x55ac3bd173b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 shutdown_connections 2024-09-05T12:36:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.888+0000 7fa78ecc4740 1 -- 172.21.15.71:0/2905477263 wait complete. 2024-09-05T12:36:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.889+0000 7fa78ecc4740 20 client.16410 trim_cache size 0 max 16384 2024-09-05T12:36:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:36:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.889+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa76c0a2e20 msgr2=0x7fa76c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.889+0000 7fa78be00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa76c0a2e20 0x7fa76c0a52e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009a60 tx=0x7fa77c005d80 comp rx=0 tx=0).stop 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.889+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 msgr2=0x7fa784076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.889+0000 7fa78be00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 0x7fa784076700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa77800e9f0 tx=0x7fa77800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 shutdown_connections 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa784077180 0x7fa7841aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7841090a0 0x7fa784076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa76c0a2e20 0x7fa76c0a52e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 --2- 172.21.15.71:0/4007852658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa784103f50 0x7fa784076700 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 >> 172.21.15.71:0/4007852658 conn(0x7fa7840ff9e0 msgr2=0x7fa7841004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 shutdown_connections 2024-09-05T12:36:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:37.890+0000 7fa78be00640 1 -- 172.21.15.71:0/4007852658 wait complete. 2024-09-05T12:36:38.251 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.34 MB/sec execute 334 sec latency 7049.645 ms 2024-09-05T12:36:38.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:38 smithi071 ceph-mon[29845]: pgmap v275: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 911 KiB/s wr, 105 op/s 2024-09-05T12:36:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:38 smithi123 ceph-mon[34321]: pgmap v275: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 911 KiB/s wr, 105 op/s 2024-09-05T12:36:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:38 smithi174 ceph-mon[34313]: pgmap v275: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 911 KiB/s wr, 105 op/s 2024-09-05T12:36:39.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.33 MB/sec execute 335 sec latency 8049.733 ms 2024-09-05T12:36:40.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.32 MB/sec execute 336 sec latency 9049.802 ms 2024-09-05T12:36:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:40 smithi071 ceph-mon[29845]: pgmap v276: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 631 KiB/s wr, 81 op/s 2024-09-05T12:36:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:40 smithi123 ceph-mon[34321]: pgmap v276: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 631 KiB/s wr, 81 op/s 2024-09-05T12:36:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:40 smithi174 ceph-mon[34313]: pgmap v276: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 631 KiB/s wr, 81 op/s 2024-09-05T12:36:41.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.31 MB/sec execute 337 sec latency 10049.868 ms 2024-09-05T12:36:42.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.30 MB/sec execute 338 sec latency 11049.936 ms 2024-09-05T12:36:42.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:42 smithi071 ceph-mon[29845]: pgmap v277: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 745 KiB/s wr, 96 op/s 2024-09-05T12:36:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:42 smithi123 ceph-mon[34321]: pgmap v277: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 745 KiB/s wr, 96 op/s 2024-09-05T12:36:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:42 smithi174 ceph-mon[34313]: pgmap v277: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 892 KiB/s rd, 745 KiB/s wr, 96 op/s 2024-09-05T12:36:43.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.29 MB/sec execute 339 sec latency 12050.085 ms 2024-09-05T12:36:44.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.28 MB/sec execute 340 sec latency 13050.212 ms 2024-09-05T12:36:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:44 smithi071 ceph-mon[29845]: pgmap v278: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 165 KiB/s wr, 34 op/s 2024-09-05T12:36:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:44 smithi123 ceph-mon[34321]: pgmap v278: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 165 KiB/s wr, 34 op/s 2024-09-05T12:36:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:44 smithi174 ceph-mon[34313]: pgmap v278: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 165 KiB/s wr, 34 op/s 2024-09-05T12:36:45.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.27 MB/sec execute 341 sec latency 14050.298 ms 2024-09-05T12:36:46.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.26 MB/sec execute 342 sec latency 15050.424 ms 2024-09-05T12:36:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:46 smithi071 ceph-mon[29845]: pgmap v279: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 158 KiB/s wr, 31 op/s 2024-09-05T12:36:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:46 smithi123 ceph-mon[34321]: pgmap v279: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 158 KiB/s wr, 31 op/s 2024-09-05T12:36:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:46 smithi174 ceph-mon[34313]: pgmap v279: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 158 KiB/s wr, 31 op/s 2024-09-05T12:36:47.252 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.25 MB/sec execute 343 sec latency 16050.531 ms 2024-09-05T12:36:48.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.24 MB/sec execute 344 sec latency 17050.665 ms 2024-09-05T12:36:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:48 smithi071 ceph-mon[29845]: pgmap v280: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 165 KiB/s wr, 38 op/s 2024-09-05T12:36:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:48 smithi123 ceph-mon[34321]: pgmap v280: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 165 KiB/s wr, 38 op/s 2024-09-05T12:36:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:48 smithi174 ceph-mon[34313]: pgmap v280: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 165 KiB/s wr, 38 op/s 2024-09-05T12:36:49.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.23 MB/sec execute 345 sec latency 18050.736 ms 2024-09-05T12:36:50.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.22 MB/sec execute 346 sec latency 19050.827 ms 2024-09-05T12:36:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:50 smithi071 ceph-mon[29845]: pgmap v281: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 123 KiB/s wr, 22 op/s 2024-09-05T12:36:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:50 smithi123 ceph-mon[34321]: pgmap v281: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 123 KiB/s wr, 22 op/s 2024-09-05T12:36:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:50 smithi174 ceph-mon[34313]: pgmap v281: 129 pgs: 129 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 123 KiB/s wr, 22 op/s 2024-09-05T12:36:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 -- 172.21.15.71:0/146665473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c1078c0 msgr2=0x7f4c2c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 --2- 172.21.15.71:0/146665473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c1078c0 0x7f4c2c107ca0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009900 tx=0x7f4c1c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 -- 172.21.15.71:0/146665473 shutdown_connections 2024-09-05T12:36:51.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 --2- 172.21.15.71:0/146665473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c102250 0x7f4c2c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 --2- 172.21.15.71:0/146665473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 --2- 172.21.15.71:0/146665473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c1078c0 0x7f4c2c107ca0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 -- 172.21.15.71:0/146665473 >> 172.21.15.71:0/146665473 conn(0x7f4c2c0fd5d0 msgr2=0x7f4c2c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 -- 172.21.15.71:0/146665473 shutdown_connections 2024-09-05T12:36:51.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.012+0000 7f4c31a00640 1 -- 172.21.15.71:0/146665473 wait complete. 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 Processor -- start 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 -- start start 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c102250 0x7f4c2c198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1078c0 0x7f4c2c19cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c111d50 con 0x7f4c2c1018b0 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c111bd0 con 0x7f4c2c102250 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.013+0000 7f4c31a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c111ed0 con 0x7f4c2c1078c0 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40732/0 (socket says 172.21.15.71:40732) 2024-09-05T12:36:51.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 -- 172.21.15.71:0/435993602 learned_addr learned my addr 172.21.15.71:0/435993602 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c31000640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1078c0 0x7f4c2c19cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2b400640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c102250 0x7f4c2c198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 -- 172.21.15.71:0/435993602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1078c0 msgr2=0x7f4c2c19cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1078c0 0x7f4c2c19cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 -- 172.21.15.71:0/435993602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c102250 msgr2=0x7f4c2c198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c102250 0x7f4c2c198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 -- 172.21.15.71:0/435993602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2c19d210 con 0x7f4c2c1018b0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2b400640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c102250 0x7f4c2c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.014+0000 7f4c2be00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c198220 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c02a9e0 tx=0x7f4c1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.015+0000 7f4c28c00640 1 -- 172.21.15.71:0/435993602 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c03f070 con 0x7f4c2c1018b0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.015+0000 7f4c28c00640 1 -- 172.21.15.71:0/435993602 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c1c031c30 con 0x7f4c2c1018b0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.015+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2c19d4a0 con 0x7f4c2c1018b0 2024-09-05T12:36:51.017 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.015+0000 7f4c28c00640 1 -- 172.21.15.71:0/435993602 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1c035030 con 0x7f4c2c1018b0 2024-09-05T12:36:51.018 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.015+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2c1a4d90 con 0x7f4c2c1018b0 2024-09-05T12:36:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.018+0000 7f4c35dbe740 1 Processor -- start 2024-09-05T12:36:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.018+0000 7f4c28c00640 1 -- 172.21.15.71:0/435993602 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4c1c042070 con 0x7f4c2c1018b0 2024-09-05T12:36:51.019 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.018+0000 7f4c35dbe740 1 -- start start 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323afb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323afbb90 0x559323b0a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323b0a980 0x559323b0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559323a61100 con 0x559323afb270 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559323a091b0 con 0x559323b0a980 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559323a5d0c0 con 0x559323afbb90 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323afb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c2be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323afbb90 0x559323b0a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323afb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40746/0 (socket says 172.21.15.71:40746) 2024-09-05T12:36:51.021 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 -- 172.21.15.71:0/3378116024 learned_addr learned my addr 172.21.15.71:0/3378116024 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c2b400640 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323b0a980 0x559323b0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 -- 172.21.15.71:0/3378116024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323afbb90 msgr2=0x559323b0a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323afbb90 0x559323b0a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 -- 172.21.15.71:0/3378116024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323b0a980 msgr2=0x559323b0cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323b0a980 0x559323b0cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c31000640 1 -- 172.21.15.71:0/3378116024 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559323774210 con 0x559323afb270 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c2be00640 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323afbb90 0x559323b0a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.019+0000 7f4c2b400640 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323b0a980 0x559323b0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c28c00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4c100a2cf0 0x7f4c100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c31000640 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323afb650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009f50 tx=0x7f4c240118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dc505bc6d1644a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c28c00640 1 -- 172.21.15.71:0/435993602 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4c1c0efdf0 con 0x7f4c2c1018b0 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c2be00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4c100a2cf0 0x7f4c100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.022 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c21600640 1 -- 172.21.15.71:0/3378116024 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24004d20 con 0x559323afb270 2024-09-05T12:36:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c21600640 1 -- 172.21.15.71:0/3378116024 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c24004050 con 0x559323afb270 2024-09-05T12:36:51.023 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.020+0000 7f4c21600640 1 -- 172.21.15.71:0/3378116024 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c24005260 con 0x559323afb270 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3378116024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 msgr2=0x559323afb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c2be00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4c100a2cf0 0x7f4c100a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x559323afc520 tx=0x7f4c1c0f38a0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323afb650 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009f50 tx=0x7f4c240118d0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3378116024 shutdown_connections 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323b0a980 0x559323b0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323afbb90 0x559323b0a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.026 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3378116024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323afb650 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3378116024 >> 172.21.15.71:0/3378116024 conn(0x559323af7920 msgr2=0x559323af7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3378116024 shutdown_connections 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.021+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3378116024 wait complete. 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 -- start start 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 Processor -- start 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b07880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b07dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559323b0d5e0 con 0x559323afb270 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593239ea090 con 0x559323afbb90 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559323b0d760 con 0x559323b0a980 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.022+0000 7f4c35dbe740 10 client.0 _ll_register_callbacks cb 0x559323a3ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:51.027 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b07880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b07dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b07880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40760/0 (socket says 172.21.15.71:40760) 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c31000640 1 -- 172.21.15.71:0/2610296341 learned_addr learned my addr 172.21.15.71:0/2610296341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 -- 172.21.15.71:0/2610296341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 msgr2=0x559323b08300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 -- 172.21.15.71:0/2610296341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 msgr2=0x559323b07880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b07880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 -- 172.21.15.71:0/2610296341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559323840620 con 0x559323afbb90 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c31000640 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c2b400640 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b07dc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c18012cf0 tx=0x7f4c1800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=939593ad33d0db95 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c16000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:36:51.028 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c16000640 1 -- 172.21.15.71:0/2610296341 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c180120d0 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.023+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c24004050 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.024+0000 7f4c16000640 1 -- 172.21.15.71:0/2610296341 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4c18010430 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.024+0000 7f4c16000640 1 -- 172.21.15.71:0/2610296341 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4c18012270 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.024+0000 7f4c35dbe740 10 client.26450 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.024+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559323b19360 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.024+0000 7f4c16000640 1 -- 172.21.15.71:0/2610296341 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c24004050 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.024+0000 7f4c16000640 1 -- 172.21.15.71:0/2610296341 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f4c1801c7b0 con 0x559323afbb90 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c35dbe740 10 client.26450 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c35dbe740 10 client.26450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c35dbe740 20 client.26450 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b1a8e0 0x559323b1cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c35dbe740 4 client.26450 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:51.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559323a6e830 con 0x559323b1a8e0 2024-09-05T12:36:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c31000640 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b1a8e0 0x559323b1cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c31000640 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b1a8e0 0x559323b1cd10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac588d85a206f710 server_cookie=e753c91bbe24f3de in_seq=0 out_seq=0 2024-09-05T12:36:51.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.025+0000 7f4c16000640 10 client.26450 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c16000640 1 -- 172.21.15.71:0/2610296341 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559323a6e830 con 0x559323b1a8e0 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c16000640 10 client.26450 handle_command_reply: tid=0 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 client.26450 shutdown 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 msgr2=0x559323b07dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b07dc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c18012cf0 tx=0x7f4c1800f790 comp rx=0 tx=0).stop 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 shutdown_connections 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b1a8e0 0x559323b1cd10 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b07dc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/2610296341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b07880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 >> 172.21.15.71:0/2610296341 conn(0x559323af7920 msgr2=0x559323afd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.026+0000 7f4c16000640 0 client.26450 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.027+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 shutdown_connections 2024-09-05T12:36:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.027+0000 7f4c35dbe740 1 -- 172.21.15.71:0/2610296341 wait complete. 2024-09-05T12:36:51.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.027+0000 7f4c35dbe740 20 client.26450 trim_cache size 0 max 16384 2024-09-05T12:36:51.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.041+0000 7f4c35dbe740 1 Processor -- start 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 -- start start 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b22b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1acb80 con 0x559323afb270 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1ae780 con 0x559323afbb90 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1ad250 con 0x559323b0a980 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c31000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40772/0 (socket says 172.21.15.71:40772) 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c31000640 1 -- 172.21.15.71:0/3915639906 learned_addr learned my addr 172.21.15.71:0/3915639906 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:51.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.042+0000 7f4c2b400640 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b22b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c31000640 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c2b400640 1 -- 172.21.15.71:0/3915639906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 msgr2=0x559323b08fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c2b400640 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c2b400640 1 -- 172.21.15.71:0/3915639906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 msgr2=0x559323b225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c2b400640 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c2b400640 1 -- 172.21.15.71:0/3915639906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559323b02610 con 0x559323afbb90 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c31000640 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:36:51.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c2b400640 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b22b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c1801cca0 tx=0x7f4c18005870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681f56f62290b828 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c20c00640 1 -- 172.21.15.71:0/3915639906 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c1801c420 con 0x559323afbb90 2024-09-05T12:36:51.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c20c00640 1 -- 172.21.15.71:0/3915639906 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c180050b0 con 0x559323afbb90 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.043+0000 7f4c20c00640 1 -- 172.21.15.71:0/3915639906 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c18003970 con 0x559323afbb90 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3915639906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 msgr2=0x559323b22b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b22b20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c1801cca0 tx=0x7f4c18005870 comp rx=0 tx=0).stop 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3915639906 shutdown_connections 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b08fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afbb90 0x559323b22b20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/3915639906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afb270 0x559323b225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3915639906 >> 172.21.15.71:0/3915639906 conn(0x559323af7920 msgr2=0x559323b02b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3915639906 shutdown_connections 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.044+0000 7f4c35dbe740 1 -- 172.21.15.71:0/3915639906 wait complete. 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 -- start start 2024-09-05T12:36:51.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 Processor -- start 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 0x559323b12dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afbb90 0x559323b13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1adff0 con 0x559323afbb90 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1ad920 con 0x559323afb270 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c2c1aab80 con 0x559323b0a980 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.045+0000 7f4c35dbe740 10 client.0 _ll_register_callbacks cb 0x7f4c18009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c2b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afbb90 0x559323b13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 0x559323b12dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 0x559323b12dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60674/0 (socket says 172.21.15.71:60674) 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 -- 172.21.15.71:0/942272426 learned_addr learned my addr 172.21.15.71:0/942272426 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.050 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 -- 172.21.15.71:0/942272426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 msgr2=0x559323b0e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b0e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 -- 172.21.15.71:0/942272426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afbb90 msgr2=0x559323b13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afbb90 0x559323b13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 -- 172.21.15.71:0/942272426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559323b0eb70 con 0x559323afb270 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c31000640 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 0x559323b12dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009d60 tx=0x7f4c2402c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b228a3704dcedb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c16000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c16000640 1 -- 172.21.15.71:0/942272426 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2401e040 con 0x559323afb270 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c180050b0 con 0x559323afb270 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.046+0000 7f4c16000640 1 -- 172.21.15.71:0/942272426 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f4c24003c30 con 0x559323afb270 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c16000640 1 -- 172.21.15.71:0/942272426 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4c24003e30 con 0x559323afb270 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 10 client.26456 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x559323b85220 con 0x559323afb270 2024-09-05T12:36:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c16000640 1 -- 172.21.15.71:0/942272426 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c180050b0 con 0x559323afb270 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c16000640 1 -- 172.21.15.71:0/942272426 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f4c24024040 con 0x559323afb270 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 10 client.26456 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 10 client.26456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 20 client.26456 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b85760 0x559323b87b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 4 client.26456 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.047+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559323a68b80 con 0x559323b85760 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.048+0000 7f4c2b400640 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b85760 0x559323b87b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.048+0000 7f4c16000640 10 client.26456 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.048+0000 7f4c2b400640 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b85760 0x559323b87b50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972afd95b23375f7 server_cookie=768971fa190fa219 in_seq=0 out_seq=0 2024-09-05T12:36:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c16000640 1 -- 172.21.15.71:0/942272426 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559323a68b80 con 0x559323b85760 2024-09-05T12:36:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c16000640 10 client.26456 handle_command_reply: tid=0 2024-09-05T12:36:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 client.26456 shutdown 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 msgr2=0x559323b12dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 0x559323b12dd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009d60 tx=0x7f4c2402c040 comp rx=0 tx=0).stop 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 shutdown_connections 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559323b85760 0x559323b87b50 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559323b0a980 0x559323b0e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559323afbb90 0x559323b13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 --2- 172.21.15.71:0/942272426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559323afb270 0x559323b12dd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c16000640 0 client.26456 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 >> 172.21.15.71:0/942272426 conn(0x559323af7920 msgr2=0x559323b07840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 shutdown_connections 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.068+0000 7f4c35dbe740 1 -- 172.21.15.71:0/942272426 wait complete. 2024-09-05T12:36:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c35dbe740 20 client.26456 trim_cache size 0 max 16384 2024-09-05T12:36:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:36:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:36:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:36:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4c100a2cf0 msgr2=0x7f4c100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4c100a2cf0 0x7f4c100a51b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x559323afc520 tx=0x7f4c1c0f38a0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 msgr2=0x7f4c2c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c198220 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c02a9e0 tx=0x7f4c1c003c10 comp rx=0 tx=0).stop 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 shutdown_connections 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c2c1078c0 0x7f4c2c19cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c2c102250 0x7f4c2c198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4c100a2cf0 0x7f4c100a51b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 --2- 172.21.15.71:0/435993602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c2c1018b0 0x7f4c2c198220 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 >> 172.21.15.71:0/435993602 conn(0x7f4c2c0fd5d0 msgr2=0x7f4c2c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.069+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 shutdown_connections 2024-09-05T12:36:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:36:51.070+0000 7f4c31a00640 1 -- 172.21.15.71:0/435993602 wait complete. 2024-09-05T12:36:51.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.22 MB/sec execute 347 sec latency 20050.914 ms 2024-09-05T12:36:52.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.21 MB/sec execute 348 sec latency 21051.031 ms 2024-09-05T12:36:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:52 smithi071 ceph-mon[29845]: pgmap v282: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 370 KiB/s wr, 33 op/s 2024-09-05T12:36:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:52 smithi123 ceph-mon[34321]: pgmap v282: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 370 KiB/s wr, 33 op/s 2024-09-05T12:36:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:52 smithi174 ceph-mon[34313]: pgmap v282: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 370 KiB/s wr, 33 op/s 2024-09-05T12:36:53.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.20 MB/sec execute 349 sec latency 22051.119 ms 2024-09-05T12:36:53.752 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:36:54.133 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:54.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.19 MB/sec execute 350 sec latency 23051.212 ms 2024-09-05T12:36:54.646 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:54 smithi071 ceph-mon[29845]: pgmap v283: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 256 KiB/s wr, 19 op/s 2024-09-05T12:36:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:54 smithi123 ceph-mon[34321]: pgmap v283: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 256 KiB/s wr, 19 op/s 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 -- 172.21.15.71:0/2684340782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1019d0 msgr2=0x7f0a8c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 --2- 172.21.15.71:0/2684340782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c101e30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009930 tx=0x7f0a7802a420 comp rx=0 tx=0).stop 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 -- 172.21.15.71:0/2684340782 shutdown_connections 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 --2- 172.21.15.71:0/2684340782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c102370 0x7f0a8c10e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 --2- 172.21.15.71:0/2684340782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c101e30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 --2- 172.21.15.71:0/2684340782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c1079d0 0x7f0a8c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 -- 172.21.15.71:0/2684340782 >> 172.21.15.71:0/2684340782 conn(0x7f0a8c0fd710 msgr2=0x7f0a8c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:54.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 -- 172.21.15.71:0/2684340782 shutdown_connections 2024-09-05T12:36:54.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.757+0000 7f0a8b200640 1 -- 172.21.15.71:0/2684340782 wait complete. 2024-09-05T12:36:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:54 smithi174 ceph-mon[34313]: pgmap v283: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 256 KiB/s wr, 19 op/s 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.758+0000 7f0a8b200640 1 Processor -- start 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 -- start start 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c102370 0x7f0a8c1987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1079d0 0x7f0a8c19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111dd0 con 0x7f0a8c1079d0 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111c50 con 0x7f0a8c102370 2024-09-05T12:36:54.762 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a8c111f50 con 0x7f0a8c1019d0 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c102370 0x7f0a8c1987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a89e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a89400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c102370 0x7f0a8c1987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40094/0 (socket says 172.21.15.71:40094) 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a89e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c1982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40840/0 (socket says 172.21.15.71:40840) 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a89400640 1 -- 172.21.15.71:0/3096577279 learned_addr learned my addr 172.21.15.71:0/3096577279 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.759+0000 7f0a8a800640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1079d0 0x7f0a8c19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a89e00640 1 -- 172.21.15.71:0/3096577279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c102370 msgr2=0x7f0a8c1987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a89e00640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c102370 0x7f0a8c1987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a89e00640 1 -- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1079d0 msgr2=0x7f0a8c19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a89e00640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1079d0 0x7f0a8c19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.763 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a89e00640 1 -- 172.21.15.71:0/3096577279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8c19d250 con 0x7f0a8c1019d0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a89e00640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c1982a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e9f0 tx=0x7f0a7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00ccc0 con 0x7f0a8c1019d0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.760+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a7c010910 con 0x7f0a8c1019d0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.761+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8c19d540 con 0x7f0a8c1019d0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.761+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c018960 con 0x7f0a8c1019d0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.761+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8c1a4e20 con 0x7f0a8c1019d0 2024-09-05T12:36:54.764 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.762+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a8c101e30 con 0x7f0a8c1019d0 2024-09-05T12:36:54.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.764+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0a7c010ac0 con 0x7f0a8c1019d0 2024-09-05T12:36:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.765+0000 7f0a82a00640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0a700a2d70 0x7f0a700a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.765+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7f0a7c0cff50 con 0x7f0a8c1019d0 2024-09-05T12:36:54.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.765+0000 7f0a89400640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0a700a2d70 0x7f0a700a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:54.767 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.766+0000 7f0a89400640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0a700a2d70 0x7f0a700a5230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0ffd20 tx=0x7f0a78009560 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:54.768 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.767+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a7c09c130 con 0x7f0a8c1019d0 2024-09-05T12:36:54.978 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.977+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a8c199600 con 0x7f0a8c1019d0 2024-09-05T12:36:54.979 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.978+0000 7f0a82a00640 1 -- 172.21.15.71:0/3096577279 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11976 (secure 0 0 0) 0x7f0a8c199600 con 0x7f0a8c1019d0 2024-09-05T12:36:54.979 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:36:54.979 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":41,"btime":"2024-09-05T12:36:02:247256+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:36:01.371812+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24478":{"gid":24478,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/626911382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":626911382},{"type":"v1","addr":"172.21.15.174:6835","nonce":626911382}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:36:54.980 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 41 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.980+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0a700a2d70 msgr2=0x7f0a700a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0a700a2d70 0x7f0a700a5230 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c0ffd20 tx=0x7f0a78009560 comp rx=0 tx=0).stop 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 msgr2=0x7f0a8c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c1982a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e9f0 tx=0x7f0a7c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 shutdown_connections 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0a700a2d70 0x7f0a700a5230 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a8c1079d0 0x7f0a8c19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a8c102370 0x7f0a8c1987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 --2- 172.21.15.71:0/3096577279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a8c1019d0 0x7f0a8c1982a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 >> 172.21.15.71:0/3096577279 conn(0x7f0a8c0fd710 msgr2=0x7f0a8c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 shutdown_connections 2024-09-05T12:36:54.982 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:54.981+0000 7f0a8b200640 1 -- 172.21.15.71:0/3096577279 wait complete. 2024-09-05T12:36:55.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.18 MB/sec execute 351 sec latency 24051.326 ms 2024-09-05T12:36:55.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:55 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3096577279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:55.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:55 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3096577279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:55.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:55 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3096577279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:36:55.855 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 2f1ff020 --release --await' 2024-09-05T12:36:55.855 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 2f1ff020 --release --await 2024-09-05T12:36:56.220 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:36:56.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303217 3.17 MB/sec execute 352 sec latency 25051.397 ms 2024-09-05T12:36:56.618 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:56 smithi071 ceph-mon[29845]: pgmap v284: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 255 KiB/s wr, 18 op/s 2024-09-05T12:36:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:56 smithi123 ceph-mon[34321]: pgmap v284: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 255 KiB/s wr, 18 op/s 2024-09-05T12:36:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:56 smithi174 ceph-mon[34313]: pgmap v284: 129 pgs: 129 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 255 KiB/s wr, 18 op/s 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 -- 172.21.15.71:0/3486542590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074e30 msgr2=0x7fc7f010df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/3486542590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074e30 0x7fc7f010df10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009900 tx=0x7fc7d802a480 comp rx=0 tx=0).stop 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 -- 172.21.15.71:0/3486542590 shutdown_connections 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/3486542590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f010e450 0x7fc7f0110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/3486542590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074e30 0x7fc7f010df10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/3486542590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f0074510 0x7fc7f00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 -- 172.21.15.71:0/3486542590 >> 172.21.15.71:0/3486542590 conn(0x7fc7f00fd8b0 msgr2=0x7fc7f00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:56.907 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 -- 172.21.15.71:0/3486542590 shutdown_connections 2024-09-05T12:36:56.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.906+0000 7fc7f5e00640 1 -- 172.21.15.71:0/3486542590 wait complete. 2024-09-05T12:36:56.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.907+0000 7fc7f5e00640 1 Processor -- start 2024-09-05T12:36:56.908 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.907+0000 7fc7f5e00640 1 -- start start 2024-09-05T12:36:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 0x7fc7f019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f0074e30 0x7fc7f019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7f5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f010e450 0x7fc7f019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7f5e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01141c0 con 0x7fc7f0074510 2024-09-05T12:36:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7f5e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0114040 con 0x7fc7f010e450 2024-09-05T12:36:56.909 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7f5e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f0114340 con 0x7fc7f0074e30 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 0x7fc7f019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 0x7fc7f019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52170/0 (socket says 172.21.15.71:52170) 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 -- 172.21.15.71:0/4226078651 learned_addr learned my addr 172.21.15.71:0/4226078651 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7e6000640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f0074e30 0x7fc7f019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7ef400640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f010e450 0x7fc7f019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 -- 172.21.15.71:0/4226078651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f0074e30 msgr2=0x7fc7f019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f0074e30 0x7fc7f019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 -- 172.21.15.71:0/4226078651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f010e450 msgr2=0x7fc7f019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f010e450 0x7fc7f019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.908+0000 7fc7eea00640 1 -- 172.21.15.71:0/4226078651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f019f6d0 con 0x7fc7f0074510 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.909+0000 7fc7eea00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 0x7fc7f019a720 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000ea60 tx=0x7fc7e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.909+0000 7fc7f4a00640 1 -- 172.21.15.71:0/4226078651 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e000cde0 con 0x7fc7f0074510 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.909+0000 7fc7f4a00640 1 -- 172.21.15.71:0/4226078651 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e00035f0 con 0x7fc7f0074510 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.909+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f019f9c0 con 0x7fc7f0074510 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.909+0000 7fc7f4a00640 1 -- 172.21.15.71:0/4226078651 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0010670 con 0x7fc7f0074510 2024-09-05T12:36:56.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.909+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f01a72a0 con 0x7fc7f0074510 2024-09-05T12:36:56.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.912+0000 7fc7f4a00640 1 -- 172.21.15.71:0/4226078651 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc7e00037a0 con 0x7fc7f0074510 2024-09-05T12:36:56.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.912+0000 7fc7f8a2f740 1 Processor -- start 2024-09-05T12:36:56.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.912+0000 7fc7f8a2f740 1 -- start start 2024-09-05T12:36:56.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d196f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d23fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563da8c7f650 con 0x563da8d19c30 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563da8c26f10 con 0x563da8d244f0 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563da8c7b610 con 0x563da8d19310 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7ef400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7ef400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d196f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40870/0 (socket says 172.21.15.71:40870) 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7ef400640 1 -- 172.21.15.71:0/2810450363 learned_addr learned my addr 172.21.15.71:0/2810450363 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7eea00640 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d23fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.913+0000 7fc7e6000640 1 -- 172.21.15.71:0/2810450363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 msgr2=0x563da8d196f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7e6000640 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d196f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7e6000640 1 -- 172.21.15.71:0/2810450363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 msgr2=0x563da8d23fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7e6000640 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d23fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7e6000640 1 -- 172.21.15.71:0/2810450363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da898e580 con 0x563da8d244f0 2024-09-05T12:36:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7eea00640 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d23fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:36:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7f4a00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d40a2d70 0x7fc7d40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7e6000640 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d268e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7d802aa70 tx=0x7fc7d8005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fd02fc6faa16947 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:56.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7f4a00640 1 -- 172.21.15.71:0/4226078651 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc7e00cfad0 con 0x7fc7f0074510 2024-09-05T12:36:56.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7dfe00640 1 -- 172.21.15.71:0/2810450363 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8035040 con 0x563da8d244f0 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7ef400640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d40a2d70 0x7fc7d40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7dfe00640 1 -- 172.21.15.71:0/2810450363 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d8004400 con 0x563da8d244f0 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.914+0000 7fc7dfe00640 1 -- 172.21.15.71:0/2810450363 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80035f0 con 0x563da8d244f0 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/2810450363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 msgr2=0x563da8d268e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7ef400640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d40a2d70 0x7fc7d40a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8011630 tx=0x7fc7e8009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d268e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc7d802aa70 tx=0x7fc7d8005de0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/2810450363 shutdown_connections 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d268e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d23fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/2810450363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d196f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/2810450363 >> 172.21.15.71:0/2810450363 conn(0x563da8d158a0 msgr2=0x563da8d15c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/2810450363 shutdown_connections 2024-09-05T12:36:56.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.915+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/2810450363 wait complete. 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.916+0000 7fc7f8a2f740 1 -- start start 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.916+0000 7fc7f8a2f740 1 Processor -- start 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.916+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19310 0x563da8d21430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.916+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d21970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.916+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d21eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563da8d27060 con 0x563da8d19310 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563da8c3a350 con 0x563da8d244f0 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563da8d271e0 con 0x563da8d19c30 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7f8a2f740 10 client.0 _ll_register_callbacks cb 0x563da8c5d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19310 0x563da8d21430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7ef400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d21eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19310 0x563da8d21430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52196/0 (socket says 172.21.15.71:52196) 2024-09-05T12:36:56.919 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7eea00640 1 -- 172.21.15.71:0/1007789553 learned_addr learned my addr 172.21.15.71:0/1007789553 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:56.920 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7e6000640 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d21970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7ef400640 1 -- 172.21.15.71:0/1007789553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 msgr2=0x563da8d21970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7ef400640 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d21970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7ef400640 1 -- 172.21.15.71:0/1007789553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19310 msgr2=0x563da8d21430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7ef400640 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19310 0x563da8d21430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.917+0000 7fc7ef400640 1 -- 172.21.15.71:0/1007789553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da8a58960 con 0x563da8d244f0 2024-09-05T12:36:56.922 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7ef400640 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d21eb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8003f50 tx=0x7fc7e8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c10fdaf46ba37d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7dcc00640 1 -- 172.21.15.71:0/1007789553 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f830 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7dcc00640 1 -- 172.21.15.71:0/1007789553 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc7e800f9d0 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7d8004400 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7dcc00640 1 -- 172.21.15.71:0/1007789553 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc7e800fbd0 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7f8a2f740 10 client.26474 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563da8d32d20 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.918+0000 7fc7dcc00640 1 -- 172.21.15.71:0/1007789553 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7d8004400 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7dcc00640 1 -- 172.21.15.71:0/1007789553 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7fc7e8026020 con 0x563da8d244f0 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7f8a2f740 10 client.26474 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7f8a2f740 10 client.26474 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7f8a2f740 20 client.26474 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:56.923 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8d34230 0x563da8d36690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7f8a2f740 4 client.26474 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563da8c8cbc0 con 0x563da8d34230 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.919+0000 7fc7eea00640 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8d34230 0x563da8d36690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.920+0000 7fc7eea00640 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8d34230 0x563da8d36690 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc010ea3830669f server_cookie=9827f3146cf2290b in_seq=0 out_seq=0 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.920+0000 7fc7dcc00640 10 client.26474 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.920+0000 7fc7dcc00640 1 -- 172.21.15.71:0/1007789553 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563da8c8cbc0 con 0x563da8d34230 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.920+0000 7fc7dcc00640 10 client.26474 handle_command_reply: tid=0 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 client.26474 shutdown 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 msgr2=0x563da8d21eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d21eb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8003f50 tx=0x7fc7e8004a40 comp rx=0 tx=0).stop 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 shutdown_connections 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8d34230 0x563da8d36690 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d21eb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d21970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7dcc00640 0 client.26474 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/1007789553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19310 0x563da8d21430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 >> 172.21.15.71:0/1007789553 conn(0x563da8d158a0 msgr2=0x563da8d1b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 shutdown_connections 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/1007789553 wait complete. 2024-09-05T12:36:56.925 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.921+0000 7fc7f8a2f740 20 client.26474 trim_cache size 0 max 16384 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 Processor -- start 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 -- start start 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 0x563da8d1b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d244f0 0x563da8d3bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01b1210 con 0x563da8d244f0 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01b2e10 con 0x563da8d19310 2024-09-05T12:36:56.940 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01b18e0 con 0x563da8d19c30 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 0x563da8d1b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 0x563da8d1b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40138/0 (socket says 172.21.15.71:40138) 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7eea00640 1 -- 172.21.15.71:0/958536164 learned_addr learned my addr 172.21.15.71:0/958536164 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.939+0000 7fc7e6000640 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7ef400640 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d244f0 0x563da8d3bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7eea00640 1 -- 172.21.15.71:0/958536164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 msgr2=0x563da8d1fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7eea00640 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d1fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7eea00640 1 -- 172.21.15.71:0/958536164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d244f0 msgr2=0x563da8d3bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7eea00640 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d244f0 0x563da8d3bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7eea00640 1 -- 172.21.15.71:0/958536164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563da8d36d10 con 0x563da8d19310 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7ef400640 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d244f0 0x563da8d3bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7eea00640 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 0x563da8d1b420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0003ee0 tx=0x7fc7e00d1f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fecdf75d0d45949 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7df400640 1 -- 172.21.15.71:0/958536164 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e009c300 con 0x563da8d19310 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7df400640 1 -- 172.21.15.71:0/958536164 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e00b2070 con 0x563da8d19310 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.940+0000 7fc7df400640 1 -- 172.21.15.71:0/958536164 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0098270 con 0x563da8d19310 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/958536164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 msgr2=0x563da8d1b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 0x563da8d1b420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0003ee0 tx=0x7fc7e00d1f20 comp rx=0 tx=0).stop 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/958536164 shutdown_connections 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d244f0 0x563da8d3bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19c30 0x563da8d1fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/958536164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d19310 0x563da8d1b420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/958536164 >> 172.21.15.71:0/958536164 conn(0x563da8d158a0 msgr2=0x563da8d14aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/958536164 shutdown_connections 2024-09-05T12:36:56.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/958536164 wait complete. 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.941+0000 7fc7f8a2f740 1 -- start start 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 Processor -- start 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d2c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d29160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01b2680 con 0x563da8d19c30 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01b1fb0 con 0x563da8d244f0 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f01af210 con 0x563da8d19310 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7f8a2f740 10 client.0 _ll_register_callbacks cb 0x7fc7e8005df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:36:56.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7e6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d28c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52214/0 (socket says 172.21.15.71:52214) 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.942+0000 7fc7e6000640 1 -- 172.21.15.71:0/3271907309 learned_addr learned my addr 172.21.15.71:0/3271907309 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7ef400640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d29160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7eea00640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d2c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7e6000640 1 -- 172.21.15.71:0/3271907309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 msgr2=0x563da8d2c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7e6000640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d2c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7e6000640 1 -- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 msgr2=0x563da8d29160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7e6000640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d29160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7e6000640 1 -- 172.21.15.71:0/3271907309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563da8d296d0 con 0x563da8d19c30 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7eea00640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7e6000640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d28c20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x563da8d16c20 tx=0x7fc7d802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f6807144fdaf31 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.943+0000 7fc7dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7dcc00640 1 -- 172.21.15.71:0/3271907309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8035040 con 0x563da8d19c30 2024-09-05T12:36:56.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7e00b2070 con 0x563da8d19c30 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7dcc00640 1 -- 172.21.15.71:0/3271907309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9438+0+0 (secure 0 0 0) 0x7fc7d80035f0 con 0x563da8d19c30 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7dcc00640 1 -- 172.21.15.71:0/3271907309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fc7d805d050 con 0x563da8d19c30 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7f8a2f740 10 client.16449 fetch_fsmap learned FSMap version 41 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x563da8da8460 con 0x563da8d19c30 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7dcc00640 1 -- 172.21.15.71:0/3271907309 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e00b2070 con 0x563da8d19c30 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7dcc00640 1 -- 172.21.15.71:0/3271907309 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7fc7d80319a0 con 0x563da8d19c30 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7f8a2f740 10 client.16449 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7f8a2f740 10 client.16449 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.944+0000 7fc7f8a2f740 20 client.16449 populate_metadata read hostname 'smithi071' 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.945+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8da8880 0x563da8daac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.945+0000 7fc7f8a2f740 4 client.16449 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2f1ff020", "release": true, "await": true}] 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.945+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2f1ff020", "release": true, "await": true}) -- 0x563da8c870d0 con 0x563da8da8880 2024-09-05T12:36:56.948 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.945+0000 7fc7eea00640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8da8880 0x563da8daac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:36:56.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.945+0000 7fc7eea00640 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8da8880 0x563da8daac70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a289be95f85a8d46 server_cookie=4a517c3dae22b51b in_seq=0 out_seq=0 2024-09-05T12:36:56.949 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.945+0000 7fc7dcc00640 10 client.16449 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:56.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7dcc00640 1 -- 172.21.15.71:0/3271907309 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563da8c870d0 con 0x563da8da8880 2024-09-05T12:36:56.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7dcc00640 10 client.16449 handle_command_reply: tid=0 2024-09-05T12:36:56.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 client.16449 shutdown 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 msgr2=0x563da8d28c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d28c20 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x563da8d16c20 tx=0x7fc7d802a480 comp rx=0 tx=0).stop 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 shutdown_connections 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563da8da8880 0x563da8daac70 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563da8d244f0 0x563da8d29160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563da8d19c30 0x563da8d28c20 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 --2- 172.21.15.71:0/3271907309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563da8d19310 0x563da8d2c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 >> 172.21.15.71:0/3271907309 conn(0x563da8d158a0 msgr2=0x563da8d213f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.952+0000 7fc7dcc00640 0 client.16449 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:36:56.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 shutdown_connections 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f8a2f740 1 -- 172.21.15.71:0/3271907309 wait complete. 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f8a2f740 20 client.16449 trim_cache size 0 max 16384 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 41, 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 32, 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:36:56.955 INFO:teuthology.orchestra.run.smithi071.stdout: "2f1ff020": { 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 32, 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 146.2, 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:36:56.956 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d40a2d70 msgr2=0x7fc7d40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d40a2d70 0x7fc7d40a5230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8011630 tx=0x7fc7e8009500 comp rx=0 tx=0).stop 2024-09-05T12:36:56.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 msgr2=0x7fc7f019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.953+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 0x7fc7f019a720 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc7e000ea60 tx=0x7fc7e000ef20 comp rx=0 tx=0).stop 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 shutdown_connections 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7f010e450 0x7fc7f019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7f0074e30 0x7fc7f019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d40a2d70 0x7fc7d40a5230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 --2- 172.21.15.71:0/4226078651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7f0074510 0x7fc7f019a720 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 >> 172.21.15.71:0/4226078651 conn(0x7fc7f00fd8b0 msgr2=0x7fc7f00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 shutdown_connections 2024-09-05T12:36:56.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:36:56.954+0000 7fc7f5e00640 1 -- 172.21.15.71:0/4226078651 wait complete. 2024-09-05T12:36:57.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 303899 3.17 MB/sec execute 353 sec latency 25754.259 ms 2024-09-05T12:36:57.802 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2f1ff020', total seconds elapsed: 30.2 2024-09-05T12:36:58.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 305475 3.19 MB/sec execute 354 sec latency 9.668 ms 2024-09-05T12:36:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:36:58 smithi071 ceph-mon[29845]: pgmap v285: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 294 KiB/s wr, 34 op/s 2024-09-05T12:36:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:36:58 smithi123 ceph-mon[34321]: pgmap v285: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 294 KiB/s wr, 34 op/s 2024-09-05T12:36:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:36:58 smithi174 ceph-mon[34313]: pgmap v285: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 294 KiB/s wr, 34 op/s 2024-09-05T12:36:59.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 306634 3.20 MB/sec execute 355 sec latency 16.447 ms 2024-09-05T12:37:00.253 INFO:tasks.workunit.client.0.smithi071.stdout: 1 306757 3.19 MB/sec execute 356 sec latency 897.637 ms 2024-09-05T12:37:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:00 smithi071 ceph-mon[29845]: pgmap v286: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 287 KiB/s wr, 27 op/s 2024-09-05T12:37:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:37:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:37:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:37:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:37:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:00 smithi123 ceph-mon[34321]: pgmap v286: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 287 KiB/s wr, 27 op/s 2024-09-05T12:37:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:37:00.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:37:00.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:37:00.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:37:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:00 smithi174 ceph-mon[34313]: pgmap v286: 129 pgs: 129 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 287 KiB/s wr, 27 op/s 2024-09-05T12:37:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:37:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:37:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:37:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:37:01.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 306757 3.18 MB/sec execute 357 sec latency 1897.760 ms 2024-09-05T12:37:02.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 307339 3.17 MB/sec execute 358 sec latency 2026.195 ms 2024-09-05T12:37:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:02 smithi071 ceph-mon[29845]: pgmap v287: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 586 KiB/s wr, 78 op/s 2024-09-05T12:37:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:02 smithi123 ceph-mon[34321]: pgmap v287: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 586 KiB/s wr, 78 op/s 2024-09-05T12:37:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:02 smithi174 ceph-mon[34313]: pgmap v287: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 586 KiB/s wr, 78 op/s 2024-09-05T12:37:03.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 307631 3.17 MB/sec execute 359 sec latency 203.122 ms 2024-09-05T12:37:03.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:03 smithi071 ceph-mon[29845]: osdmap e102: 12 total, 12 up, 12 in 2024-09-05T12:37:03.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:03 smithi123 ceph-mon[34321]: osdmap e102: 12 total, 12 up, 12 in 2024-09-05T12:37:03.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:03 smithi174 ceph-mon[34313]: osdmap e102: 12 total, 12 up, 12 in 2024-09-05T12:37:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 -- 172.21.15.71:0/2121136819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6804103f50 msgr2=0x7f6804104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 --2- 172.21.15.71:0/2121136819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6804103f50 0x7f6804104330 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009900 tx=0x7f67f402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 -- 172.21.15.71:0/2121136819 shutdown_connections 2024-09-05T12:37:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 --2- 172.21.15.71:0/2121136819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68041090a0 0x7f6804110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 --2- 172.21.15.71:0/2121136819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6804104900 0x7f6804108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 --2- 172.21.15.71:0/2121136819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6804103f50 0x7f6804104330 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 -- 172.21.15.71:0/2121136819 >> 172.21.15.71:0/2121136819 conn(0x7f68040ff9e0 msgr2=0x7f6804101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 -- 172.21.15.71:0/2121136819 shutdown_connections 2024-09-05T12:37:04.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.193+0000 7f680a000640 1 -- 172.21.15.71:0/2121136819 wait complete. 2024-09-05T12:37:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 Processor -- start 2024-09-05T12:37:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 -- start start 2024-09-05T12:37:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6804103f50 0x7f680419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 0x7f680419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.198 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f680419ee80 0x7f680419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6804114110 con 0x7f6804103f50 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6804113f90 con 0x7f68041090a0 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.194+0000 7f680a000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6804114290 con 0x7f680419ee80 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 0x7f680419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6803e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f680419ee80 0x7f680419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6803400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6804103f50 0x7f680419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 0x7f680419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44142/0 (socket says 172.21.15.71:44142) 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 -- 172.21.15.71:0/2590215416 learned_addr learned my addr 172.21.15.71:0/2590215416 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6803e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f680419ee80 0x7f680419f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55652/0 (socket says 172.21.15.71:55652) 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 -- 172.21.15.71:0/2590215416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f680419ee80 msgr2=0x7f680419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f680419ee80 0x7f680419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.199 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 -- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6804103f50 msgr2=0x7f680419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6804103f50 0x7f680419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 -- 172.21.15.71:0/2590215416 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680419f970 con 0x7f68041090a0 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f6802a00640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 0x7f680419ab00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f67f000e9a0 tx=0x7f67f000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.195+0000 7f67fbe00640 1 -- 172.21.15.71:0/2590215416 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f000cd80 con 0x7f68041090a0 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.196+0000 7f67fbe00640 1 -- 172.21.15.71:0/2590215416 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f00035f0 con 0x7f68041090a0 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.196+0000 7f67fbe00640 1 -- 172.21.15.71:0/2590215416 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f0010670 con 0x7f68041090a0 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.196+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680419fc60 con 0x7f68041090a0 2024-09-05T12:37:04.200 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.196+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041a7400 con 0x7f68041090a0 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 Processor -- start 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 -- start start 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e59297f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 0x55e592989e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e5928e5100 con 0x55e59298a3a0 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e59288d1b0 con 0x55e59297fb00 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.199+0000 7f680d034740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5928e10c0 con 0x55e59297f1e0 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 0x55e592989e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 0x55e592989e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44146/0 (socket says 172.21.15.71:44146) 2024-09-05T12:37:04.202 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 -- 172.21.15.71:0/2052589778 learned_addr learned my addr 172.21.15.71:0/2052589778 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803e00640 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e59297f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 -- 172.21.15.71:0/2052589778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 msgr2=0x55e59297f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e59297f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 -- 172.21.15.71:0/2052589778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 msgr2=0x55e59298c790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298c790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 -- 172.21.15.71:0/2052589778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5925f8210 con 0x55e59297fb00 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803e00640 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e59297f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:04.203 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f67fbe00640 1 -- 172.21.15.71:0/2590215416 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f67f00037a0 con 0x7f68041090a0 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.200+0000 7f6803400640 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 0x55e592989e60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009900 tx=0x7f67f4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511707a6b993c367 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.201+0000 7f67efe00640 1 -- 172.21.15.71:0/2052589778 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4035040 con 0x55e59297fb00 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.201+0000 7f67efe00640 1 -- 172.21.15.71:0/2052589778 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4004400 con 0x55e59297fb00 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.201+0000 7f67efe00640 1 -- 172.21.15.71:0/2052589778 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f40035f0 con 0x55e59297fb00 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.201+0000 7f680d034740 1 -- 172.21.15.71:0/2052589778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 msgr2=0x55e592989e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.201+0000 7f680d034740 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 0x55e592989e60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009900 tx=0x7f67f4005de0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 -- 172.21.15.71:0/2052589778 shutdown_connections 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297fb00 0x55e592989e60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 --2- 172.21.15.71:0/2052589778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e59297f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.206 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 -- 172.21.15.71:0/2052589778 >> 172.21.15.71:0/2052589778 conn(0x55e59297b750 msgr2=0x55e59297bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f67fbe00640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67e80a2e20 0x7f67e80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 -- 172.21.15.71:0/2052589778 shutdown_connections 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 -- 172.21.15.71:0/2052589778 wait complete. 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f67fbe00640 1 -- 172.21.15.71:0/2590215416 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f67f00cfa70 con 0x7f68041090a0 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f6803400640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67e80a2e20 0x7f67e80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.202+0000 7f680d034740 1 -- start start 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 Processor -- start 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f6803400640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67e80a2e20 0x7f67e80a52e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67f4005c40 tx=0x7f67f402a3f0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e5929873c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592987900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e592987e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e59298cf70 con 0x55e59298a3a0 2024-09-05T12:37:04.207 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e59286e090 con 0x55e59297f1e0 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e59298d0f0 con 0x55e59297fb00 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.203+0000 7f680d034740 10 client.0 _ll_register_callbacks cb 0x55e5928c2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e5929873c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e5929873c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44158/0 (socket says 172.21.15.71:44158) 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 -- 172.21.15.71:0/538340812 learned_addr learned my addr 172.21.15.71:0/538340812 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6802a00640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592987900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803e00640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e592987e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 -- 172.21.15.71:0/538340812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 msgr2=0x55e592987900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592987900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 -- 172.21.15.71:0/538340812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 msgr2=0x55e592987e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e592987e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 -- 172.21.15.71:0/538340812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5926c4620 con 0x55e59297f1e0 2024-09-05T12:37:04.208 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f6803400640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e5929873c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009900 tx=0x7f67f4004be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2437d9fe0cb2a5df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:04.209 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.204+0000 7f67ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f67ecc00640 1 -- 172.21.15.71:0/538340812 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4035070 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f67ecc00640 1 -- 172.21.15.71:0/538340812 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f67f404a030 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f4004400 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f67ecc00640 1 -- 172.21.15.71:0/538340812 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f67f40054e0 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f680d034740 10 client.26492 fetch_fsmap learned FSMap version 41 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e592998c40 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f67ecc00640 1 -- 172.21.15.71:0/538340812 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4004400 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f67ecc00640 1 -- 172.21.15.71:0/538340812 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f67f4031810 con 0x55e59297f1e0 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f680d034740 10 client.26492 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f680d034740 10 client.26492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.205+0000 7f680d034740 20 client.26492 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:04.211 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.206+0000 7f680d034740 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592999f90 0x55e59299c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.206+0000 7f680d034740 4 client.26492 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.206+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5928f2830 con 0x55e592999f90 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.206+0000 7f6802a00640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592999f90 0x55e59299c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.206+0000 7f6802a00640 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592999f90 0x55e59299c380 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f88ded8879b92db server_cookie=18193ddde1f4d68b in_seq=0 out_seq=0 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.206+0000 7f67ecc00640 10 client.26492 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f67ecc00640 1 -- 172.21.15.71:0/538340812 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5928f2830 con 0x55e592999f90 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f67ecc00640 10 client.26492 handle_command_reply: tid=0 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 client.26492 shutdown 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 msgr2=0x55e5929873c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e5929873c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009900 tx=0x7f67f4004be0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 shutdown_connections 2024-09-05T12:37:04.212 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592999f90 0x55e59299c380 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e592987e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592987900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 --2- 172.21.15.71:0/538340812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e5929873c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 >> 172.21.15.71:0/538340812 conn(0x55e59297b750 msgr2=0x55e592981460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.207+0000 7f67ecc00640 0 client.26492 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.208+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 shutdown_connections 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.208+0000 7f680d034740 1 -- 172.21.15.71:0/538340812 wait complete. 2024-09-05T12:37:04.213 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.208+0000 7f680d034740 20 client.26492 trim_cache size 0 max 16384 2024-09-05T12:37:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.224+0000 7f680d034740 1 Processor -- start 2024-09-05T12:37:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.224+0000 7f680d034740 1 -- start start 2024-09-05T12:37:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.224+0000 7f680d034740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e592989220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f680d034740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 0x55e5929a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f680d034740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59298a3a0 0x55e5929a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f680d034740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68041b1290 con 0x55e59297fb00 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f680d034740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68041b2e90 con 0x55e59298a3a0 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f680d034740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68041b1960 con 0x55e59297f1e0 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e592989220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6802a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 0x55e5929a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e592989220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55676/0 (socket says 172.21.15.71:55676) 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 -- 172.21.15.71:0/1582099809 learned_addr learned my addr 172.21.15.71:0/1582099809 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6802a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 0x55e5929a1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43242/0 (socket says 172.21.15.71:43242) 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803e00640 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59298a3a0 0x55e5929a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 -- 172.21.15.71:0/1582099809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59298a3a0 msgr2=0x55e5929a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59298a3a0 0x55e5929a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 -- 172.21.15.71:0/1582099809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 msgr2=0x55e5929a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 0x55e5929a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 -- 172.21.15.71:0/1582099809 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59297c8c0 con 0x55e59297f1e0 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6802a00640 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 0x55e5929a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.225+0000 7f6803400640 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e592989220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67f4000c00 tx=0x7f67f4004380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32d9d6882beae47 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.226+0000 7f67ef400640 1 -- 172.21.15.71:0/1582099809 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4035040 con 0x55e59297f1e0 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.226+0000 7f67ef400640 1 -- 172.21.15.71:0/1582099809 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4004990 con 0x55e59297f1e0 2024-09-05T12:37:04.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.226+0000 7f67ef400640 1 -- 172.21.15.71:0/1582099809 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4003c80 con 0x55e59297f1e0 2024-09-05T12:37:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.226+0000 7f680d034740 1 -- 172.21.15.71:0/1582099809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 msgr2=0x55e592989220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.232 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e592989220 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f67f4000c00 tx=0x7f67f4004380 comp rx=0 tx=0).stop 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 -- 172.21.15.71:0/1582099809 shutdown_connections 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59298a3a0 0x55e5929a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59297fb00 0x55e5929a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 --2- 172.21.15.71:0/1582099809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297f1e0 0x55e592989220 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 -- 172.21.15.71:0/1582099809 >> 172.21.15.71:0/1582099809 conn(0x55e59297b750 msgr2=0x55e592988b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 -- 172.21.15.71:0/1582099809 shutdown_connections 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 -- 172.21.15.71:0/1582099809 wait complete. 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 -- start start 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.227+0000 7f680d034740 1 Processor -- start 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e592992340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592992880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68041b2700 con 0x55e59298a3a0 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68041b2030 con 0x55e59297f1e0 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68041af290 con 0x55e59297fb00 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f680d034740 10 client.0 _ll_register_callbacks cb 0x7f67fc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6802a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592992880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e592992340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e592992340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44172/0 (socket says 172.21.15.71:44172) 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803400640 1 -- 172.21.15.71:0/3034254571 learned_addr learned my addr 172.21.15.71:0/3034254571 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6802a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592992880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55692/0 (socket says 172.21.15.71:55692) 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803400640 1 -- 172.21.15.71:0/3034254571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 msgr2=0x55e592992880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803400640 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592992880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.228+0000 7f6803400640 1 -- 172.21.15.71:0/3034254571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 msgr2=0x55e59298f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:04.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f6803400640 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f6803400640 1 -- 172.21.15.71:0/3034254571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59297c3e0 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f6803e00640 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f6803400640 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e592992340 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67f40092a0 tx=0x7f67f4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7193fab5912a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f67ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f67ecc00640 1 -- 172.21.15.71:0/3034254571 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f406c020 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f67ecc00640 1 -- 172.21.15.71:0/3034254571 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9526+0+0 (secure 0 0 0) 0x7f67f4056040 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67f4004990 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f67ecc00640 1 -- 172.21.15.71:0/3034254571 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f67f4035040 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f680d034740 10 client.26498 fetch_fsmap learned FSMap version 41 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.229+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55e592a05140 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f67ecc00640 1 -- 172.21.15.71:0/3034254571 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67f4004990 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f67ecc00640 1 -- 172.21.15.71:0/3034254571 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f67f40687b0 con 0x55e59297f1e0 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f680d034740 10 client.26498 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:37:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f680d034740 10 client.26498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f680d034740 20 client.26498 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f680d034740 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592a056a0 0x55e592a07ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f680d034740 4 client.26498 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5928ecb80 con 0x55e592a056a0 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.230+0000 7f6802a00640 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592a056a0 0x55e592a07ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.231+0000 7f6802a00640 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592a056a0 0x55e592a07ac0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86116022a539c689 server_cookie=1f39f2a1ade96a83 in_seq=0 out_seq=0 2024-09-05T12:37:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.231+0000 7f67ecc00640 10 client.26498 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f67ecc00640 1 -- 172.21.15.71:0/3034254571 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5928ecb80 con 0x55e592a056a0 2024-09-05T12:37:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f67ecc00640 10 client.26498 handle_command_reply: tid=0 2024-09-05T12:37:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 client.26498 shutdown 2024-09-05T12:37:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 msgr2=0x55e592992340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e592992340 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67f40092a0 tx=0x7f67f4003820 comp rx=0 tx=0).stop 2024-09-05T12:37:04.252 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 shutdown_connections 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e592a056a0 0x55e592a07ac0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e59298a3a0 0x55e59298f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e59297fb00 0x55e592992880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 --2- 172.21.15.71:0/3034254571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59297f1e0 0x55e592992340 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f67ecc00640 0 client.26498 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 >> 172.21.15.71:0/3034254571 conn(0x55e59297b750 msgr2=0x55e59297a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 shutdown_connections 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 1 -- 172.21.15.71:0/3034254571 wait complete. 2024-09-05T12:37:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.250+0000 7f680d034740 20 client.26498 trim_cache size 0 max 16384 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67e80a2e20 msgr2=0x7f67e80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67e80a2e20 0x7f67e80a52e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f67f4005c40 tx=0x7f67f402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 msgr2=0x7f680419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 0x7f680419ab00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f67f000e9a0 tx=0x7f67f000ee60 comp rx=0 tx=0).stop 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 shutdown_connections 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f680419ee80 0x7f680419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68041090a0 0x7f680419ab00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67e80a2e20 0x7f67e80a52e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 --2- 172.21.15.71:0/2590215416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6804103f50 0x7f680419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 >> 172.21.15.71:0/2590215416 conn(0x7f68040ff9e0 msgr2=0x7f6804100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 shutdown_connections 2024-09-05T12:37:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:04.251+0000 7f680a000640 1 -- 172.21.15.71:0/2590215416 wait complete. 2024-09-05T12:37:04.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 309164 3.19 MB/sec execute 360 sec latency 10.815 ms 2024-09-05T12:37:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:04 smithi123 ceph-mon[34321]: pgmap v289: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 415 KiB/s wr, 83 op/s 2024-09-05T12:37:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:04 smithi174 ceph-mon[34313]: pgmap v289: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 415 KiB/s wr, 83 op/s 2024-09-05T12:37:04.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:04 smithi071 ceph-mon[29845]: pgmap v289: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 415 KiB/s wr, 83 op/s 2024-09-05T12:37:05.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 310305 3.20 MB/sec execute 361 sec latency 55.140 ms 2024-09-05T12:37:06.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 310305 3.19 MB/sec execute 362 sec latency 1055.279 ms 2024-09-05T12:37:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:06 smithi123 ceph-mon[34321]: pgmap v290: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 415 KiB/s wr, 83 op/s 2024-09-05T12:37:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:06 smithi174 ceph-mon[34313]: pgmap v290: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 415 KiB/s wr, 83 op/s 2024-09-05T12:37:06.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:06 smithi071 ceph-mon[29845]: pgmap v290: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 415 KiB/s wr, 83 op/s 2024-09-05T12:37:07.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 311821 3.20 MB/sec execute 363 sec latency 1184.608 ms 2024-09-05T12:37:08.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 313069 3.21 MB/sec execute 364 sec latency 13.066 ms 2024-09-05T12:37:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:08 smithi123 ceph-mon[34321]: pgmap v291: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 944 KiB/s wr, 179 op/s 2024-09-05T12:37:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:08 smithi174 ceph-mon[34313]: pgmap v291: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 944 KiB/s wr, 179 op/s 2024-09-05T12:37:08.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:08 smithi071 ceph-mon[29845]: pgmap v291: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 944 KiB/s wr, 179 op/s 2024-09-05T12:37:09.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 313847 3.21 MB/sec execute 365 sec latency 353.742 ms 2024-09-05T12:37:10.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 313847 3.21 MB/sec execute 366 sec latency 1353.807 ms 2024-09-05T12:37:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:10 smithi123 ceph-mon[34321]: pgmap v292: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 944 KiB/s wr, 179 op/s 2024-09-05T12:37:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:10 smithi123 ceph-mon[34321]: osdmap e103: 12 total, 12 up, 12 in 2024-09-05T12:37:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:10 smithi174 ceph-mon[34313]: pgmap v292: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 944 KiB/s wr, 179 op/s 2024-09-05T12:37:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:10 smithi174 ceph-mon[34313]: osdmap e103: 12 total, 12 up, 12 in 2024-09-05T12:37:10.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:10 smithi071 ceph-mon[29845]: pgmap v292: 129 pgs: 129 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 944 KiB/s wr, 179 op/s 2024-09-05T12:37:10.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:10 smithi071 ceph-mon[29845]: osdmap e103: 12 total, 12 up, 12 in 2024-09-05T12:37:11.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 313847 3.20 MB/sec execute 367 sec latency 2353.926 ms 2024-09-05T12:37:12.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 315070 3.21 MB/sec execute 368 sec latency 2481.109 ms 2024-09-05T12:37:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:12 smithi123 ceph-mon[34321]: pgmap v294: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-09-05T12:37:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:12 smithi174 ceph-mon[34313]: pgmap v294: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-09-05T12:37:12.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:12 smithi071 ceph-mon[29845]: pgmap v294: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-09-05T12:37:13.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 316371 3.22 MB/sec execute 369 sec latency 14.030 ms 2024-09-05T12:37:14.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 317396 3.22 MB/sec execute 370 sec latency 308.172 ms 2024-09-05T12:37:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:14 smithi174 ceph-mon[34313]: pgmap v295: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 962 KiB/s wr, 184 op/s 2024-09-05T12:37:14.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:14 smithi071 ceph-mon[29845]: pgmap v295: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 962 KiB/s wr, 184 op/s 2024-09-05T12:37:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:14 smithi123 ceph-mon[34321]: pgmap v295: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 962 KiB/s wr, 184 op/s 2024-09-05T12:37:15.254 INFO:tasks.workunit.client.0.smithi071.stdout: 1 317396 3.21 MB/sec execute 371 sec latency 1308.270 ms 2024-09-05T12:37:16.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 317396 3.21 MB/sec execute 372 sec latency 2308.344 ms 2024-09-05T12:37:16.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:16 smithi071 ceph-mon[29845]: pgmap v296: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 962 KiB/s wr, 184 op/s 2024-09-05T12:37:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:16 smithi174 ceph-mon[34313]: pgmap v296: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 962 KiB/s wr, 184 op/s 2024-09-05T12:37:16.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:16 smithi123 ceph-mon[34321]: pgmap v296: 129 pgs: 129 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 962 KiB/s wr, 184 op/s 2024-09-05T12:37:17.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 319029 3.22 MB/sec execute 373 sec latency 2443.082 ms 2024-09-05T12:37:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.381+0000 7f045be00640 1 -- 172.21.15.71:0/735165780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c109ba0 msgr2=0x7f045c109f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.381+0000 7f045be00640 1 --2- 172.21.15.71:0/735165780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c109ba0 0x7f045c109f80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f044c009900 tx=0x7f044c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 -- 172.21.15.71:0/735165780 shutdown_connections 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 --2- 172.21.15.71:0/735165780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f045c1044a0 0x7f045c1109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 --2- 172.21.15.71:0/735165780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c103b00 0x7f045c103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 --2- 172.21.15.71:0/735165780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c109ba0 0x7f045c109f80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 -- 172.21.15.71:0/735165780 >> 172.21.15.71:0/735165780 conn(0x7f045c0ff7c0 msgr2=0x7f045c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 -- 172.21.15.71:0/735165780 shutdown_connections 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 -- 172.21.15.71:0/735165780 wait complete. 2024-09-05T12:37:17.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.382+0000 7f045be00640 1 Processor -- start 2024-09-05T12:37:17.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 -- start start 2024-09-05T12:37:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 0x7f045c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c1044a0 0x7f045c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f045c109ba0 0x7f045c19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f045c113f50 con 0x7f045c109ba0 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f045c113dd0 con 0x7f045c103b00 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045be00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1140d0 con 0x7f045c1044a0 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 0x7f045c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c1044a0 0x7f045c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 0x7f045c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53646/0 (socket says 172.21.15.71:53646) 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045aa00640 1 -- 172.21.15.71:0/2809032292 learned_addr learned my addr 172.21.15.71:0/2809032292 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.383+0000 7f045b400640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f045c109ba0 0x7f045c19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045aa00640 1 -- 172.21.15.71:0/2809032292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c1044a0 msgr2=0x7f045c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045aa00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c1044a0 0x7f045c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045aa00640 1 -- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f045c109ba0 msgr2=0x7f045c19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045aa00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f045c109ba0 0x7f045c19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045aa00640 1 -- 172.21.15.71:0/2809032292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c19f430 con 0x7f045c103b00 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045a000640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c1044a0 0x7f045c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045aa00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 0x7f045c19a440 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f044c006010 tx=0x7f044c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f0453400640 1 -- 172.21.15.71:0/2809032292 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c03f070 con 0x7f045c103b00 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f0453400640 1 -- 172.21.15.71:0/2809032292 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f044c031c30 con 0x7f045c103b00 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f0453400640 1 -- 172.21.15.71:0/2809032292 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c035030 con 0x7f045c103b00 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c19f6c0 con 0x7f045c103b00 2024-09-05T12:37:17.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.384+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1a6fb0 con 0x7f045c103b00 2024-09-05T12:37:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.388+0000 7f0453400640 1 -- 172.21.15.71:0/2809032292 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f044c042070 con 0x7f045c103b00 2024-09-05T12:37:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.388+0000 7f04646d7740 1 Processor -- start 2024-09-05T12:37:17.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.388+0000 7f04646d7740 1 -- start start 2024-09-05T12:37:17.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f04646d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 0x55d50ca21790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f04646d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f04646d7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca2c570 0x55d50ca2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f04646d7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d50c987100 con 0x55d50ca2c570 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f04646d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d50c92f1b0 con 0x55d50ca213b0 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f04646d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d50c9830c0 con 0x55d50ca21cd0 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f045b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 0x55d50ca21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f045b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 0x55d50ca21790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53654/0 (socket says 172.21.15.71:53654) 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f045b400640 1 -- 172.21.15.71:0/2516731140 learned_addr learned my addr 172.21.15.71:0/2516731140 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f045aa00640 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f045a000640 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca2c570 0x55d50ca2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f0453400640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04400a2d70 0x7f04400a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.389+0000 7f0453400640 1 -- 172.21.15.71:0/2809032292 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f044c0efdf0 con 0x7f045c103b00 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 -- 172.21.15.71:0/2516731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 msgr2=0x55d50ca2c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca2c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 -- 172.21.15.71:0/2516731140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca2c570 msgr2=0x55d50ca2e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca2c570 0x55d50ca2e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 -- 172.21.15.71:0/2516731140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50c69a210 con 0x55d50ca213b0 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045aa00640 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04400a2d70 0x7f04400a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f045b400640 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 0x55d50ca21790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0454011590 tx=0x7f0454011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d405487a016cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f0447e00640 1 -- 172.21.15.71:0/2516731140 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454017050 con 0x55d50ca213b0 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f0447e00640 1 -- 172.21.15.71:0/2516731140 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045401bb90 con 0x55d50ca213b0 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.390+0000 7f0447e00640 1 -- 172.21.15.71:0/2516731140 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400e3b0 con 0x55d50ca213b0 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 -- 172.21.15.71:0/2516731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 msgr2=0x55d50ca21790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f045b400640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04400a2d70 0x7f04400a5230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0454022000 tx=0x7f0454023000 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 0x55d50ca21790 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0454011590 tx=0x7f0454011a50 comp rx=0 tx=0).stop 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 -- 172.21.15.71:0/2516731140 shutdown_connections 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca2c570 0x55d50ca2e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 --2- 172.21.15.71:0/2516731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca213b0 0x55d50ca21790 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 -- 172.21.15.71:0/2516731140 >> 172.21.15.71:0/2516731140 conn(0x55d50ca1d920 msgr2=0x55d50ca1dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 -- 172.21.15.71:0/2516731140 shutdown_connections 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.391+0000 7f04646d7740 1 -- 172.21.15.71:0/2516731140 wait complete. 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.392+0000 7f04646d7740 1 -- start start 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.392+0000 7f04646d7740 1 Processor -- start 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 0x55d50ca295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca21cd0 0x55d50ca29b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d50ca2eff0 con 0x55d50ca21cd0 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d50c910090 con 0x55d50ca2c570 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d50ca2f170 con 0x55d50ca213b0 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f04646d7740 10 client.0 _ll_register_callbacks cb 0x55d50c964f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 0x55d50ca295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 0x55d50ca295c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55806/0 (socket says 172.21.15.71:55806) 2024-09-05T12:37:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 -- 172.21.15.71:0/729068060 learned_addr learned my addr 172.21.15.71:0/729068060 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045a000640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca21cd0 0x55d50ca29b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045b400640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 -- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 msgr2=0x55d50ca2a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca2a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 -- 172.21.15.71:0/729068060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca21cd0 msgr2=0x55d50ca29b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca21cd0 0x55d50ca29b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045aa00640 1 -- 172.21.15.71:0/729068060 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50c766620 con 0x55d50ca213b0 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.393+0000 7f045a000640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca21cd0 0x55d50ca29b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f045aa00640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 0x55d50ca295c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f044c0bbf60 tx=0x7f044c0dc110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca5459fe26725d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f0444c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f0444c00640 1 -- 172.21.15.71:0/729068060 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c049070 con 0x55d50ca213b0 2024-09-05T12:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045401bb90 con 0x55d50ca213b0 2024-09-05T12:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f0444c00640 1 -- 172.21.15.71:0/729068060 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f044c0b8640 con 0x55d50ca213b0 2024-09-05T12:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f0444c00640 1 -- 172.21.15.71:0/729068060 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f044c031de0 con 0x55d50ca213b0 2024-09-05T12:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f04646d7740 10 client.26533 fetch_fsmap learned FSMap version 41 2024-09-05T12:37:17.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d50ca2f040 con 0x55d50ca213b0 2024-09-05T12:37:17.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f0444c00640 1 -- 172.21.15.71:0/729068060 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045401bb90 con 0x55d50ca213b0 2024-09-05T12:37:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.394+0000 7f0444c00640 1 -- 172.21.15.71:0/729068060 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f044c0ed3d0 con 0x55d50ca213b0 2024-09-05T12:37:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f04646d7740 10 client.26533 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:37:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f04646d7740 10 client.26533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:17.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f04646d7740 20 client.26533 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f04646d7740 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50ca3c330 0x55d50ca3e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f04646d7740 4 client.26533 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:37:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50c994830 con 0x55d50ca3c330 2024-09-05T12:37:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f045a000640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50ca3c330 0x55d50ca3e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.404 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f045a000640 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50ca3c330 0x55d50ca3e770 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b17146c041f783f server_cookie=20e59ebb029a3819 in_seq=0 out_seq=0 2024-09-05T12:37:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.395+0000 7f0444c00640 10 client.26533 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.396+0000 7f0444c00640 1 -- 172.21.15.71:0/729068060 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d50c994830 con 0x55d50ca3c330 2024-09-05T12:37:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.396+0000 7f0444c00640 10 client.26533 handle_command_reply: tid=0 2024-09-05T12:37:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 client.26533 shutdown 2024-09-05T12:37:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 msgr2=0x55d50ca295c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 0x55d50ca295c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f044c0bbf60 tx=0x7f044c0dc110 comp rx=0 tx=0).stop 2024-09-05T12:37:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 shutdown_connections 2024-09-05T12:37:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50ca3c330 0x55d50ca3e770 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca2a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca21cd0 0x55d50ca29b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 --2- 172.21.15.71:0/729068060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca213b0 0x55d50ca295c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f0444c00640 0 client.26533 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 >> 172.21.15.71:0/729068060 conn(0x55d50ca1d920 msgr2=0x55d50ca23630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 shutdown_connections 2024-09-05T12:37:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 1 -- 172.21.15.71:0/729068060 wait complete. 2024-09-05T12:37:17.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.397+0000 7f04646d7740 20 client.26533 trim_cache size 0 max 16384 2024-09-05T12:37:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.416+0000 7f04646d7740 1 Processor -- start 2024-09-05T12:37:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 -- start start 2024-09-05T12:37:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca29da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca44080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b0fa0 con 0x55d50ca213b0 2024-09-05T12:37:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b2ba0 con 0x55d50ca2c570 2024-09-05T12:37:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f04646d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b1670 con 0x55d50ca21cd0 2024-09-05T12:37:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca44080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca44080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55814/0 (socket says 172.21.15.71:55814) 2024-09-05T12:37:17.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045a000640 1 -- 172.21.15.71:0/2760967226 learned_addr learned my addr 172.21.15.71:0/2760967226 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045aa00640 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca29da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045a000640 1 -- 172.21.15.71:0/2760967226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 msgr2=0x55d50ca445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045a000640 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.417+0000 7f045a000640 1 -- 172.21.15.71:0/2760967226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 msgr2=0x55d50ca29da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.418+0000 7f045a000640 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca29da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.418+0000 7f045a000640 1 -- 172.21.15.71:0/2760967226 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50ca28550 con 0x55d50ca21cd0 2024-09-05T12:37:17.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.418+0000 7f045a000640 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca44080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55d50ca1eca0 tx=0x7f0448005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac64fe40034efc86 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.418+0000 7f0447400640 1 -- 172.21.15.71:0/2760967226 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448004220 con 0x55d50ca21cd0 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.418+0000 7f0447400640 1 -- 172.21.15.71:0/2760967226 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04480043c0 con 0x55d50ca21cd0 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.418+0000 7f0447400640 1 -- 172.21.15.71:0/2760967226 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0448004a00 con 0x55d50ca21cd0 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 -- 172.21.15.71:0/2760967226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 msgr2=0x55d50ca44080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca44080 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55d50ca1eca0 tx=0x7f0448005f90 comp rx=0 tx=0).stop 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 -- 172.21.15.71:0/2760967226 shutdown_connections 2024-09-05T12:37:17.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50ca445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca44080 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 --2- 172.21.15.71:0/2760967226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca29da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 -- 172.21.15.71:0/2760967226 >> 172.21.15.71:0/2760967226 conn(0x55d50ca1d920 msgr2=0x55d50ca186a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 -- 172.21.15.71:0/2760967226 shutdown_connections 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 -- 172.21.15.71:0/2760967226 wait complete. 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.419+0000 7f04646d7740 1 -- start start 2024-09-05T12:37:17.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 Processor -- start 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca30be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50caa6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b2410 con 0x55d50ca213b0 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b1d40 con 0x55d50ca2c570 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1aefa0 con 0x55d50ca21cd0 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f04646d7740 10 client.0 _ll_register_callbacks cb 0x7f044c005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:17.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f045a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca30be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f045aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f045b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50caa6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f045a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca30be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55822/0 (socket says 172.21.15.71:55822) 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.420+0000 7f045a000640 1 -- 172.21.15.71:0/653679550 learned_addr learned my addr 172.21.15.71:0/653679550 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f045a000640 1 -- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 msgr2=0x55d50caa6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f045a000640 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50caa6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f045a000640 1 -- 172.21.15.71:0/653679550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 msgr2=0x55d50ca306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:17.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f045a000640 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f045a000640 1 -- 172.21.15.71:0/653679550 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50caa6590 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f045a000640 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca30be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0448010580 tx=0x7f0448005300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ac3e020b85172b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f0444c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f0444c00640 1 -- 172.21.15.71:0/653679550 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044800fab0 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04480043c0 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f0444c00640 1 -- 172.21.15.71:0/653679550 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9438+0+0 (secure 0 0 0) 0x7f044800fc50 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f0444c00640 1 -- 172.21.15.71:0/653679550 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0448038050 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f04646d7740 10 client.26539 fetch_fsmap learned FSMap version 41 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.421+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55d50caa7800 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f0444c00640 1 -- 172.21.15.71:0/653679550 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04480043c0 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f0444c00640 1 -- 172.21.15.71:0/653679550 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 41) ==== 8390+0+0 (secure 0 0 0) 0x7f0448025030 con 0x55d50ca21cd0 2024-09-05T12:37:17.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f04646d7740 10 client.26539 fetch_fsmap finished waiting for FSMap version 41 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f04646d7740 10 client.26539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f04646d7740 20 client.26539 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f04646d7740 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50caa8020 0x55d50caaa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f04646d7740 4 client.26539 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d50c98eb80 con 0x55d50caa8020 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.422+0000 7f045aa00640 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50caa8020 0x55d50caaa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.423+0000 7f045aa00640 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50caa8020 0x55d50caaa440 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48db8a3467302202 server_cookie=3c95c2daa60125c0 in_seq=0 out_seq=0 2024-09-05T12:37:17.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.423+0000 7f0444c00640 10 client.26539 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.447+0000 7f0444c00640 1 -- 172.21.15.71:0/653679550 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d50c98eb80 con 0x55d50caa8020 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.447+0000 7f0444c00640 10 client.26539 handle_command_reply: tid=0 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.447+0000 7f04646d7740 1 client.26539 shutdown 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.447+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 msgr2=0x55d50ca30be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.447+0000 7f04646d7740 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca30be0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0448010580 tx=0x7f0448005300 comp rx=0 tx=0).stop 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 shutdown_connections 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d50caa8020 0x55d50caaa440 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d50ca2c570 0x55d50caa6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d50ca21cd0 0x55d50ca30be0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 --2- 172.21.15.71:0/653679550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50ca213b0 0x55d50ca306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 >> 172.21.15.71:0/653679550 conn(0x55d50ca1d920 msgr2=0x55d50ca1c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f0444c00640 0 client.26539 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 shutdown_connections 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 1 -- 172.21.15.71:0/653679550 wait complete. 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.448+0000 7f04646d7740 20 client.26539 trim_cache size 0 max 16384 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:37:17.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:37:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04400a2d70 msgr2=0x7f04400a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04400a2d70 0x7f04400a5230 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0454022000 tx=0x7f0454023000 comp rx=0 tx=0).stop 2024-09-05T12:37:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 msgr2=0x7f045c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 0x7f045c19a440 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f044c006010 tx=0x7f044c003c10 comp rx=0 tx=0).stop 2024-09-05T12:37:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 shutdown_connections 2024-09-05T12:37:17.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04400a2d70 0x7f04400a5230 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f045c109ba0 0x7f045c19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f045c1044a0 0x7f045c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 --2- 172.21.15.71:0/2809032292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c103b00 0x7f045c19a440 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.449+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 >> 172.21.15.71:0/2809032292 conn(0x7f045c0ff7c0 msgr2=0x7f045c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.450+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 shutdown_connections 2024-09-05T12:37:17.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:17.450+0000 7f045be00640 1 -- 172.21.15.71:0/2809032292 wait complete. 2024-09-05T12:37:18.010 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:37:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: ignoring --setuser ceph since I am not root 2024-09-05T12:37:18.010 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:37:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: ignoring --setgroup ceph since I am not root 2024-09-05T12:37:18.010 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:37:17 smithi174 ceph-mds[63797]: -- 172.21.15.174:0/105610239 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563eb9951860 con 0x563eb992f000 2024-09-05T12:37:18.010 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:37:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: starting mds.f at 2024-09-05T12:37:18.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 320149 3.23 MB/sec execute 374 sec latency 10.657 ms 2024-09-05T12:37:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:18 smithi174 ceph-mon[34313]: pgmap v297: 129 pgs: 129 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 772 KiB/s wr, 149 op/s 2024-09-05T12:37:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:18 smithi174 ceph-mon[34313]: daemon mds.f restarted 2024-09-05T12:37:18.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:18 smithi071 ceph-mon[29845]: pgmap v297: 129 pgs: 129 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 772 KiB/s wr, 149 op/s 2024-09-05T12:37:18.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:18 smithi071 ceph-mon[29845]: daemon mds.f restarted 2024-09-05T12:37:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:18 smithi123 ceph-mon[34321]: pgmap v297: 129 pgs: 129 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 772 KiB/s wr, 149 op/s 2024-09-05T12:37:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:18 smithi123 ceph-mon[34321]: daemon mds.f restarted 2024-09-05T12:37:19.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 320938 3.23 MB/sec execute 375 sec latency 492.492 ms 2024-09-05T12:37:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:19 smithi174 ceph-mon[34313]: osdmap e104: 12 total, 12 up, 12 in 2024-09-05T12:37:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:19 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6834/2481371871,v1:172.21.15.174:6835/2481371871] up:boot 2024-09-05T12:37:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:19 smithi174 ceph-mon[34313]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-05T12:37:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:19 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:37:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:19 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:19.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:19 smithi071 ceph-mon[29845]: osdmap e104: 12 total, 12 up, 12 in 2024-09-05T12:37:19.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:19 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6834/2481371871,v1:172.21.15.174:6835/2481371871] up:boot 2024-09-05T12:37:19.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:19 smithi071 ceph-mon[29845]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-05T12:37:19.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:19 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:37:19.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:19 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:19.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:19 smithi123 ceph-mon[34321]: osdmap e104: 12 total, 12 up, 12 in 2024-09-05T12:37:19.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:19 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6834/2481371871,v1:172.21.15.174:6835/2481371871] up:boot 2024-09-05T12:37:19.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:19 smithi123 ceph-mon[34321]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-05T12:37:19.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:19 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:37:19.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:19 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:20.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 320938 3.22 MB/sec execute 376 sec latency 1492.628 ms 2024-09-05T12:37:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:20 smithi174 ceph-mon[34313]: pgmap v299: 129 pgs: 129 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 908 KiB/s wr, 175 op/s 2024-09-05T12:37:20.812 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:20 smithi071 ceph-mon[29845]: pgmap v299: 129 pgs: 129 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 908 KiB/s wr, 175 op/s 2024-09-05T12:37:20.831 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:20 smithi123 ceph-mon[34321]: pgmap v299: 129 pgs: 129 active+clean; 105 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 908 KiB/s wr, 175 op/s 2024-09-05T12:37:21.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 320938 3.21 MB/sec execute 377 sec latency 2492.770 ms 2024-09-05T12:37:22.255 INFO:tasks.workunit.client.0.smithi071.stdout: 1 322259 3.22 MB/sec execute 378 sec latency 2621.497 ms 2024-09-05T12:37:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:22 smithi174 ceph-mon[34313]: pgmap v300: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 829 KiB/s wr, 156 op/s 2024-09-05T12:37:22.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:22 smithi071 ceph-mon[29845]: pgmap v300: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 829 KiB/s wr, 156 op/s 2024-09-05T12:37:22.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:22 smithi123 ceph-mon[34321]: pgmap v300: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 829 KiB/s wr, 156 op/s 2024-09-05T12:37:23.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 323527 3.23 MB/sec execute 379 sec latency 12.100 ms 2024-09-05T12:37:24.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 324835 3.24 MB/sec execute 380 sec latency 250.546 ms 2024-09-05T12:37:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:24 smithi174 ceph-mon[34313]: pgmap v301: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 803 KiB/s wr, 157 op/s 2024-09-05T12:37:24.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:24 smithi071 ceph-mon[29845]: pgmap v301: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 803 KiB/s wr, 157 op/s 2024-09-05T12:37:24.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:24 smithi123 ceph-mon[34321]: pgmap v301: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 803 KiB/s wr, 157 op/s 2024-09-05T12:37:25.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 326111 3.25 MB/sec execute 381 sec latency 272.168 ms 2024-09-05T12:37:26.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 327250 3.26 MB/sec execute 382 sec latency 19.408 ms 2024-09-05T12:37:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:26 smithi174 ceph-mon[34313]: pgmap v302: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 803 KiB/s wr, 157 op/s 2024-09-05T12:37:26.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:26 smithi071 ceph-mon[29845]: pgmap v302: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 803 KiB/s wr, 157 op/s 2024-09-05T12:37:26.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:26 smithi123 ceph-mon[34321]: pgmap v302: 129 pgs: 129 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 803 KiB/s wr, 157 op/s 2024-09-05T12:37:27.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 328029 3.26 MB/sec execute 383 sec latency 349.669 ms 2024-09-05T12:37:28.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 328029 3.25 MB/sec execute 384 sec latency 1349.764 ms 2024-09-05T12:37:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:28 smithi174 ceph-mon[34313]: pgmap v303: 129 pgs: 129 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 969 KiB/s wr, 205 op/s 2024-09-05T12:37:28.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:28 smithi071 ceph-mon[29845]: pgmap v303: 129 pgs: 129 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 969 KiB/s wr, 205 op/s 2024-09-05T12:37:28.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:28 smithi123 ceph-mon[34321]: pgmap v303: 129 pgs: 129 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 969 KiB/s wr, 205 op/s 2024-09-05T12:37:29.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 328029 3.25 MB/sec execute 385 sec latency 2349.826 ms 2024-09-05T12:37:30.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 328029 3.24 MB/sec execute 386 sec latency 3349.963 ms 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.571+0000 7fd43b200640 1 -- 172.21.15.71:0/2453647925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 msgr2=0x7fd43c109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.571+0000 7fd43b200640 1 --2- 172.21.15.71:0/2453647925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 0x7fd43c109f60 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009cd0 tx=0x7fd42c02a8f0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 -- 172.21.15.71:0/2453647925 shutdown_connections 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 --2- 172.21.15.71:0/2453647925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c104480 0x7fd43c1109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 --2- 172.21.15.71:0/2453647925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd43c103ae0 0x7fd43c103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 --2- 172.21.15.71:0/2453647925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 0x7fd43c109f60 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 -- 172.21.15.71:0/2453647925 >> 172.21.15.71:0/2453647925 conn(0x7fd43c0ff7c0 msgr2=0x7fd43c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 -- 172.21.15.71:0/2453647925 shutdown_connections 2024-09-05T12:37:30.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.572+0000 7fd43b200640 1 -- 172.21.15.71:0/2453647925 wait complete. 2024-09-05T12:37:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 Processor -- start 2024-09-05T12:37:30.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 -- start start 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 0x7fd43c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd43c104480 0x7fd43c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 0x7fd43c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c113f30 con 0x7fd43c109b80 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c113db0 con 0x7fd43c103ae0 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1140b0 con 0x7fd43c104480 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd439e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 0x7fd43c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd439e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 0x7fd43c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37282/0 (socket says 172.21.15.71:37282) 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd439e00640 1 -- 172.21.15.71:0/3508048698 learned_addr learned my addr 172.21.15.71:0/3508048698 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd43a800640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 0x7fd43c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.573+0000 7fd439400640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd43c104480 0x7fd43c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd439e00640 1 -- 172.21.15.71:0/3508048698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd43c104480 msgr2=0x7fd43c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd439e00640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd43c104480 0x7fd43c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd439e00640 1 -- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 msgr2=0x7fd43c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd439e00640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 0x7fd43c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd439e00640 1 -- 172.21.15.71:0/3508048698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c19f490 con 0x7fd43c103ae0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd439e00640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 0x7fd43c19a4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009df0 tx=0x7fd42c030ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd432a00640 1 -- 172.21.15.71:0/3508048698 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c003d80 con 0x7fd43c103ae0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c19f720 con 0x7fd43c103ae0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd432a00640 1 -- 172.21.15.71:0/3508048698 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42c040070 con 0x7fd43c103ae0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.574+0000 7fd432a00640 1 -- 172.21.15.71:0/3508048698 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c031a30 con 0x7fd43c103ae0 2024-09-05T12:37:30.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.575+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1a7010 con 0x7fd43c103ae0 2024-09-05T12:37:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.578+0000 7fd443e33740 1 Processor -- start 2024-09-05T12:37:30.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.578+0000 7fd443e33740 1 -- start start 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd443e33740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e703b0 0x564f40e70790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd443e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e70cd0 0x564f40e7b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd443e33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40e7d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd443e33740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564f40dd6100 con 0x564f40e703b0 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd443e33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f40d7e1b0 con 0x564f40e7b570 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd443e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f40dd20c0 con 0x564f40e70cd0 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd432a00640 1 -- 172.21.15.71:0/3508048698 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd42c004690 con 0x7fd43c103ae0 2024-09-05T12:37:30.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd43a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e703b0 0x564f40e70790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd439400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40e7d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd439400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40e7d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37298/0 (socket says 172.21.15.71:37298) 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd439400640 1 -- 172.21.15.71:0/2368691550 learned_addr learned my addr 172.21.15.71:0/2368691550 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.579+0000 7fd439e00640 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e70cd0 0x564f40e7b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd439400640 1 -- 172.21.15.71:0/2368691550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e70cd0 msgr2=0x564f40e7b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd439400640 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e70cd0 0x564f40e7b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd439400640 1 -- 172.21.15.71:0/2368691550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e703b0 msgr2=0x564f40e70790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd439400640 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e703b0 0x564f40e70790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd439400640 1 -- 172.21.15.71:0/2368691550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f40ae9210 con 0x564f40e7b570 2024-09-05T12:37:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd439400640 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40e7d960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd428009ca0 tx=0x7fd428011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783c8115378d144b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd427400640 1 -- 172.21.15.71:0/2368691550 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42801e070 con 0x564f40e7b570 2024-09-05T12:37:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd427400640 1 -- 172.21.15.71:0/2368691550 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd428019b10 con 0x564f40e7b570 2024-09-05T12:37:30.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.580+0000 7fd427400640 1 -- 172.21.15.71:0/2368691550 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42800ea80 con 0x564f40e7b570 2024-09-05T12:37:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd432a00640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd4200a2cf0 0x7fd4200a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd432a00640 1 -- 172.21.15.71:0/3508048698 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7fd42c0f07c0 con 0x7fd43c103ae0 2024-09-05T12:37:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 -- 172.21.15.71:0/2368691550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 msgr2=0x564f40e7d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd43a800640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd4200a2cf0 0x7fd4200a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40e7d960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd428009ca0 tx=0x7fd428011810 comp rx=0 tx=0).stop 2024-09-05T12:37:30.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 -- 172.21.15.71:0/2368691550 shutdown_connections 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40e7d960 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e70cd0 0x564f40e7b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 --2- 172.21.15.71:0/2368691550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e703b0 0x564f40e70790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 -- 172.21.15.71:0/2368691550 >> 172.21.15.71:0/2368691550 conn(0x564f40e6c920 msgr2=0x564f40e6cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.581+0000 7fd443e33740 1 -- 172.21.15.71:0/2368691550 shutdown_connections 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.582+0000 7fd443e33740 1 -- 172.21.15.71:0/2368691550 wait complete. 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.582+0000 7fd43a800640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd4200a2cf0 0x7fd4200a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd434009d20 tx=0x7fd434005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.582+0000 7fd443e33740 1 -- start start 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.582+0000 7fd443e33740 1 Processor -- start 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.582+0000 7fd443e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.582+0000 7fd443e33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e78a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd443e33740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e78f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd443e33740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564f40e7e050 con 0x564f40e7b570 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd443e33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564f40d5f090 con 0x564f40e70cd0 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd443e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564f40e7e1d0 con 0x564f40e703b0 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd443e33740 10 client.0 _ll_register_callbacks cb 0x564f40db3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e78f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e78f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34458/0 (socket says 172.21.15.71:34458) 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 -- 172.21.15.71:0/1088224697 learned_addr learned my addr 172.21.15.71:0/1088224697 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd439e00640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd439400640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e78a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 -- 172.21.15.71:0/1088224697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 msgr2=0x564f40e784e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e784e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 -- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 msgr2=0x564f40e78a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e78a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd43a800640 1 -- 172.21.15.71:0/1088224697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f40bb5620 con 0x564f40e7b570 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.583+0000 7fd439e00640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd43a800640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e78f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd434009d90 tx=0x7fd434004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b98a289354bc69e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd41be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd41be00640 1 -- 172.21.15.71:0/1088224697 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43401f610 con 0x564f40e7b570 2024-09-05T12:37:30.591 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd41be00640 1 -- 172.21.15.71:0/1088224697 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7fd43401f7b0 con 0x564f40e7b570 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd428019b10 con 0x564f40e7b570 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd41be00640 1 -- 172.21.15.71:0/1088224697 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd43401f9b0 con 0x564f40e7b570 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd443e33740 10 client.16497 fetch_fsmap learned FSMap version 43 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564f40e89eb0 con 0x564f40e7b570 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.584+0000 7fd41be00640 1 -- 172.21.15.71:0/1088224697 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd428019b10 con 0x564f40e7b570 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd41be00640 1 -- 172.21.15.71:0/1088224697 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) ==== 8390+0+0 (secure 0 0 0) 0x7fd434027020 con 0x564f40e7b570 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd443e33740 10 client.16497 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd443e33740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd443e33740 20 client.16497 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:30.592 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd443e33740 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40e8b430 0x564f40e8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd443e33740 4 client.16497 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f40de3830 con 0x564f40e8b430 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.585+0000 7fd439e00640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40e8b430 0x564f40e8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.586+0000 7fd41be00640 10 client.16497 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.586+0000 7fd439e00640 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40e8b430 0x564f40e8d880 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93af529867f44c9d server_cookie=b2e93e88926eb1aa in_seq=0 out_seq=0 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.586+0000 7fd41be00640 1 -- 172.21.15.71:0/1088224697 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f40de3830 con 0x564f40e8b430 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.586+0000 7fd41be00640 10 client.16497 handle_command_reply: tid=0 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.586+0000 7fd443e33740 1 client.16497 shutdown 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 msgr2=0x564f40e78f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e78f60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd434009d90 tx=0x7fd434004730 comp rx=0 tx=0).stop 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 shutdown_connections 2024-09-05T12:37:30.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40e8b430 0x564f40e8d880 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e78f60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e78a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 --2- 172.21.15.71:0/1088224697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd41be00640 0 client.16497 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 >> 172.21.15.71:0/1088224697 conn(0x564f40e6c920 msgr2=0x564f40e72680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 shutdown_connections 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 1 -- 172.21.15.71:0/1088224697 wait complete. 2024-09-05T12:37:30.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.587+0000 7fd443e33740 20 client.16497 trim_cache size 0 max 16384 2024-09-05T12:37:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.604+0000 7fd443e33740 1 Processor -- start 2024-09-05T12:37:30.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.604+0000 7fd443e33740 1 -- start start 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd443e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e72e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd443e33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e7a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd443e33740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e93190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd443e33740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1b0e90 con 0x564f40e7b570 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd443e33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1b2ac0 con 0x564f40e70cd0 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd443e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1b1590 con 0x564f40e703b0 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd43a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e93190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd439400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e7a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd439e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e72e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd439400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e7a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:37312/0 (socket says 172.21.15.71:37312) 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd43a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e93190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34466/0 (socket says 172.21.15.71:34466) 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd439400640 1 -- 172.21.15.71:0/2236973774 learned_addr learned my addr 172.21.15.71:0/2236973774 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.605+0000 7fd439400640 1 -- 172.21.15.71:0/2236973774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 msgr2=0x564f40e72e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd439400640 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e72e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd439400640 1 -- 172.21.15.71:0/2236973774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 msgr2=0x564f40e93190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd439400640 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e93190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd439400640 1 -- 172.21.15.71:0/2236973774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f40e66c20 con 0x564f40e70cd0 2024-09-05T12:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd43a800640 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e93190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd439e00640 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e72e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd439400640 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e7a340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd428009ca0 tx=0x7fd42800e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efe474553f9ee81 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd426a00640 1 -- 172.21.15.71:0/2236973774 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42801e040 con 0x564f40e70cd0 2024-09-05T12:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd426a00640 1 -- 172.21.15.71:0/2236973774 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42801f040 con 0x564f40e70cd0 2024-09-05T12:37:30.609 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.606+0000 7fd426a00640 1 -- 172.21.15.71:0/2236973774 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428019690 con 0x564f40e70cd0 2024-09-05T12:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 -- 172.21.15.71:0/2236973774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 msgr2=0x564f40e7a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e7a340 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd428009ca0 tx=0x7fd42800e640 comp rx=0 tx=0).stop 2024-09-05T12:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 -- 172.21.15.71:0/2236973774 shutdown_connections 2024-09-05T12:37:30.612 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e7b570 0x564f40e93190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e70cd0 0x564f40e7a340 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 --2- 172.21.15.71:0/2236973774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e72e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 -- 172.21.15.71:0/2236973774 >> 172.21.15.71:0/2236973774 conn(0x564f40e6c920 msgr2=0x564f40e79c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 -- 172.21.15.71:0/2236973774 shutdown_connections 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 -- 172.21.15.71:0/2236973774 wait complete. 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.607+0000 7fd443e33740 1 -- start start 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 Processor -- start 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e7ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 0x564f40e7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40eee120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.613 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1b23f0 con 0x564f40e70cd0 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1b1c60 con 0x564f40e7b570 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c1aee70 con 0x564f40e703b0 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.608+0000 7fd443e33740 10 client.0 _ll_register_callbacks cb 0x7fd434009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 0x564f40e7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 0x564f40e7f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34472/0 (socket says 172.21.15.71:34472) 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 -- 172.21.15.71:0/2100869255 learned_addr learned my addr 172.21.15.71:0/2100869255 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439e00640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e7ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd43a800640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40eee120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 -- 172.21.15.71:0/2100869255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 msgr2=0x564f40e7ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e7ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 -- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 msgr2=0x564f40eee120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:30.614 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40eee120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 -- 172.21.15.71:0/2100869255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f40eee7c0 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd439400640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 0x564f40e7f240 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd4280054b0 tx=0x7fd4280099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecbb740c5a9ef17 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd41be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd41be00640 1 -- 172.21.15.71:0/2100869255 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428033070 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd41be00640 1 -- 172.21.15.71:0/2100869255 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7fd428019840 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.609+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd42801f040 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd41be00640 1 -- 172.21.15.71:0/2100869255 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd42803f050 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 10 client.16503 fetch_fsmap learned FSMap version 43 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564f40ef5be0 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd41be00640 1 -- 172.21.15.71:0/2100869255 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd42801f040 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd41be00640 1 -- 172.21.15.71:0/2100869255 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) ==== 8390+0+0 (secure 0 0 0) 0x7fd42800ed20 con 0x564f40e70cd0 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 10 client.16503 fetch_fsmap finished waiting for FSMap version 43 2024-09-05T12:37:30.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 20 client.16503 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40ef6150 0x564f40ef8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 4 client.16503 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.610+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f40dddb80 con 0x564f40ef6150 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.611+0000 7fd439e00640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40ef6150 0x564f40ef8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.611+0000 7fd439e00640 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40ef6150 0x564f40ef8570 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d73a3af90327bde server_cookie=cb5b950ca5c1dc32 in_seq=0 out_seq=0 2024-09-05T12:37:30.616 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.611+0000 7fd41be00640 10 client.16503 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.642+0000 7fd41be00640 1 -- 172.21.15.71:0/2100869255 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f40dddb80 con 0x564f40ef6150 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.642+0000 7fd41be00640 10 client.16503 handle_command_reply: tid=0 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.642+0000 7fd443e33740 1 client.16503 shutdown 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 msgr2=0x564f40e7f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 0x564f40e7f240 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd4280054b0 tx=0x7fd4280099b0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 shutdown_connections 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x564f40ef6150 0x564f40ef8570 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564f40e7b570 0x564f40eee120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564f40e70cd0 0x564f40e7f240 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 --2- 172.21.15.71:0/2100869255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564f40e703b0 0x564f40e7ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 >> 172.21.15.71:0/2100869255 conn(0x564f40e6c920 msgr2=0x564f40e78cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd41be00640 0 client.16503 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 shutdown_connections 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 1 -- 172.21.15.71:0/2100869255 wait complete. 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.643+0000 7fd443e33740 20 client.16503 trim_cache size 0 max 16384 2024-09-05T12:37:30.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd4200a2cf0 msgr2=0x7fd4200a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd4200a2cf0 0x7fd4200a51b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd434009d20 tx=0x7fd434005f90 comp rx=0 tx=0).stop 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 msgr2=0x7fd43c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:30.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 0x7fd43c19a4a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009df0 tx=0x7fd42c030ed0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 shutdown_connections 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd4200a2cf0 0x7fd4200a51b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd43c109b80 0x7fd43c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd43c104480 0x7fd43c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 --2- 172.21.15.71:0/3508048698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c103ae0 0x7fd43c19a4a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 >> 172.21.15.71:0/3508048698 conn(0x7fd43c0ff7c0 msgr2=0x7fd43c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 shutdown_connections 2024-09-05T12:37:30.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:30.644+0000 7fd43b200640 1 -- 172.21.15.71:0/3508048698 wait complete. 2024-09-05T12:37:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:30 smithi174 ceph-mon[34313]: pgmap v304: 129 pgs: 129 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 945 KiB/s wr, 200 op/s 2024-09-05T12:37:30.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:30 smithi071 ceph-mon[29845]: pgmap v304: 129 pgs: 129 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 945 KiB/s wr, 200 op/s 2024-09-05T12:37:30.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:30 smithi123 ceph-mon[34321]: pgmap v304: 129 pgs: 129 active+clean; 105 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 945 KiB/s wr, 200 op/s 2024-09-05T12:37:31.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 328029 3.23 MB/sec execute 387 sec latency 4350.050 ms 2024-09-05T12:37:32.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 329850 3.25 MB/sec execute 388 sec latency 4477.931 ms 2024-09-05T12:37:32.726 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:32 smithi071 ceph-mon[29845]: pgmap v305: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-09-05T12:37:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:32 smithi174 ceph-mon[34313]: pgmap v305: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-09-05T12:37:32.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:32 smithi123 ceph-mon[34321]: pgmap v305: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.2 MiB/s wr, 203 op/s 2024-09-05T12:37:33.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 330490 3.25 MB/sec execute 389 sec latency 294.805 ms 2024-09-05T12:37:34.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 331577 3.25 MB/sec execute 390 sec latency 276.326 ms 2024-09-05T12:37:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:34 smithi174 ceph-mon[34313]: pgmap v306: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 894 KiB/s wr, 144 op/s 2024-09-05T12:37:34.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:34 smithi071 ceph-mon[29845]: pgmap v306: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 894 KiB/s wr, 144 op/s 2024-09-05T12:37:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:34 smithi123 ceph-mon[34321]: pgmap v306: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 894 KiB/s wr, 144 op/s 2024-09-05T12:37:35.256 INFO:tasks.workunit.client.0.smithi071.stdout: 1 331577 3.24 MB/sec execute 391 sec latency 1276.426 ms 2024-09-05T12:37:36.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 331577 3.24 MB/sec execute 392 sec latency 2276.546 ms 2024-09-05T12:37:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:36 smithi174 ceph-mon[34313]: pgmap v307: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 883 KiB/s wr, 141 op/s 2024-09-05T12:37:36.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:36 smithi071 ceph-mon[29845]: pgmap v307: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 883 KiB/s wr, 141 op/s 2024-09-05T12:37:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:36 smithi123 ceph-mon[34321]: pgmap v307: 129 pgs: 129 active+clean; 109 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 883 KiB/s wr, 141 op/s 2024-09-05T12:37:37.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 332814 3.25 MB/sec execute 393 sec latency 2404.683 ms 2024-09-05T12:37:38.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 333846 3.26 MB/sec execute 394 sec latency 93.053 ms 2024-09-05T12:37:38.625 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:38 smithi071 ceph-mon[29845]: pgmap v308: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-05T12:37:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:38 smithi174 ceph-mon[34313]: pgmap v308: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-05T12:37:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:38 smithi123 ceph-mon[34321]: pgmap v308: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-05T12:37:39.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 334686 3.25 MB/sec execute 395 sec latency 343.870 ms 2024-09-05T12:37:40.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 335120 3.25 MB/sec execute 396 sec latency 726.694 ms 2024-09-05T12:37:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:40 smithi174 ceph-mon[34313]: pgmap v309: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 853 KiB/s wr, 103 op/s 2024-09-05T12:37:40.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:40 smithi071 ceph-mon[29845]: pgmap v309: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 853 KiB/s wr, 103 op/s 2024-09-05T12:37:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:40 smithi123 ceph-mon[34321]: pgmap v309: 129 pgs: 129 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 853 KiB/s wr, 103 op/s 2024-09-05T12:37:41.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 335120 3.24 MB/sec execute 397 sec latency 1726.809 ms 2024-09-05T12:37:42.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 336031 3.24 MB/sec execute 398 sec latency 1861.284 ms 2024-09-05T12:37:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:42 smithi174 ceph-mon[34313]: pgmap v310: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-05T12:37:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:42 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:42.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:42 smithi071 ceph-mon[29845]: pgmap v310: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-05T12:37:42.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:42 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:42 smithi123 ceph-mon[34321]: pgmap v310: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-09-05T12:37:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:42 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:43.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 337300 3.26 MB/sec execute 399 sec latency 107.663 ms 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.773+0000 7f502b800640 1 -- 172.21.15.71:0/1895086137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024102480 msgr2=0x7f502410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.773+0000 7f502b800640 1 --2- 172.21.15.71:0/1895086137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024102480 0x7f502410e9b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f501c009c10 tx=0x7f501c02a6f0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.773+0000 7f502b800640 1 -- 172.21.15.71:0/1895086137 shutdown_connections 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.773+0000 7f502b800640 1 --2- 172.21.15.71:0/1895086137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024102480 0x7f502410e9b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.773+0000 7f502b800640 1 --2- 172.21.15.71:0/1895086137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5024101ae0 0x7f5024101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.773+0000 7f502b800640 1 --2- 172.21.15.71:0/1895086137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5024107b80 0x7f5024107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.774+0000 7f502b800640 1 -- 172.21.15.71:0/1895086137 >> 172.21.15.71:0/1895086137 conn(0x7f50240fd7e0 msgr2=0x7f50240ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.774+0000 7f502b800640 1 -- 172.21.15.71:0/1895086137 shutdown_connections 2024-09-05T12:37:43.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.774+0000 7f502b800640 1 -- 172.21.15.71:0/1895086137 wait complete. 2024-09-05T12:37:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.774+0000 7f502b800640 1 Processor -- start 2024-09-05T12:37:43.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 -- start start 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 0x7f5024198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5024102480 0x7f5024198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5024107b80 0x7f502419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5024111ea0 con 0x7f5024101ae0 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5024111d20 con 0x7f5024107b80 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f502b800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5024112020 con 0x7f5024102480 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f5028e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 0x7f5024198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f5028e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 0x7f5024198310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45340/0 (socket says 172.21.15.71:45340) 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f5028e00640 1 -- 172.21.15.71:0/3588415595 learned_addr learned my addr 172.21.15.71:0/3588415595 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f5023e00640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5024102480 0x7f5024198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.775+0000 7f5029800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5024107b80 0x7f502419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5028e00640 1 -- 172.21.15.71:0/3588415595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5024102480 msgr2=0x7f5024198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5028e00640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5024102480 0x7f5024198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5028e00640 1 -- 172.21.15.71:0/3588415595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5024107b80 msgr2=0x7f502419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5028e00640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5024107b80 0x7f502419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5028e00640 1 -- 172.21.15.71:0/3588415595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502419d300 con 0x7f5024101ae0 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5028e00640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 0x7f5024198310 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f501800e9f0 tx=0x7f501800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5021600640 1 -- 172.21.15.71:0/3588415595 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501800cd80 con 0x7f5024101ae0 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5021600640 1 -- 172.21.15.71:0/3588415595 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5018010910 con 0x7f5024101ae0 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502419d5f0 con 0x7f5024101ae0 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.776+0000 7f5021600640 1 -- 172.21.15.71:0/3588415595 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018018990 con 0x7f5024101ae0 2024-09-05T12:37:43.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.777+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50241a4e80 con 0x7f5024101ae0 2024-09-05T12:37:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.780+0000 7f502e65d740 1 Processor -- start 2024-09-05T12:37:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.780+0000 7f5021600640 1 -- 172.21.15.71:0/3588415595 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5018010ac0 con 0x7f5024101ae0 2024-09-05T12:37:43.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.780+0000 7f502e65d740 1 -- start start 2024-09-05T12:37:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f502e65d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d4080 0x55ef7d8d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f502e65d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f502e65d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f502e65d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d83a100 con 0x55ef7d8e37b0 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f502e65d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d7e21b0 con 0x55ef7d8d4080 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f502e65d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d8360c0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f5028e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f5023e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f5028e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33370/0 (socket says 172.21.15.71:33370) 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f5028e00640 1 -- 172.21.15.71:0/2899183729 learned_addr learned my addr 172.21.15.71:0/2899183729 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.781+0000 7f5029800640 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d4080 0x55ef7d8d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5028e00640 1 -- 172.21.15.71:0/2899183729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d4080 msgr2=0x55ef7d8d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5028e00640 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d4080 0x55ef7d8d4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5028e00640 1 -- 172.21.15.71:0/2899183729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 msgr2=0x55ef7d8e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5028e00640 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5028e00640 1 -- 172.21.15.71:0/2899183729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7d54d210 con 0x55ef7d8d49a0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5028e00640 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e3270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50180ccd70 tx=0x7f50180d8170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e2dd39e4c8b0094 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5021600640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f500c0a2da0 0x7f500c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5016000640 1 -- 172.21.15.71:0/2899183729 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50180d9040 con 0x55ef7d8d49a0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5016000640 1 -- 172.21.15.71:0/2899183729 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501809bcb0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.782+0000 7f5016000640 1 -- 172.21.15.71:0/2899183729 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50180b89f0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f5021600640 1 -- 172.21.15.71:0/3588415595 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7f50180cfa80 con 0x7f5024101ae0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f5029800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f500c0a2da0 0x7f500c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 -- 172.21.15.71:0/2899183729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 msgr2=0x55ef7d8e3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e3270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f50180ccd70 tx=0x7f50180d8170 comp rx=0 tx=0).stop 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 -- 172.21.15.71:0/2899183729 shutdown_connections 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e3270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 --2- 172.21.15.71:0/2899183729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d4080 0x55ef7d8d4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.783+0000 7f502e65d740 1 -- 172.21.15.71:0/2899183729 >> 172.21.15.71:0/2899183729 conn(0x55ef7d8d0750 msgr2=0x55ef7d8d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.784+0000 7f5029800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f500c0a2da0 0x7f500c0a5260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f501c004af0 tx=0x7f501c009280 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.784+0000 7f502e65d740 1 -- 172.21.15.71:0/2899183729 shutdown_connections 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.784+0000 7f502e65d740 1 -- 172.21.15.71:0/2899183729 wait complete. 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.784+0000 7f502e65d740 1 -- start start 2024-09-05T12:37:43.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.784+0000 7f502e65d740 1 Processor -- start 2024-09-05T12:37:43.788 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8e0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d8e63e0 con 0x55ef7d8e37b0 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d7c3090 con 0x55ef7d8d49a0 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef7d8e6560 con 0x55ef7d8d4080 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f502e65d740 10 client.0 _ll_register_callbacks cb 0x55ef7d817f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f5023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f5023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50326/0 (socket says 172.21.15.71:50326) 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f5023e00640 1 -- 172.21.15.71:0/1211090037 learned_addr learned my addr 172.21.15.71:0/1211090037 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f5029800640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.789 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.785+0000 7f5028e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8e0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5023e00640 1 -- 172.21.15.71:0/1211090037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 msgr2=0x55ef7d8e0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5023e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8e0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5023e00640 1 -- 172.21.15.71:0/1211090037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 msgr2=0x55ef7d8e10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5023e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5023e00640 1 -- 172.21.15.71:0/1211090037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7d619620 con 0x55ef7d8d49a0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5028e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5029800640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f5023e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e0ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f50100116d0 tx=0x7f5010011b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c435b05c34b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f500aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f500aa00640 1 -- 172.21.15.71:0/1211090037 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501001e060 con 0x55ef7d8d49a0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f500aa00640 1 -- 172.21.15.71:0/1211090037 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7f501000fbf0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f501809bcb0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.790 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f500aa00640 1 -- 172.21.15.71:0/1211090037 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5010025050 con 0x55ef7d8d49a0 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f502e65d740 10 client.26552 fetch_fsmap learned FSMap version 45 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.786+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ef7d8f2090 con 0x55ef7d8d49a0 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.787+0000 7f500aa00640 1 -- 172.21.15.71:0/1211090037 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501809bcb0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.787+0000 7f500aa00640 1 -- 172.21.15.71:0/1211090037 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) ==== 8378+0+0 (secure 0 0 0) 0x7f5010019070 con 0x55ef7d8d49a0 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f502e65d740 10 client.26552 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f502e65d740 10 client.26552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f502e65d740 20 client.26552 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f502e65d740 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d8f3610 0x55ef7d8f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f502e65d740 4 client.26552 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef7d847830 con 0x55ef7d8f3610 2024-09-05T12:37:43.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f5028e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d8f3610 0x55ef7d8f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f5028e00640 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d8f3610 0x55ef7d8f5a00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa600879e41b9d38 server_cookie=21f41d2dbb7f5dc7 in_seq=0 out_seq=0 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.788+0000 7f500aa00640 10 client.26552 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f500aa00640 1 -- 172.21.15.71:0/1211090037 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef7d847830 con 0x55ef7d8f3610 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f500aa00640 10 client.26552 handle_command_reply: tid=0 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 client.26552 shutdown 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 msgr2=0x55ef7d8e0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e0ba0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f50100116d0 tx=0x7f5010011b90 comp rx=0 tx=0).stop 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 shutdown_connections 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d8f3610 0x55ef7d8f5a00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f500aa00640 0 client.26552 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8e0ba0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 --2- 172.21.15.71:0/1211090037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8e0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.789+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 >> 172.21.15.71:0/1211090037 conn(0x55ef7d8d0750 msgr2=0x55ef7d8d64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.790+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 shutdown_connections 2024-09-05T12:37:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.790+0000 7f502e65d740 1 -- 172.21.15.71:0/1211090037 wait complete. 2024-09-05T12:37:43.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.790+0000 7f502e65d740 20 client.26552 trim_cache size 0 max 16384 2024-09-05T12:37:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 Processor -- start 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 -- start start 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8fb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50241acc50 con 0x55ef7d8d49a0 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50241ae7f0 con 0x55ef7d8e37b0 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.806+0000 7f502e65d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50241ad320 con 0x55ef7d8d4080 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5028e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5028e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8fb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33380/0 (socket says 172.21.15.71:33380) 2024-09-05T12:37:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5028e00640 1 -- 172.21.15.71:0/3895818417 learned_addr learned my addr 172.21.15.71:0/3895818417 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5023e00640 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 -- 172.21.15.71:0/3895818417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 msgr2=0x55ef7d8fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 -- 172.21.15.71:0/3895818417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8d49a0 msgr2=0x55ef7d8fb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8fb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 -- 172.21.15.71:0/3895818417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7d8d47d0 con 0x55ef7d8e37b0 2024-09-05T12:37:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.807+0000 7f5029800640 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e0620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f501c02ac10 tx=0x7f501c041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5323790faf3548db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f5015600640 1 -- 172.21.15.71:0/3895818417 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c036050 con 0x55ef7d8e37b0 2024-09-05T12:37:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f5015600640 1 -- 172.21.15.71:0/3895818417 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501c051070 con 0x55ef7d8e37b0 2024-09-05T12:37:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f5015600640 1 -- 172.21.15.71:0/3895818417 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c056b40 con 0x55ef7d8e37b0 2024-09-05T12:37:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 -- 172.21.15.71:0/3895818417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8e37b0 msgr2=0x55ef7d8e0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e0620 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f501c02ac10 tx=0x7f501c041000 comp rx=0 tx=0).stop 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 -- 172.21.15.71:0/3895818417 shutdown_connections 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8e0620 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 --2- 172.21.15.71:0/3895818417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.808+0000 7f502e65d740 1 -- 172.21.15.71:0/3895818417 >> 172.21.15.71:0/3895818417 conn(0x55ef7d8d0750 msgr2=0x55ef7d8df7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.809+0000 7f502e65d740 1 -- 172.21.15.71:0/3895818417 shutdown_connections 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.809+0000 7f502e65d740 1 -- 172.21.15.71:0/3895818417 wait complete. 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.809+0000 7f502e65d740 1 -- start start 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.809+0000 7f502e65d740 1 Processor -- start 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.809+0000 7f502e65d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8ebc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f502e65d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f502e65d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f502e65d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f50241ae120 con 0x55ef7d8e37b0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f502e65d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50241ad9f0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f502e65d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50241aac30 con 0x55ef7d8d4080 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f502e65d740 10 client.0 _ll_register_callbacks cb 0x7f5010009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5029800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50344/0 (socket says 172.21.15.71:50344) 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5028e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8ebc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 -- 172.21.15.71:0/216775929 learned_addr learned my addr 172.21.15.71:0/216775929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 -- 172.21.15.71:0/216775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 msgr2=0x55ef7d8ebc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8ebc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 -- 172.21.15.71:0/216775929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 msgr2=0x55ef7d8dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8dd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 -- 172.21.15.71:0/216775929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7d8ddb00 con 0x55ef7d8d49a0 2024-09-05T12:37:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f5023e00640 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8dd080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5010009de0 tx=0x7f501000fe20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5928ac5d97bcb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.810+0000 7f500aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f500aa00640 1 -- 172.21.15.71:0/216775929 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010005250 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f500aa00640 1 -- 172.21.15.71:0/216775929 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7f50100053f0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f501c051070 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f500aa00640 1 -- 172.21.15.71:0/216775929 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5010039050 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 10 client.26558 fetch_fsmap learned FSMap version 45 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ef7d95dfc0 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f500aa00640 1 -- 172.21.15.71:0/216775929 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f501c051070 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f500aa00640 1 -- 172.21.15.71:0/216775929 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) ==== 8378+0+0 (secure 0 0 0) 0x7f5010019040 con 0x55ef7d8d49a0 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 10 client.26558 fetch_fsmap finished waiting for FSMap version 45 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 10 client.26558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 20 client.26558 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d95e7c0 0x55ef7d960be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:43.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 4 client.26558 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:37:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.811+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef7d841b80 con 0x55ef7d95e7c0 2024-09-05T12:37:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.812+0000 7f5028e00640 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d95e7c0 0x55ef7d960be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.812+0000 7f5028e00640 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d95e7c0 0x55ef7d960be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=883ab1a9e9e3b06c server_cookie=716d4925b60711ee in_seq=0 out_seq=0 2024-09-05T12:37:43.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.812+0000 7f500aa00640 10 client.26558 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.844+0000 7f500aa00640 1 -- 172.21.15.71:0/216775929 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef7d841b80 con 0x55ef7d95e7c0 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.844+0000 7f500aa00640 10 client.26558 handle_command_reply: tid=0 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.844+0000 7f502e65d740 1 client.26558 shutdown 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.844+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 msgr2=0x55ef7d8dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.844+0000 7f502e65d740 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8dd080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5010009de0 tx=0x7f501000fe20 comp rx=0 tx=0).stop 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 shutdown_connections 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef7d95e7c0 0x55ef7d960be0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef7d8e37b0 0x55ef7d8dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef7d8d49a0 0x55ef7d8dd080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f500aa00640 0 client.26558 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 --2- 172.21.15.71:0/216775929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7d8d4080 0x55ef7d8ebc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 >> 172.21.15.71:0/216775929 conn(0x55ef7d8d0750 msgr2=0x55ef7d8e1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 shutdown_connections 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 1 -- 172.21.15.71:0/216775929 wait complete. 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.845+0000 7f502e65d740 20 client.26558 trim_cache size 0 max 16384 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:37:43.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f500c0a2da0 msgr2=0x7f500c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f500c0a2da0 0x7f500c0a5260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f501c004af0 tx=0x7f501c009280 comp rx=0 tx=0).stop 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 msgr2=0x7f5024198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 0x7f5024198310 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f501800e9f0 tx=0x7f501800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 shutdown_connections 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5024107b80 0x7f502419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5024102480 0x7f5024198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f500c0a2da0 0x7f500c0a5260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.848 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 --2- 172.21.15.71:0/3588415595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5024101ae0 0x7f5024198310 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 >> 172.21.15.71:0/3588415595 conn(0x7f50240fd7e0 msgr2=0x7f50240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 shutdown_connections 2024-09-05T12:37:43.849 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:43.846+0000 7f502b800640 1 -- 172.21.15.71:0/3588415595 wait complete. 2024-09-05T12:37:43.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:43 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:43.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:43 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:44.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:43 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:44.260 INFO:tasks.workunit.client.0.smithi071.stdout: 1 338084 3.26 MB/sec execute 400 sec latency 58.051 ms 2024-09-05T12:37:44.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:44 smithi071 ceph-mon[29845]: pgmap v311: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-05T12:37:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:44 smithi123 ceph-mon[34321]: pgmap v311: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-05T12:37:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:44 smithi174 ceph-mon[34313]: pgmap v311: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-09-05T12:37:45.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 338670 3.26 MB/sec execute 401 sec latency 636.157 ms 2024-09-05T12:37:46.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 338670 3.25 MB/sec execute 402 sec latency 1636.268 ms 2024-09-05T12:37:46.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:46 smithi071 ceph-mon[29845]: pgmap v312: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T12:37:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:46 smithi123 ceph-mon[34321]: pgmap v312: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T12:37:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:46 smithi174 ceph-mon[34313]: pgmap v312: 129 pgs: 129 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-09-05T12:37:47.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 339518 3.25 MB/sec execute 403 sec latency 1765.105 ms 2024-09-05T12:37:48.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 340925 3.27 MB/sec execute 404 sec latency 79.584 ms 2024-09-05T12:37:48.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:48 smithi071 ceph-mon[29845]: pgmap v313: 129 pgs: 129 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-09-05T12:37:48.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:48 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:48 smithi123 ceph-mon[34321]: pgmap v313: 129 pgs: 129 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-09-05T12:37:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:48 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:49.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:48 smithi174 ceph-mon[34313]: pgmap v313: 129 pgs: 129 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-09-05T12:37:49.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:48 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:49.257 INFO:tasks.workunit.client.0.smithi071.stdout: 1 341661 3.27 MB/sec execute 405 sec latency 94.206 ms 2024-09-05T12:37:50.258 INFO:tasks.workunit.client.0.smithi071.stdout: 1 342210 3.27 MB/sec execute 406 sec latency 659.204 ms 2024-09-05T12:37:50.812 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:50 smithi174 ceph-mon[34313]: pgmap v314: 129 pgs: 129 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-09-05T12:37:50.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:50 smithi071 ceph-mon[29845]: pgmap v314: 129 pgs: 129 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-09-05T12:37:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:50 smithi123 ceph-mon[34321]: pgmap v314: 129 pgs: 129 active+clean; 108 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-09-05T12:37:51.258 INFO:tasks.workunit.client.0.smithi071.stdout: 1 342210 3.26 MB/sec execute 407 sec latency 1659.321 ms 2024-09-05T12:37:52.258 INFO:tasks.workunit.client.0.smithi071.stdout: 1 343056 3.26 MB/sec execute 408 sec latency 1784.950 ms 2024-09-05T12:37:52.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:52 smithi071 ceph-mon[29845]: pgmap v315: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-09-05T12:37:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:52 smithi123 ceph-mon[34321]: pgmap v315: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-09-05T12:37:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:52 smithi174 ceph-mon[34313]: pgmap v315: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-09-05T12:37:53.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 344491 3.28 MB/sec execute 409 sec latency 72.232 ms 2024-09-05T12:37:53.824 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:53 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:53.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:53 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:54.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:53 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:37:54.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 345161 3.27 MB/sec execute 410 sec latency 201.025 ms 2024-09-05T12:37:54.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:54 smithi071 ceph-mon[29845]: pgmap v316: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-09-05T12:37:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:54 smithi123 ceph-mon[34321]: pgmap v316: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-09-05T12:37:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:54 smithi174 ceph-mon[34313]: pgmap v316: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.3 MiB/s wr, 185 op/s 2024-09-05T12:37:55.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 345759 3.27 MB/sec execute 411 sec latency 650.094 ms 2024-09-05T12:37:56.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 345759 3.27 MB/sec execute 412 sec latency 1650.209 ms 2024-09-05T12:37:56.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:56 smithi071 ceph-mon[29845]: pgmap v317: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-05T12:37:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:56 smithi123 ceph-mon[34321]: pgmap v317: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.964+0000 7fc4ec400640 1 -- 172.21.15.71:0/800357868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e410e320 msgr2=0x7fc4e4110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.964+0000 7fc4ec400640 1 --2- 172.21.15.71:0/800357868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e410e320 0x7fc4e4110770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009ca0 tx=0x7fc4dc02a940 comp rx=0 tx=0).stop 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 -- 172.21.15.71:0/800357868 shutdown_connections 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 --2- 172.21.15.71:0/800357868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e410e320 0x7fc4e4110770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 --2- 172.21.15.71:0/800357868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e4074f40 0x7fc4e410dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 --2- 172.21.15.71:0/800357868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e4074620 0x7fc4e4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 -- 172.21.15.71:0/800357868 >> 172.21.15.71:0/800357868 conn(0x7fc4e40fd7e0 msgr2=0x7fc4e40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 -- 172.21.15.71:0/800357868 shutdown_connections 2024-09-05T12:37:56.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 -- 172.21.15.71:0/800357868 wait complete. 2024-09-05T12:37:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.965+0000 7fc4ec400640 1 Processor -- start 2024-09-05T12:37:56.969 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 -- start start 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e4074620 0x7fc4e419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e4074f40 0x7fc4e419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 0x7fc4e419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4114110 con 0x7fc4e4074620 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4113f90 con 0x7fc4e410e320 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ec400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e4114290 con 0x7fc4e4074f40 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e4074620 0x7fc4e419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ea400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 0x7fc4e419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ea400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 0x7fc4e419ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35168/0 (socket says 172.21.15.71:35168) 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ea400640 1 -- 172.21.15.71:0/2502713217 learned_addr learned my addr 172.21.15.71:0/2502713217 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4e9000640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e4074f40 0x7fc4e419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.966+0000 7fc4ea400640 1 -- 172.21.15.71:0/2502713217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e4074f40 msgr2=0x7fc4e419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ea400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e4074f40 0x7fc4e419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ea400640 1 -- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e4074620 msgr2=0x7fc4e419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ea400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e4074620 0x7fc4e419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ea400640 1 -- 172.21.15.71:0/2502713217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e419f590 con 0x7fc4e410e320 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ea400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 0x7fc4e419ee60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005e30 tx=0x7fc4dc02f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4e2000640 1 -- 172.21.15.71:0/2502713217 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc034040 con 0x7fc4e410e320 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4e2000640 1 -- 172.21.15.71:0/2502713217 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4dc02fe10 con 0x7fc4e410e320 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4e2000640 1 -- 172.21.15.71:0/2502713217 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0045d0 con 0x7fc4e410e320 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e419f820 con 0x7fc4e410e320 2024-09-05T12:37:56.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.967+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e41a7110 con 0x7fc4e410e320 2024-09-05T12:37:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.970+0000 7fc4e2000640 1 -- 172.21.15.71:0/2502713217 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc4dc037020 con 0x7fc4e410e320 2024-09-05T12:37:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.970+0000 7fc4ef3ee740 1 Processor -- start 2024-09-05T12:37:56.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 -- start start 2024-09-05T12:37:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 0x5633d66ace60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66ad3a0 0x5633d66af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5633d6608100 con 0x5633d66a21e0 2024-09-05T12:37:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5633d65b01b0 con 0x5633d66a2b00 2024-09-05T12:37:56.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633d66040c0 con 0x5633d66ad3a0 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4e9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 0x5633d66ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 0x5633d66ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35172/0 (socket says 172.21.15.71:35172) 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4e9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66a25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56518/0 (socket says 172.21.15.71:56518) 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1675439785 learned_addr learned my addr 172.21.15.71:0/1675439785 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.971+0000 7fc4e9000640 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66ad3a0 0x5633d66af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1675439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66ad3a0 msgr2=0x5633d66af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66ad3a0 0x5633d66af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1675439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 msgr2=0x5633d66a25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66a25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1675439785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d631b210 con 0x5633d66a2b00 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 0x5633d66ace60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009d20 tx=0x7fc4d8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4f69b2bc0b3c9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:56.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e2000640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4cc0a2e20 0x7fc4cc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4d6a00640 1 -- 172.21.15.71:0/1675439785 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d801e070 con 0x5633d66a2b00 2024-09-05T12:37:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4d6a00640 1 -- 172.21.15.71:0/1675439785 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d8009e10 con 0x5633d66a2b00 2024-09-05T12:37:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e2000640 1 -- 172.21.15.71:0/2502713217 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7fc4dc0f03b0 con 0x7fc4e410e320 2024-09-05T12:37:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4d6a00640 1 -- 172.21.15.71:0/1675439785 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80188f0 con 0x5633d66a2b00 2024-09-05T12:37:56.976 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.972+0000 7fc4e9000640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4cc0a2e20 0x7fc4cc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1675439785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 msgr2=0x5633d66ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 0x5633d66ace60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009d20 tx=0x7fc4d8011810 comp rx=0 tx=0).stop 2024-09-05T12:37:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4e9000640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4cc0a2e20 0x7fc4cc0a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5633d669fad0 tx=0x7fc4d0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1675439785 shutdown_connections 2024-09-05T12:37:56.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66ad3a0 0x5633d66af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a2b00 0x5633d66ace60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1675439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66a25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1675439785 >> 172.21.15.71:0/1675439785 conn(0x5633d669e750 msgr2=0x5633d669eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1675439785 shutdown_connections 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.973+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1675439785 wait complete. 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 -- start start 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 Processor -- start 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66aa900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d66aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5633d66affd0 con 0x5633d66ad3a0 2024-09-05T12:37:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5633d6591090 con 0x5633d66a21e0 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633d66b0150 con 0x5633d66a2b00 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.974+0000 7fc4ef3ee740 10 client.0 _ll_register_callbacks cb 0x5633d65e5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66aa900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35182/0 (socket says 172.21.15.71:35182) 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1373002865 learned_addr learned my addr 172.21.15.71:0/1373002865 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66aa900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46064/0 (socket says 172.21.15.71:46064) 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4ea400640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d66aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1373002865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 msgr2=0x5633d66aa900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66aa900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1373002865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 msgr2=0x5633d66aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d66aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 -- 172.21.15.71:0/1373002865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d63e7620 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4ea400640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d66aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66aa3c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009cf0 tx=0x7fc4d80045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9851c8a9e294ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4cb400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4cb400640 1 -- 172.21.15.71:0/1373002865 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8026020 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d8009e10 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.975+0000 7fc4cb400640 1 -- 172.21.15.71:0/1373002865 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7fc4d8018670 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4cb400640 1 -- 172.21.15.71:0/1373002865 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc4d8031050 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 10 client.26570 fetch_fsmap learned FSMap version 47 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5633d66bbc10 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4cb400640 1 -- 172.21.15.71:0/1373002865 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d8009e10 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4cb400640 1 -- 172.21.15.71:0/1373002865 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fc4d8021050 con 0x5633d66a21e0 2024-09-05T12:37:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 10 client.26570 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 10 client.26570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 20 client.26570 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d66bd010 0x5633d66bf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 4 client.26570 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.976+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633d6615830 con 0x5633d66bd010 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.977+0000 7fc4e9000640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d66bd010 0x5633d66bf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.977+0000 7fc4e9000640 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d66bd010 0x5633d66bf400 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd333713a6009b2b server_cookie=34169d2f20f93550 in_seq=0 out_seq=0 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.977+0000 7fc4cb400640 10 client.26570 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4cb400640 1 -- 172.21.15.71:0/1373002865 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633d6615830 con 0x5633d66bd010 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4cb400640 10 client.26570 handle_command_reply: tid=0 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 client.26570 shutdown 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 msgr2=0x5633d66aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66aa3c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009cf0 tx=0x7fc4d80045c0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 shutdown_connections 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d66bd010 0x5633d66bf400 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d66aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66aa900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1373002865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66aa3c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 >> 172.21.15.71:0/1373002865 conn(0x5633d669e750 msgr2=0x5633d66a44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.978+0000 7fc4cb400640 0 client.26570 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.979+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 shutdown_connections 2024-09-05T12:37:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.979+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1373002865 wait complete. 2024-09-05T12:37:56.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.979+0000 7fc4ef3ee740 20 client.26570 trim_cache size 0 max 16384 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 Processor -- start 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 -- start start 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66996a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66ad3a0 0x5633d66c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41b0f50 con 0x5633d66a21e0 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41b2a90 con 0x5633d66ad3a0 2024-09-05T12:37:57.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41b1620 con 0x5633d66a2b00 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4e9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4ea400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66ad3a0 0x5633d66c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4e9000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66c4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46080/0 (socket says 172.21.15.71:46080) 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.998+0000 7fc4e9000640 1 -- 172.21.15.71:0/1967551889 learned_addr learned my addr 172.21.15.71:0/1967551889 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4e9000640 1 -- 172.21.15.71:0/1967551889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66ad3a0 msgr2=0x5633d66c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4e9000640 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66ad3a0 0x5633d66c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4e9000640 1 -- 172.21.15.71:0/1967551889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 msgr2=0x5633d66996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4e9000640 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4e9000640 1 -- 172.21.15.71:0/1967551889 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d66a88f0 con 0x5633d66a2b00 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4ea400640 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66ad3a0 0x5633d66c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4e9000640 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66c4b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5633d66aba00 tx=0x7fc4d0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63f3620e6f8930d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4d6000640 1 -- 172.21.15.71:0/1967551889 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0013e60 con 0x5633d66a2b00 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4d6000640 1 -- 172.21.15.71:0/1967551889 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d0020e00 con 0x5633d66a2b00 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:56.999+0000 7fc4d6000640 1 -- 172.21.15.71:0/1967551889 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0005530 con 0x5633d66a2b00 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1967551889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 msgr2=0x5633d66c4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66c4b90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5633d66aba00 tx=0x7fc4d0012000 comp rx=0 tx=0).stop 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1967551889 shutdown_connections 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66ad3a0 0x5633d66c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66c4b90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/1967551889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66a21e0 0x5633d66996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1967551889 >> 172.21.15.71:0/1967551889 conn(0x5633d669e750 msgr2=0x5633d66a8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1967551889 shutdown_connections 2024-09-05T12:37:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.000+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/1967551889 wait complete. 2024-09-05T12:37:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 -- start start 2024-09-05T12:37:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 Processor -- start 2024-09-05T12:37:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d6725b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41b23c0 con 0x5633d66ad3a0 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41b1cf0 con 0x5633d66a21e0 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e41aef30 con 0x5633d66a2b00 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4ef3ee740 10 client.0 _ll_register_callbacks cb 0x7fc4d801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4e9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66b0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35196/0 (socket says 172.21.15.71:35196) 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.001+0000 7fc4e9a00640 1 -- 172.21.15.71:0/3199019034 learned_addr learned my addr 172.21.15.71:0/3199019034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9000640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4ea400640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d6725b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9a00640 1 -- 172.21.15.71:0/3199019034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 msgr2=0x5633d66b0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66b0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9a00640 1 -- 172.21.15.71:0/3199019034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 msgr2=0x5633d6725b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d6725b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9a00640 1 -- 172.21.15.71:0/3199019034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d6726090 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4e9a00640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66b0340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003a70 tx=0x7fc4d8011e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=430bfb69831ef45e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4cb400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4cb400640 1 -- 172.21.15.71:0/3199019034 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8026040 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4cb400640 1 -- 172.21.15.71:0/3199019034 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9350+0+0 (secure 0 0 0) 0x7fc4d80037f0 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d0020e00 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4cb400640 1 -- 172.21.15.71:0/3199019034 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc4d801e040 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4ef3ee740 10 client.26576 fetch_fsmap learned FSMap version 47 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.002+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5633d67271e0 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4cb400640 1 -- 172.21.15.71:0/3199019034 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4d0020e00 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4cb400640 1 -- 172.21.15.71:0/3199019034 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7fc4d8021040 con 0x5633d66a21e0 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4ef3ee740 10 client.26576 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T12:37:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4ef3ee740 10 client.26576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4ef3ee740 20 client.26576 populate_metadata read hostname 'smithi071' 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d6727840 0x5633d6729c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4ef3ee740 4 client.26576 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5633d6699d80 con 0x5633d6727840 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.003+0000 7fc4e9000640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d6727840 0x5633d6729c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.004+0000 7fc4e9000640 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d6727840 0x5633d6729c60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ec4194afdb42cc server_cookie=5d73323cf6032316 in_seq=0 out_seq=0 2024-09-05T12:37:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.004+0000 7fc4cb400640 10 client.26576 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:57.009 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:56 smithi174 ceph-mon[34313]: pgmap v317: 129 pgs: 129 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.1 MiB/s wr, 182 op/s 2024-09-05T12:37:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4cb400640 1 -- 172.21.15.71:0/3199019034 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5633d6699d80 con 0x5633d6727840 2024-09-05T12:37:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4cb400640 10 client.26576 handle_command_reply: tid=0 2024-09-05T12:37:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 client.26576 shutdown 2024-09-05T12:37:57.029 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 msgr2=0x5633d66b0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66b0340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003a70 tx=0x7fc4d8011e90 comp rx=0 tx=0).stop 2024-09-05T12:37:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 shutdown_connections 2024-09-05T12:37:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5633d6727840 0x5633d6729c60 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d66ad3a0 0x5633d6725b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4cb400640 0 client.26576 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:37:57.030 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633d66a2b00 0x5633d66b0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 --2- 172.21.15.71:0/3199019034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633d66a21e0 0x5633d66b0340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 >> 172.21.15.71:0/3199019034 conn(0x5633d669e750 msgr2=0x5633d66aa360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 shutdown_connections 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.027+0000 7fc4ef3ee740 1 -- 172.21.15.71:0/3199019034 wait complete. 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ef3ee740 20 client.26576 trim_cache size 0 max 16384 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:37:57.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:37:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4cc0a2e20 msgr2=0x7fc4cc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4cc0a2e20 0x7fc4cc0a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5633d669fad0 tx=0x7fc4d0005f90 comp rx=0 tx=0).stop 2024-09-05T12:37:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 msgr2=0x7fc4e419ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:37:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 0x7fc4e419ee60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc005e30 tx=0x7fc4dc02f630 comp rx=0 tx=0).stop 2024-09-05T12:37:57.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 shutdown_connections 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4e410e320 0x7fc4e419ee60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4e4074f40 0x7fc4e419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4cc0a2e20 0x7fc4cc0a52e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 --2- 172.21.15.71:0/2502713217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e4074620 0x7fc4e419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.028+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 >> 172.21.15.71:0/2502713217 conn(0x7fc4e40fd7e0 msgr2=0x7fc4e40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.029+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 shutdown_connections 2024-09-05T12:37:57.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:37:57.029+0000 7fc4ec400640 1 -- 172.21.15.71:0/2502713217 wait complete. 2024-09-05T12:37:57.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 347274 3.28 MB/sec execute 413 sec latency 1767.106 ms 2024-09-05T12:37:58.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 348513 3.28 MB/sec execute 414 sec latency 47.855 ms 2024-09-05T12:37:58.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:37:58 smithi071 ceph-mon[29845]: pgmap v318: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-05T12:37:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:37:58 smithi123 ceph-mon[34321]: pgmap v318: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-05T12:37:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:37:58 smithi174 ceph-mon[34313]: pgmap v318: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.7 MiB/s wr, 272 op/s 2024-09-05T12:37:59.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 348869 3.28 MB/sec execute 415 sec latency 456.739 ms 2024-09-05T12:38:00.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 349302 3.28 MB/sec execute 416 sec latency 753.993 ms 2024-09-05T12:38:00.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:00 smithi071 ceph-mon[29845]: pgmap v319: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-05T12:38:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:00 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:38:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:00 smithi123 ceph-mon[34321]: pgmap v319: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-05T12:38:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:00 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:38:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:00 smithi174 ceph-mon[34313]: pgmap v319: 129 pgs: 129 active+clean; 114 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.1 MiB/s wr, 179 op/s 2024-09-05T12:38:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:00 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:38:01.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 349302 3.27 MB/sec execute 417 sec latency 1754.087 ms 2024-09-05T12:38:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:02 smithi071 ceph-mon[29845]: pgmap v320: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-09-05T12:38:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:02 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:02.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:02 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:02 smithi174 ceph-mon[34313]: pgmap v320: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-09-05T12:38:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:02 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:02 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:02.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 349884 3.27 MB/sec execute 418 sec latency 1868.084 ms 2024-09-05T12:38:02.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:02 smithi123 ceph-mon[34321]: pgmap v320: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 258 op/s 2024-09-05T12:38:02.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:02 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:02.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:02 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.268 INFO:tasks.workunit.client.0.smithi071.stdout: 1 350102 3.26 MB/sec execute 419 sec latency 199.460 ms 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:38:03.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:03 smithi071 ceph-mon[29845]: osdmap e105: 12 total, 12 up, 12 in 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:38:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:38:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:38:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:38:03.431 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:03 smithi123 ceph-mon[34321]: osdmap e105: 12 total, 12 up, 12 in 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:38:03.511 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:03 smithi174 ceph-mon[34313]: osdmap e105: 12 total, 12 up, 12 in 2024-09-05T12:38:04.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 351287 3.28 MB/sec execute 420 sec latency 370.482 ms 2024-09-05T12:38:04.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: pgmap v322: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:38:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: pgmap v322: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-05T12:38:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:38:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:38:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: pgmap v322: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-05T12:38:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:38:04.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:38:04.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:38:04.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:38:05.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 352341 3.28 MB/sec execute 421 sec latency 15.559 ms 2024-09-05T12:38:05.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:05 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:38:05.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:05 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:38:05.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:05 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:38:05.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:05 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:38:05.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:05 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:38:05.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:05 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:38:06.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 352850 3.28 MB/sec execute 422 sec latency 534.394 ms 2024-09-05T12:38:06.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:06 smithi071 ceph-mon[29845]: pgmap v323: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-05T12:38:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:06 smithi123 ceph-mon[34321]: pgmap v323: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-05T12:38:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:06 smithi174 ceph-mon[34313]: pgmap v323: 129 pgs: 129 active+clean; 113 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 207 op/s 2024-09-05T12:38:07.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 354168 3.29 MB/sec execute 423 sec latency 648.161 ms 2024-09-05T12:38:08.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 355377 3.30 MB/sec execute 424 sec latency 16.568 ms 2024-09-05T12:38:08.564 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:08 smithi071 ceph-mon[29845]: pgmap v324: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-09-05T12:38:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:08 smithi123 ceph-mon[34321]: pgmap v324: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-09-05T12:38:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:08 smithi174 ceph-mon[34313]: pgmap v324: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-09-05T12:38:09.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 355984 3.29 MB/sec execute 425 sec latency 380.945 ms 2024-09-05T12:38:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 -- 172.21.15.71:0/1559044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 msgr2=0x7ff3980749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 --2- 172.21.15.71:0/1559044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff3980749a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff388009900 tx=0x7ff38802a480 comp rx=0 tx=0).stop 2024-09-05T12:38:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 -- 172.21.15.71:0/1559044547 shutdown_connections 2024-09-05T12:38:10.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 --2- 172.21.15.71:0/1559044547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39810e360 0x7ff398110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 --2- 172.21.15.71:0/1559044547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff398074ee0 0x7ff39810de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 --2- 172.21.15.71:0/1559044547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff3980749a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 -- 172.21.15.71:0/1559044547 >> 172.21.15.71:0/1559044547 conn(0x7ff3980fd7e0 msgr2=0x7ff3980ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 -- 172.21.15.71:0/1559044547 shutdown_connections 2024-09-05T12:38:10.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.142+0000 7ff39e000640 1 -- 172.21.15.71:0/1559044547 wait complete. 2024-09-05T12:38:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 Processor -- start 2024-09-05T12:38:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 -- start start 2024-09-05T12:38:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff39819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff398074ee0 0x7ff39819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff39810e360 0x7ff39819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff398114130 con 0x7ff3980745c0 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff398113fb0 con 0x7ff398074ee0 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.143+0000 7ff39e000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981142b0 con 0x7ff39810e360 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff397400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff39819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff397400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff39819a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42134/0 (socket says 172.21.15.71:42134) 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff398074ee0 0x7ff39819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff397400640 1 -- 172.21.15.71:0/4263890325 learned_addr learned my addr 172.21.15.71:0/4263890325 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff397e00640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff39810e360 0x7ff39819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 -- 172.21.15.71:0/4263890325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff39810e360 msgr2=0x7ff39819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff39810e360 0x7ff39819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 -- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 msgr2=0x7ff39819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff39819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 -- 172.21.15.71:0/4263890325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39819f590 con 0x7ff398074ee0 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff397e00640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff39810e360 0x7ff39819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff397400640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff39819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.144+0000 7ff38ea00640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff398074ee0 0x7ff39819aae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e9f0 tx=0x7ff38000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.145+0000 7ff394c00640 1 -- 172.21.15.71:0/4263890325 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38000ccc0 con 0x7ff398074ee0 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.145+0000 7ff394c00640 1 -- 172.21.15.71:0/4263890325 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff380010910 con 0x7ff398074ee0 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.145+0000 7ff394c00640 1 -- 172.21.15.71:0/4263890325 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380018960 con 0x7ff398074ee0 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.145+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39819f880 con 0x7ff398074ee0 2024-09-05T12:38:10.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.145+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3981a7110 con 0x7ff398074ee0 2024-09-05T12:38:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.147+0000 7ff3a0e3e740 1 Processor -- start 2024-09-05T12:38:10.152 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 -- start start 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff394c00640 1 -- 172.21.15.71:0/4263890325 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff380010ac0 con 0x7ff398074ee0 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500d2b00 0x5556500dce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500dd3a0 0x5556500df790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555650038100 con 0x5556500d2b00 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55564ffe01b0 con 0x5556500dd3a0 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556500340c0 con 0x5556500d21e0 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff397400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500d2b00 0x5556500dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff397400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500d2b00 0x5556500dce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42148/0 (socket says 172.21.15.71:42148) 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff397e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff397400640 1 -- 172.21.15.71:0/2304469688 learned_addr learned my addr 172.21.15.71:0/2304469688 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:10.153 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.148+0000 7ff38ea00640 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500dd3a0 0x5556500df790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff397e00640 1 -- 172.21.15.71:0/2304469688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500dd3a0 msgr2=0x5556500df790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff397e00640 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500dd3a0 0x5556500df790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff397e00640 1 -- 172.21.15.71:0/2304469688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500d2b00 msgr2=0x5556500dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff397e00640 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500d2b00 0x5556500dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff397e00640 1 -- 172.21.15.71:0/2304469688 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55564fd4b210 con 0x5556500d21e0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff397e00640 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d25c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff398076600 tx=0x7ff3900107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7658a1679038600e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff387e00640 1 -- 172.21.15.71:0/2304469688 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff390011640 con 0x5556500d21e0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff387e00640 1 -- 172.21.15.71:0/2304469688 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390015070 con 0x5556500d21e0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff387e00640 1 -- 172.21.15.71:0/2304469688 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39001bc70 con 0x5556500d21e0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.149+0000 7ff394c00640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff37c0a2e20 0x7ff37c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff394c00640 1 -- 172.21.15.71:0/4263890325 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9526+0+0 (secure 0 0 0) 0x7ff3800cfc60 con 0x7ff398074ee0 2024-09-05T12:38:10.154 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff397400640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff37c0a2e20 0x7ff37c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2304469688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 msgr2=0x5556500d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d25c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff398076600 tx=0x7ff3900107a0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2304469688 shutdown_connections 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500dd3a0 0x5556500df790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500d2b00 0x5556500dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2304469688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d25c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2304469688 >> 172.21.15.71:0/2304469688 conn(0x5556500ce750 msgr2=0x5556500ceb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2304469688 shutdown_connections 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2304469688 wait complete. 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.150+0000 7ff397400640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff37c0a2e20 0x7ff37c0a52e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3880096c0 tx=0x7ff3880046c0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 -- start start 2024-09-05T12:38:10.155 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 Processor -- start 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d21e0 0x5556500da350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d2b00 0x5556500da890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5556500dffd0 con 0x5556500dd3a0 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55564ffc1090 con 0x5556500d21e0 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556500e0150 con 0x5556500d2b00 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff3a0e3e740 10 client.0 _ll_register_callbacks cb 0x555650015f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.151+0000 7ff397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d21e0 0x5556500da350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42164/0 (socket says 172.21.15.71:42164) 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff38ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d2b00 0x5556500da890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 -- 172.21.15.71:0/1242785415 learned_addr learned my addr 172.21.15.71:0/1242785415 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 -- 172.21.15.71:0/1242785415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d2b00 msgr2=0x5556500da890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d2b00 0x5556500da890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 -- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d21e0 msgr2=0x5556500da350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d21e0 0x5556500da350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 -- 172.21.15.71:0/1242785415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55564fe17620 con 0x5556500dd3a0 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff38ea00640 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d2b00 0x5556500da890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff397e00640 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dadd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff390010fe0 tx=0x7ff39001b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2918188db5882eda server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff384c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff384c00640 1 -- 172.21.15.71:0/1242785415 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39002a060 con 0x5556500dd3a0 2024-09-05T12:38:10.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff390015070 con 0x5556500dd3a0 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.152+0000 7ff384c00640 1 -- 172.21.15.71:0/1242785415 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9526+0+0 (secure 0 0 0) 0x7ff390004d50 con 0x5556500dd3a0 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff384c00640 1 -- 172.21.15.71:0/1242785415 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff390031050 con 0x5556500dd3a0 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 10 client.16551 fetch_fsmap learned FSMap version 47 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5556500ebba0 con 0x5556500dd3a0 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff384c00640 1 -- 172.21.15.71:0/1242785415 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390015070 con 0x5556500dd3a0 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff384c00640 1 -- 172.21.15.71:0/1242785415 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7ff390005840 con 0x5556500dd3a0 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 10 client.16551 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T12:38:10.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 20 client.16551 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556500ecfa0 0x5556500ef390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 4 client.16551 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.153+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555650045830 con 0x5556500ecfa0 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.154+0000 7ff397400640 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556500ecfa0 0x5556500ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.154+0000 7ff397400640 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556500ecfa0 0x5556500ef390 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4688a97fd23e8c4d server_cookie=6e0d69f6ad1b7de2 in_seq=0 out_seq=0 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.154+0000 7ff384c00640 10 client.16551 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff384c00640 1 -- 172.21.15.71:0/1242785415 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555650045830 con 0x5556500ecfa0 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff384c00640 10 client.16551 handle_command_reply: tid=0 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 client.16551 shutdown 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 msgr2=0x5556500dadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dadd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff390010fe0 tx=0x7ff39001b680 comp rx=0 tx=0).stop 2024-09-05T12:38:10.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 shutdown_connections 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556500ecfa0 0x5556500ef390 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dadd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d2b00 0x5556500da890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/1242785415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d21e0 0x5556500da350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff384c00640 0 client.16551 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.155+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 >> 172.21.15.71:0/1242785415 conn(0x5556500ce750 msgr2=0x5556500d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.156+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 shutdown_connections 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.156+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/1242785415 wait complete. 2024-09-05T12:38:10.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.156+0000 7ff3a0e3e740 20 client.16551 trim_cache size 0 max 16384 2024-09-05T12:38:10.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.170+0000 7ff3a0e3e740 1 Processor -- start 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.170+0000 7ff3a0e3e740 1 -- start start 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500d7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b12b0 con 0x5556500dd3a0 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b2df0 con 0x5556500d2b00 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b1980 con 0x5556500d21e0 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42170/0 (socket says 172.21.15.71:42170) 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 -- 172.21.15.71:0/782349472 learned_addr learned my addr 172.21.15.71:0/782349472 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:10.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff38ea00640 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500d7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397400640 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 -- 172.21.15.71:0/782349472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 msgr2=0x5556500d75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 -- 172.21.15.71:0/782349472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 msgr2=0x5556500d7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500d7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff397e00640 1 -- 172.21.15.71:0/782349472 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556500d2930 con 0x5556500dd3a0 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.171+0000 7ff38ea00640 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff397e00640 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dbaa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff39000a010 tx=0x7ff390011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912398afab6e51a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff387400640 1 -- 172.21.15.71:0/782349472 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39002a070 con 0x5556500dd3a0 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff387400640 1 -- 172.21.15.71:0/782349472 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390015070 con 0x5556500dd3a0 2024-09-05T12:38:10.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff387400640 1 -- 172.21.15.71:0/782349472 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff39001b770 con 0x5556500dd3a0 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/782349472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 msgr2=0x5556500dbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dbaa0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff39000a010 tx=0x7ff390011990 comp rx=0 tx=0).stop 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/782349472 shutdown_connections 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x5556500dbaa0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500d7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/782349472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500d75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/782349472 >> 172.21.15.71:0/782349472 conn(0x5556500ce750 msgr2=0x5556500d8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/782349472 shutdown_connections 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.172+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/782349472 wait complete. 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 -- start start 2024-09-05T12:38:10.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 Processor -- start 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500f0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x55565014e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b2720 con 0x5556500dd3a0 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981b2050 con 0x5556500d2b00 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.173+0000 7ff3a0e3e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3981af2b0 con 0x5556500d21e0 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff3a0e3e740 10 client.0 _ll_register_callbacks cb 0x7ff380005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:10.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x55565014e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff397e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x55565014e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42180/0 (socket says 172.21.15.71:42180) 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff397e00640 1 -- 172.21.15.71:0/2274671014 learned_addr learned my addr 172.21.15.71:0/2274671014 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff397400640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500f0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 -- 172.21.15.71:0/2274671014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 msgr2=0x5556500efc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500efc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 -- 172.21.15.71:0/2274671014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 msgr2=0x55565014e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x55565014e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 -- 172.21.15.71:0/2274671014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55565014ef30 con 0x5556500d2b00 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff397e00640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x55565014e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff38ea00640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500f0160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3800ccf60 tx=0x7ff38000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592d5004770cd0df server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.174+0000 7ff384c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:38:10.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff384c00640 1 -- 172.21.15.71:0/2274671014 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38009b690 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff384c00640 1 -- 172.21.15.71:0/2274671014 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9526+0+0 (secure 0 0 0) 0x7ff380010430 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff390015070 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff384c00640 1 -- 172.21.15.71:0/2274671014 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7ff38009c430 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 10 client.26594 fetch_fsmap learned FSMap version 47 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5556501574d0 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff384c00640 1 -- 172.21.15.71:0/2274671014 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff390015070 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff384c00640 1 -- 172.21.15.71:0/2274671014 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7ff380097a80 con 0x5556500d2b00 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 10 client.26594 fetch_fsmap finished waiting for FSMap version 47 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 10 client.26594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 20 client.26594 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555650157c70 0x55565015a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 4 client.26594 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55565003fb80 con 0x555650157c70 2024-09-05T12:38:10.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.175+0000 7ff397400640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555650157c70 0x55565015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.176+0000 7ff384c00640 10 client.26594 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:10.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.176+0000 7ff397400640 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555650157c70 0x55565015a090 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb774d553d11e8b3 server_cookie=40b91b546e7f2b44 in_seq=0 out_seq=0 2024-09-05T12:38:10.218 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff384c00640 1 -- 172.21.15.71:0/2274671014 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55565003fb80 con 0x555650157c70 2024-09-05T12:38:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff384c00640 10 client.26594 handle_command_reply: tid=0 2024-09-05T12:38:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 client.26594 shutdown 2024-09-05T12:38:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 msgr2=0x5556500f0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.219 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500f0160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff3800ccf60 tx=0x7ff38000cf60 comp rx=0 tx=0).stop 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 shutdown_connections 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555650157c70 0x55565015a090 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556500dd3a0 0x55565014e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556500d2b00 0x5556500f0160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 --2- 172.21.15.71:0/2274671014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556500d21e0 0x5556500efc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 >> 172.21.15.71:0/2274671014 conn(0x5556500ce750 msgr2=0x5556500d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.217+0000 7ff384c00640 0 client.26594 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.218+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 shutdown_connections 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.218+0000 7ff3a0e3e740 1 -- 172.21.15.71:0/2274671014 wait complete. 2024-09-05T12:38:10.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.218+0000 7ff3a0e3e740 20 client.26594 trim_cache size 0 max 16384 2024-09-05T12:38:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:38:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:38:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:38:10.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff37c0a2e20 msgr2=0x7ff37c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff37c0a2e20 0x7ff37c0a52e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff3880096c0 tx=0x7ff3880046c0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff398074ee0 msgr2=0x7ff39819aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff398074ee0 0x7ff39819aae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff38000e9f0 tx=0x7ff38000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 shutdown_connections 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff39810e360 0x7ff39819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff398074ee0 0x7ff39819aae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff37c0a2e20 0x7ff37c0a52e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 --2- 172.21.15.71:0/4263890325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3980745c0 0x7ff39819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 >> 172.21.15.71:0/4263890325 conn(0x7ff3980fd7e0 msgr2=0x7ff3980fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 shutdown_connections 2024-09-05T12:38:10.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:10.219+0000 7ff39e000640 1 -- 172.21.15.71:0/4263890325 wait complete. 2024-09-05T12:38:10.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 356392 3.29 MB/sec execute 426 sec latency 676.175 ms 2024-09-05T12:38:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:10 smithi071 ceph-mon[29845]: pgmap v325: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-09-05T12:38:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:10 smithi071 ceph-mon[29845]: osdmap e106: 12 total, 12 up, 12 in 2024-09-05T12:38:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:10 smithi123 ceph-mon[34321]: pgmap v325: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-09-05T12:38:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:10 smithi123 ceph-mon[34321]: osdmap e106: 12 total, 12 up, 12 in 2024-09-05T12:38:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:10 smithi174 ceph-mon[34313]: pgmap v325: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 214 op/s 2024-09-05T12:38:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:10 smithi174 ceph-mon[34313]: osdmap e106: 12 total, 12 up, 12 in 2024-09-05T12:38:11.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 356392 3.28 MB/sec execute 427 sec latency 1676.325 ms 2024-09-05T12:38:12.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 358063 3.30 MB/sec execute 428 sec latency 1791.548 ms 2024-09-05T12:38:12.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:12 smithi071 ceph-mon[29845]: pgmap v327: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-09-05T12:38:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:12 smithi123 ceph-mon[34321]: pgmap v327: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-09-05T12:38:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:12 smithi174 ceph-mon[34313]: pgmap v327: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 269 op/s 2024-09-05T12:38:13.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 359197 3.30 MB/sec execute 429 sec latency 13.783 ms 2024-09-05T12:38:14.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 359843 3.31 MB/sec execute 430 sec latency 360.745 ms 2024-09-05T12:38:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:14 smithi071 ceph-mon[29845]: pgmap v328: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-05T12:38:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:14 smithi123 ceph-mon[34321]: pgmap v328: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-05T12:38:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:14 smithi174 ceph-mon[34313]: pgmap v328: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-05T12:38:15.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 359941 3.30 MB/sec execute 431 sec latency 829.357 ms 2024-09-05T12:38:16.270 INFO:tasks.workunit.client.0.smithi071.stdout: 1 359941 3.29 MB/sec execute 432 sec latency 1829.473 ms 2024-09-05T12:38:16.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:16 smithi071 ceph-mon[29845]: pgmap v329: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-05T12:38:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:16 smithi123 ceph-mon[34321]: pgmap v329: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-05T12:38:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:16 smithi174 ceph-mon[34313]: pgmap v329: 129 pgs: 129 active+clean; 115 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-05T12:38:17.269 INFO:tasks.workunit.client.0.smithi071.stdout: 1 361455 3.30 MB/sec execute 433 sec latency 1960.671 ms 2024-09-05T12:38:18.270 INFO:tasks.workunit.client.0.smithi071.stdout: 1 362675 3.31 MB/sec execute 434 sec latency 17.789 ms 2024-09-05T12:38:18.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:18 smithi071 ceph-mon[29845]: pgmap v330: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-09-05T12:38:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:18 smithi123 ceph-mon[34321]: pgmap v330: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-09-05T12:38:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:18 smithi174 ceph-mon[34313]: pgmap v330: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-09-05T12:38:19.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 363483 3.31 MB/sec execute 435 sec latency 500.444 ms 2024-09-05T12:38:20.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 363483 3.30 MB/sec execute 436 sec latency 1500.556 ms 2024-09-05T12:38:20.584 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:20 smithi071 ceph-mon[29845]: pgmap v331: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-09-05T12:38:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:20 smithi123 ceph-mon[34321]: pgmap v331: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-09-05T12:38:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:20 smithi174 ceph-mon[34313]: pgmap v331: 129 pgs: 129 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-09-05T12:38:21.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 363483 3.30 MB/sec execute 437 sec latency 2500.653 ms 2024-09-05T12:38:22.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 364789 3.31 MB/sec execute 438 sec latency 2621.505 ms 2024-09-05T12:38:22.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:22 smithi071 ceph-mon[29845]: pgmap v332: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-09-05T12:38:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:22 smithi123 ceph-mon[34321]: pgmap v332: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-09-05T12:38:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:22 smithi174 ceph-mon[34313]: pgmap v332: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-09-05T12:38:23.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 366130 3.31 MB/sec execute 439 sec latency 17.616 ms 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 -- 172.21.15.71:0/3474156627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 msgr2=0x7f74c0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 --2- 172.21.15.71:0/3474156627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c0102ec0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009900 tx=0x7f74b002a480 comp rx=0 tx=0).stop 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 -- 172.21.15.71:0/3474156627 shutdown_connections 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 --2- 172.21.15.71:0/3474156627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c010bf00 0x7f74c010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 --2- 172.21.15.71:0/3474156627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0103400 0x7f74c010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 --2- 172.21.15.71:0/3474156627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c0102ec0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 -- 172.21.15.71:0/3474156627 >> 172.21.15.71:0/3474156627 conn(0x7f74c00777c0 msgr2=0x7f74c00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 -- 172.21.15.71:0/3474156627 shutdown_connections 2024-09-05T12:38:23.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.357+0000 7f74c5000640 1 -- 172.21.15.71:0/3474156627 wait complete. 2024-09-05T12:38:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 Processor -- start 2024-09-05T12:38:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 -- start start 2024-09-05T12:38:23.360 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0103400 0x7f74c01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c010bf00 0x7f74c019ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111d20 con 0x7f74c0103400 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111ba0 con 0x7f74c010bf00 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74c5000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0111ea0 con 0x7f74c0102ae0 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74bf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74bf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37250/0 (socket says 172.21.15.71:37250) 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74bf400640 1 -- 172.21.15.71:0/881885035 learned_addr learned my addr 172.21.15.71:0/881885035 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74bea00640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0103400 0x7f74c01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.358+0000 7f74bfe00640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c010bf00 0x7f74c019ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74bf400640 1 -- 172.21.15.71:0/881885035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c010bf00 msgr2=0x7f74c019ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74bf400640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c010bf00 0x7f74c019ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74bf400640 1 -- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0103400 msgr2=0x7f74c01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74bf400640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0103400 0x7f74c01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74bf400640 1 -- 172.21.15.71:0/881885035 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c019d1a0 con 0x7f74c0102ae0 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74bf400640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c01981b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f74b0004770 tx=0x7f74b0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74b7e00640 1 -- 172.21.15.71:0/881885035 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b003f070 con 0x7f74c0102ae0 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74b7e00640 1 -- 172.21.15.71:0/881885035 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74b0031ca0 con 0x7f74c0102ae0 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c019d430 con 0x7f74c0102ae0 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74b7e00640 1 -- 172.21.15.71:0/881885035 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0035030 con 0x7f74c0102ae0 2024-09-05T12:38:23.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.359+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01a4d20 con 0x7f74c0102ae0 2024-09-05T12:38:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.363+0000 7f74b7e00640 1 -- 172.21.15.71:0/881885035 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f74b0042070 con 0x7f74c0102ae0 2024-09-05T12:38:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.363+0000 7f74c93ff740 1 Processor -- start 2024-09-05T12:38:23.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 -- start start 2024-09-05T12:38:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 0x55aef862a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef8635030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef8635570 0x55aef8637960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aef8590100 con 0x55aef862acd0 2024-09-05T12:38:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aef85381b0 con 0x55aef862a3b0 2024-09-05T12:38:23.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74c93ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aef858c0c0 con 0x55aef8635570 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 0x55aef862a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 0x55aef862a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50628/0 (socket says 172.21.15.71:50628) 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74bea00640 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef8635570 0x55aef8637960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74bf400640 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef8635030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.364+0000 7f74bfe00640 1 -- 172.21.15.71:0/3115543459 learned_addr learned my addr 172.21.15.71:0/3115543459 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bea00640 1 -- 172.21.15.71:0/3115543459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 msgr2=0x55aef862a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bea00640 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 0x55aef862a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bea00640 1 -- 172.21.15.71:0/3115543459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 msgr2=0x55aef8635030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bea00640 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef8635030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bea00640 1 -- 172.21.15.71:0/3115543459 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aef82a3210 con 0x55aef8635570 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bfe00640 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 0x55aef862a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74bea00640 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef8635570 0x55aef8637960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00abc0 tx=0x7f74ac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbf406e7f416292b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:23.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74b4c00640 1 -- 172.21.15.71:0/3115543459 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac005220 con 0x55aef8635570 2024-09-05T12:38:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74b4c00640 1 -- 172.21.15.71:0/3115543459 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac004c80 con 0x55aef8635570 2024-09-05T12:38:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74b4c00640 1 -- 172.21.15.71:0/3115543459 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac018900 con 0x55aef8635570 2024-09-05T12:38:23.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.365+0000 7f74b7e00640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.366+0000 7f74b7e00640 1 -- 172.21.15.71:0/881885035 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f74b00eff20 con 0x7f74c0102ae0 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.366+0000 7f74bfe00640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.366+0000 7f74c93ff740 1 -- 172.21.15.71:0/3115543459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef8635570 msgr2=0x55aef8637960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.366+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef8635570 0x55aef8637960 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f74ac00abc0 tx=0x7f74ac00f790 comp rx=0 tx=0).stop 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 -- 172.21.15.71:0/3115543459 shutdown_connections 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef8635570 0x55aef8637960 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef8635030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3115543459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef862a3b0 0x55aef862a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 -- 172.21.15.71:0/3115543459 >> 172.21.15.71:0/3115543459 conn(0x55aef8626920 msgr2=0x55aef8626d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 -- 172.21.15.71:0/3115543459 shutdown_connections 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74bfe00640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009fd0 tx=0x7f74b8009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 -- 172.21.15.71:0/3115543459 wait complete. 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 -- start start 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.367+0000 7f74c93ff740 1 Processor -- start 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 0x55aef8639b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef863a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef863e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55aef8637ff0 con 0x55aef862acd0 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aef8519090 con 0x55aef8635570 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aef8638170 con 0x55aef862a3b0 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74c93ff740 10 client.0 _ll_register_callbacks cb 0x55aef856df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:23.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 0x55aef8639b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 0x55aef8639b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37276/0 (socket says 172.21.15.71:37276) 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 -- 172.21.15.71:0/3996651843 learned_addr learned my addr 172.21.15.71:0/3996651843 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bfe00640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef863e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bea00640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef863a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 -- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 msgr2=0x55aef863e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef863e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 -- 172.21.15.71:0/3996651843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 msgr2=0x55aef863a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef863a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bf400640 1 -- 172.21.15.71:0/3996651843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aef836f620 con 0x55aef862a3b0 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.368+0000 7f74bea00640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef863a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:38:23.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74bf400640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 0x55aef8639b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74b00b84b0 tx=0x7f74b0031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e6537fbb516bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74a9600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74a9600640 1 -- 172.21.15.71:0/3996651843 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0049070 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ac004c80 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74a9600640 1 -- 172.21.15.71:0/3996651843 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f74b00b86c0 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74a9600640 1 -- 172.21.15.71:0/3996651843 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f74b00bbb90 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74c93ff740 10 client.26629 fetch_fsmap learned FSMap version 48 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55aef8638050 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74a9600640 1 -- 172.21.15.71:0/3996651843 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac004c80 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.369+0000 7f74a9600640 1 -- 172.21.15.71:0/3996651843 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f74b00ed500 con 0x55aef862a3b0 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74c93ff740 10 client.26629 fetch_fsmap finished waiting for FSMap version 48 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74c93ff740 10 client.26629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74c93ff740 20 client.26629 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:23.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef8649680 0x55aef864ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74c93ff740 4 client.26629 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aef859d830 con 0x55aef8649680 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74bea00640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef8649680 0x55aef864ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74a9600640 10 client.26629 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.370+0000 7f74bea00640 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef8649680 0x55aef864ba70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601ebffb8522dc0a server_cookie=b6a7a86d17a93f0e in_seq=0 out_seq=0 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.371+0000 7f74a9600640 1 -- 172.21.15.71:0/3996651843 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aef859d830 con 0x55aef8649680 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.371+0000 7f74a9600640 10 client.26629 handle_command_reply: tid=0 2024-09-05T12:38:23.376 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.371+0000 7f74c93ff740 1 client.26629 shutdown 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.371+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 msgr2=0x55aef8639b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.371+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 0x55aef8639b30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74b00b84b0 tx=0x7f74b0031490 comp rx=0 tx=0).stop 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 shutdown_connections 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef8649680 0x55aef864ba70 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef863e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862acd0 0x55aef863a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 --2- 172.21.15.71:0/3996651843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862a3b0 0x55aef8639b30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74a9600640 0 client.26629 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 >> 172.21.15.71:0/3996651843 conn(0x55aef8626920 msgr2=0x55aef862c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 shutdown_connections 2024-09-05T12:38:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 1 -- 172.21.15.71:0/3996651843 wait complete. 2024-09-05T12:38:23.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.372+0000 7f74c93ff740 20 client.26629 trim_cache size 0 max 16384 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.388+0000 7f74c93ff740 1 Processor -- start 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 -- start start 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef863e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8651380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef86518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01aca70 con 0x55aef862a3b0 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ae5e0 con 0x55aef8635570 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74c93ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ad170 con 0x55aef862acd0 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8651380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8651380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37292/0 (socket says 172.21.15.71:37292) 2024-09-05T12:38:23.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 -- 172.21.15.71:0/2619349820 learned_addr learned my addr 172.21.15.71:0/2619349820 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bfe00640 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef86518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bf400640 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef863e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 -- 172.21.15.71:0/2619349820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 msgr2=0x55aef86518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef86518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 -- 172.21.15.71:0/2619349820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 msgr2=0x55aef863e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef863e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.389+0000 7f74bea00640 1 -- 172.21.15.71:0/2619349820 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aef864bde0 con 0x55aef862acd0 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.390+0000 7f74bea00640 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8651380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55aef8627ca0 tx=0x7f74ac00a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86b91383267e029a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:23.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.390+0000 7f74abe00640 1 -- 172.21.15.71:0/2619349820 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac005040 con 0x55aef862acd0 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.390+0000 7f74abe00640 1 -- 172.21.15.71:0/2619349820 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac030070 con 0x55aef862acd0 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.390+0000 7f74abe00640 1 -- 172.21.15.71:0/2619349820 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac018710 con 0x55aef862acd0 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 -- 172.21.15.71:0/2619349820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 msgr2=0x55aef8651380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8651380 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55aef8627ca0 tx=0x7f74ac00a630 comp rx=0 tx=0).stop 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 -- 172.21.15.71:0/2619349820 shutdown_connections 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef86518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8651380 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2619349820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef863e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 -- 172.21.15.71:0/2619349820 >> 172.21.15.71:0/2619349820 conn(0x55aef8626920 msgr2=0x55aef8626d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 -- 172.21.15.71:0/2619349820 shutdown_connections 2024-09-05T12:38:23.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.391+0000 7f74c93ff740 1 -- 172.21.15.71:0/2619349820 wait complete. 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 -- start start 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 Processor -- start 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef86405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8640b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef8641040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01adf10 con 0x55aef862a3b0 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01ad840 con 0x55aef8635570 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74c01aaa70 con 0x55aef862acd0 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.392+0000 7f74c93ff740 10 client.0 _ll_register_callbacks cb 0x7f74b00f36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef86405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8640b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef8641040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8640b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37302/0 (socket says 172.21.15.71:37302) 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bea00640 1 -- 172.21.15.71:0/2515544403 learned_addr learned my addr 172.21.15.71:0/2515544403 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 -- 172.21.15.71:0/2515544403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 msgr2=0x55aef8640b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8640b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 -- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 msgr2=0x55aef8641040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef8641040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 -- 172.21.15.71:0/2515544403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aef86b2340 con 0x55aef862a3b0 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74bf400640 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef86405c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f74b00c4790 tx=0x7f74b00ed730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91d46bf55aa83698 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.393+0000 7f74a9600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74a9600640 1 -- 172.21.15.71:0/2515544403 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b0049040 con 0x55aef862a3b0 2024-09-05T12:38:23.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74a9600640 1 -- 172.21.15.71:0/2515544403 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f74b00b6bc0 con 0x55aef862a3b0 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ac030070 con 0x55aef862a3b0 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74a9600640 1 -- 172.21.15.71:0/2515544403 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f74b00bb9b0 con 0x55aef862a3b0 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74c93ff740 10 client.16572 fetch_fsmap learned FSMap version 48 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55aef86b35a0 con 0x55aef862a3b0 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.394+0000 7f74a9600640 1 -- 172.21.15.71:0/2515544403 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74ac030070 con 0x55aef862a3b0 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74a9600640 1 -- 172.21.15.71:0/2515544403 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f74b00bb4a0 con 0x55aef862a3b0 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74c93ff740 10 client.16572 fetch_fsmap finished waiting for FSMap version 48 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74c93ff740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74c93ff740 20 client.16572 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef86b3a80 0x55aef86b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74c93ff740 4 client.16572 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aef8597b80 con 0x55aef86b3a80 2024-09-05T12:38:23.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.395+0000 7f74bea00640 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef86b3a80 0x55aef86b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.396+0000 7f74bea00640 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef86b3a80 0x55aef86b5e70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1704ade40f2516 server_cookie=cc148c7201d250c7 in_seq=0 out_seq=0 2024-09-05T12:38:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.396+0000 7f74a9600640 10 client.16572 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74a9600640 1 -- 172.21.15.71:0/2515544403 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aef8597b80 con 0x55aef86b3a80 2024-09-05T12:38:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74a9600640 10 client.16572 handle_command_reply: tid=0 2024-09-05T12:38:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 client.16572 shutdown 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 msgr2=0x55aef86405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef86405c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f74b00c4790 tx=0x7f74b00ed730 comp rx=0 tx=0).stop 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 shutdown_connections 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55aef86b3a80 0x55aef86b5e70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aef8635570 0x55aef8641040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aef862acd0 0x55aef8640b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 --2- 172.21.15.71:0/2515544403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aef862a3b0 0x55aef86405c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74a9600640 0 client.16572 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 >> 172.21.15.71:0/2515544403 conn(0x55aef8626920 msgr2=0x55aef862bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 shutdown_connections 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.434+0000 7f74c93ff740 1 -- 172.21.15.71:0/2515544403 wait complete. 2024-09-05T12:38:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c93ff740 20 client.16572 trim_cache size 0 max 16384 2024-09-05T12:38:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:38:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:38:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:38:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 msgr2=0x7f74a40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009fd0 tx=0x7f74b8009500 comp rx=0 tx=0).stop 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 msgr2=0x7f74c01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c01981b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f74b0004770 tx=0x7f74b0003c10 comp rx=0 tx=0).stop 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 shutdown_connections 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74c010bf00 0x7f74c019ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f74a40a2cf0 0x7f74a40a51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f74c0103400 0x7f74c01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 --2- 172.21.15.71:0/881885035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0102ae0 0x7f74c01981b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.435+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 >> 172.21.15.71:0/881885035 conn(0x7f74c00777c0 msgr2=0x7f74c00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.436+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 shutdown_connections 2024-09-05T12:38:23.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:23.436+0000 7f74c5000640 1 -- 172.21.15.71:0/881885035 wait complete. 2024-09-05T12:38:23.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:23 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:38:23.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:23 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:38:23.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:23 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:38:24.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 366863 3.32 MB/sec execute 440 sec latency 306.049 ms 2024-09-05T12:38:24.521 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:24 smithi071 ceph-mon[29845]: pgmap v333: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-05T12:38:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:24 smithi123 ceph-mon[34321]: pgmap v333: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-05T12:38:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:24 smithi174 ceph-mon[34313]: pgmap v333: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-09-05T12:38:25.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 367032 3.31 MB/sec execute 441 sec latency 614.545 ms 2024-09-05T12:38:26.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 367844 3.31 MB/sec execute 442 sec latency 1129.528 ms 2024-09-05T12:38:26.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:26 smithi071 ceph-mon[29845]: pgmap v334: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T12:38:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:26 smithi123 ceph-mon[34321]: pgmap v334: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T12:38:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:26 smithi174 ceph-mon[34313]: pgmap v334: 129 pgs: 129 active+clean; 124 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-09-05T12:38:27.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 369347 3.33 MB/sec execute 443 sec latency 64.677 ms 2024-09-05T12:38:28.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 370492 3.33 MB/sec execute 444 sec latency 18.038 ms 2024-09-05T12:38:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:28 smithi071 ceph-mon[29845]: pgmap v335: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-09-05T12:38:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:28 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:38:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:28 smithi123 ceph-mon[34321]: pgmap v335: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-09-05T12:38:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:28 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:38:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:28 smithi174 ceph-mon[34313]: pgmap v335: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-09-05T12:38:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:28 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:38:29.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 370575 3.32 MB/sec execute 445 sec latency 945.408 ms 2024-09-05T12:38:30.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 371389 3.32 MB/sec execute 446 sec latency 1480.133 ms 2024-09-05T12:38:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:30 smithi071 ceph-mon[29845]: pgmap v336: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-05T12:38:30.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:30 smithi123 ceph-mon[34321]: pgmap v336: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-05T12:38:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:30 smithi174 ceph-mon[34313]: pgmap v336: 129 pgs: 129 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-09-05T12:38:31.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 372867 3.34 MB/sec execute 447 sec latency 60.225 ms 2024-09-05T12:38:32.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 374006 3.34 MB/sec execute 448 sec latency 19.222 ms 2024-09-05T12:38:32.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:32 smithi071 ceph-mon[29845]: pgmap v337: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-09-05T12:38:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:32 smithi123 ceph-mon[34321]: pgmap v337: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-09-05T12:38:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:32 smithi174 ceph-mon[34313]: pgmap v337: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.8 MiB/s wr, 232 op/s 2024-09-05T12:38:33.273 INFO:tasks.workunit.client.0.smithi071.stdout: 1 374123 3.34 MB/sec execute 449 sec latency 905.884 ms 2024-09-05T12:38:34.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 374123 3.33 MB/sec execute 450 sec latency 1905.936 ms 2024-09-05T12:38:34.435 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:34 smithi071 ceph-mon[29845]: pgmap v338: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-09-05T12:38:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:34 smithi123 ceph-mon[34321]: pgmap v338: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-09-05T12:38:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:34 smithi174 ceph-mon[34313]: pgmap v338: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-09-05T12:38:34.931 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.3 2024-09-05T12:38:34.932 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:38:35.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 374123 3.32 MB/sec execute 451 sec latency 2905.978 ms 2024-09-05T12:38:35.321 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 -- 172.21.15.71:0/191849566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 msgr2=0x7fbd5410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 --2- 172.21.15.71:0/191849566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 0x7fbd5410dd20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002a480 comp rx=0 tx=0).stop 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 -- 172.21.15.71:0/191849566 shutdown_connections 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 --2- 172.21.15.71:0/191849566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd5410e260 0x7fbd54110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 --2- 172.21.15.71:0/191849566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 0x7fbd5410dd20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 --2- 172.21.15.71:0/191849566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd540745c0 0x7fbd540749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.948+0000 7fbd5a800640 1 -- 172.21.15.71:0/191849566 >> 172.21.15.71:0/191849566 conn(0x7fbd540fd740 msgr2=0x7fbd540ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:35.950 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.949+0000 7fbd5a800640 1 -- 172.21.15.71:0/191849566 shutdown_connections 2024-09-05T12:38:35.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.949+0000 7fbd5a800640 1 -- 172.21.15.71:0/191849566 wait complete. 2024-09-05T12:38:35.951 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 Processor -- start 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 -- start start 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 0x7fbd5419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 0x7fbd5419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd5410e260 0x7fbd5419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54113fb0 con 0x7fbd54074ee0 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54113e30 con 0x7fbd5410e260 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.950+0000 7fbd5a800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd54114130 con 0x7fbd540745c0 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 0x7fbd5419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 0x7fbd5419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39600/0 (socket says 172.21.15.71:39600) 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 -- 172.21.15.71:0/2952364756 learned_addr learned my addr 172.21.15.71:0/2952364756 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:35.952 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd58a00640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 0x7fbd5419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:35.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59e00640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd5410e260 0x7fbd5419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:35.953 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 -- 172.21.15.71:0/2952364756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd5410e260 msgr2=0x7fbd5419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd5410e260 0x7fbd5419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 -- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 msgr2=0x7fbd5419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 0x7fbd5419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.951+0000 7fbd59400640 1 -- 172.21.15.71:0/2952364756 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5419f440 con 0x7fbd540745c0 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.952+0000 7fbd59400640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 0x7fbd5419a490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009d50 tx=0x7fbd4800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.952+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48003b70 con 0x7fbd540745c0 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.952+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd48003d10 con 0x7fbd540745c0 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.952+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5419f730 con 0x7fbd540745c0 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.952+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48010a20 con 0x7fbd540745c0 2024-09-05T12:38:35.954 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.952+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd541a7010 con 0x7fbd540745c0 2024-09-05T12:38:35.955 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.953+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd540749a0 con 0x7fbd540745c0 2024-09-05T12:38:35.956 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.955+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbd48003ec0 con 0x7fbd540745c0 2024-09-05T12:38:35.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.957+0000 7fbd52000640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd3c0a2cf0 0x7fbd3c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:35.957 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.957+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fbd480cfef0 con 0x7fbd540745c0 2024-09-05T12:38:35.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.957+0000 7fbd58a00640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd3c0a2cf0 0x7fbd3c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:35.958 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.958+0000 7fbd58a00640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd3c0a2cf0 0x7fbd3c0a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009230 tx=0x7fbd400058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:35.961 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:35.960+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd4809c720 con 0x7fbd540745c0 2024-09-05T12:38:36.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.200+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd5419b7f0 con 0x7fbd540745c0 2024-09-05T12:38:36.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.201+0000 7fbd52000640 1 -- 172.21.15.71:0/2952364756 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11976 (secure 0 0 0) 0x7fbd480bc0e0 con 0x7fbd540745c0 2024-09-05T12:38:36.202 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:38:36.203 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":49,"btime":"2024-09-05T12:38:27:359285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:38:26.374701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:38:36.203 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 49 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd3c0a2cf0 msgr2=0x7fbd3c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd3c0a2cf0 0x7fbd3c0a51b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009230 tx=0x7fbd400058d0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 msgr2=0x7fbd5419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 0x7fbd5419a490 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009d50 tx=0x7fbd4800c6a0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 shutdown_connections 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd5410e260 0x7fbd5419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd3c0a2cf0 0x7fbd3c0a51b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd54074ee0 0x7fbd5419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 --2- 172.21.15.71:0/2952364756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd540745c0 0x7fbd5419a490 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 >> 172.21.15.71:0/2952364756 conn(0x7fbd540fd740 msgr2=0x7fbd540fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 shutdown_connections 2024-09-05T12:38:36.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:36.204+0000 7fbd5a800640 1 -- 172.21.15.71:0/2952364756 wait complete. 2024-09-05T12:38:36.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 374997 3.32 MB/sec execute 452 sec latency 3425.082 ms 2024-09-05T12:38:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 -- 172.21.15.71:0/1652196407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 msgr2=0x7fa75010e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 --2- 172.21.15.71:0/1652196407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa75010e780 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa748009cd0 tx=0x7fa74802a8f0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 -- 172.21.15.71:0/1652196407 shutdown_connections 2024-09-05T12:38:36.565 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 --2- 172.21.15.71:0/1652196407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa75010e780 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 --2- 172.21.15.71:0/1652196407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7501018b0 0x7fa750101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 --2- 172.21.15.71:0/1652196407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501078c0 0x7fa750107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 -- 172.21.15.71:0/1652196407 >> 172.21.15.71:0/1652196407 conn(0x7fa7500fd5d0 msgr2=0x7fa7500ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.564+0000 7fa74fe00640 1 -- 172.21.15.71:0/1652196407 shutdown_connections 2024-09-05T12:38:36.566 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.565+0000 7fa74fe00640 1 -- 172.21.15.71:0/1652196407 wait complete. 2024-09-05T12:38:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.566+0000 7fa74fe00640 1 Processor -- start 2024-09-05T12:38:36.567 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.566+0000 7fa74fe00640 1 -- start start 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 0x7fa7501981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa750198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7501078c0 0x7fa75019ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa750111d50 con 0x7fa7501078c0 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa750111bd0 con 0x7fa7501018b0 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74fe00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa750111ed0 con 0x7fa750102250 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 0x7fa7501981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa750198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 0x7fa7501981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35756/0 (socket says 172.21.15.71:35756) 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 -- 172.21.15.71:0/3831494563 learned_addr learned my addr 172.21.15.71:0/3831494563 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:36.569 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74f400640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7501078c0 0x7fa75019ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 -- 172.21.15.71:0/3831494563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 msgr2=0x7fa750198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa750198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 -- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7501078c0 msgr2=0x7fa75019ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7501078c0 0x7fa75019ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74ea00640 1 -- 172.21.15.71:0/3831494563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75019d1b0 con 0x7fa7501018b0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.567+0000 7fa74e000640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa750198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.568+0000 7fa74ea00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 0x7fa7501981c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ea60 tx=0x7fa74000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.568+0000 7fa754a00640 1 -- 172.21.15.71:0/3831494563 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74000cde0 con 0x7fa7501018b0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.568+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75019d4a0 con 0x7fa7501018b0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.568+0000 7fa754a00640 1 -- 172.21.15.71:0/3831494563 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7400035f0 con 0x7fa7501018b0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.568+0000 7fa754a00640 1 -- 172.21.15.71:0/3831494563 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa740010670 con 0x7fa7501018b0 2024-09-05T12:38:36.570 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.568+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7501a4d30 con 0x7fa7501018b0 2024-09-05T12:38:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.571+0000 7fa758e2b740 1 Processor -- start 2024-09-05T12:38:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.571+0000 7fa754a00640 1 -- 172.21.15.71:0/3831494563 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa7400037a0 con 0x7fa7501018b0 2024-09-05T12:38:36.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.571+0000 7fa758e2b740 1 -- start start 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4ed9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4edac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa758e2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556c4ed03100 con 0x556c4ed9d2b0 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa758e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c4ecab1b0 con 0x556c4edac980 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa758e2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c4ecff0c0 con 0x556c4ed9dbd0 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4edac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4ed9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35768/0 (socket says 172.21.15.71:35768) 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4edac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39626/0 (socket says 172.21.15.71:39626) 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 -- 172.21.15.71:0/1958163893 learned_addr learned my addr 172.21.15.71:0/1958163893 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 -- 172.21.15.71:0/1958163893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 msgr2=0x556c4edac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4edac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 -- 172.21.15.71:0/1958163893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 msgr2=0x556c4ed9d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4ed9d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74e000640 1 -- 172.21.15.71:0/1958163893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4ea16210 con 0x556c4edac980 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.572+0000 7fa74ea00640 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4edac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa754a00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa7340a2cf0 0x7fa7340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa754a00640 1 -- 172.21.15.71:0/3831494563 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fa7400cfab0 con 0x7fa7501018b0 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa74e000640 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaed70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0114f0 tx=0x7fa73c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dd90c9fd2343894 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa74f400640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa7340a2cf0 0x7fa7340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa73be00640 1 -- 172.21.15.71:0/1958163893 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01e070 con 0x556c4edac980 2024-09-05T12:38:36.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa73be00640 1 -- 172.21.15.71:0/1958163893 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c0035f0 con 0x556c4edac980 2024-09-05T12:38:36.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.573+0000 7fa73be00640 1 -- 172.21.15.71:0/1958163893 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00f820 con 0x556c4edac980 2024-09-05T12:38:36.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 -- 172.21.15.71:0/1958163893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 msgr2=0x556c4edaed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaed70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0114f0 tx=0x7fa73c0119b0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa74f400640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa7340a2cf0 0x7fa7340a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa748004af0 tx=0x7fa748005d80 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 -- 172.21.15.71:0/1958163893 shutdown_connections 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaed70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4edac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1958163893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4ed9d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 -- 172.21.15.71:0/1958163893 >> 172.21.15.71:0/1958163893 conn(0x556c4ed99920 msgr2=0x556c4ed99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.574+0000 7fa758e2b740 1 -- 172.21.15.71:0/1958163893 shutdown_connections 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.575+0000 7fa758e2b740 1 -- 172.21.15.71:0/1958163893 wait complete. 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.575+0000 7fa758e2b740 1 -- start start 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.575+0000 7fa758e2b740 1 Processor -- start 2024-09-05T12:38:36.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4eda98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4eda9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556c4edaf550 con 0x556c4ed9d2b0 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c4ec8c090 con 0x556c4edac980 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c4edaf6d0 con 0x556c4ed9dbd0 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa758e2b740 10 client.0 _ll_register_callbacks cb 0x556c4ece0f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4eda98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74e000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4eda9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4eda98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39922/0 (socket says 172.21.15.71:39922) 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74ea00640 1 -- 172.21.15.71:0/1556694476 learned_addr learned my addr 172.21.15.71:0/1556694476 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:36.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74e000640 1 -- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 msgr2=0x556c4edaa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74e000640 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74e000640 1 -- 172.21.15.71:0/1556694476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 msgr2=0x556c4eda98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74e000640 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4eda98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.576+0000 7fa74e000640 1 -- 172.21.15.71:0/1556694476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4eae2620 con 0x556c4ed9dbd0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa74e000640 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4eda9e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa73c011eb0 tx=0x7fa73c0041e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ff10f1a166b5cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa738c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa738c00640 1 -- 172.21.15.71:0/1556694476 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01e040 con 0x556c4ed9dbd0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa73c0035f0 con 0x556c4ed9dbd0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa738c00640 1 -- 172.21.15.71:0/1556694476 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fa73c005040 con 0x556c4ed9dbd0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa738c00640 1 -- 172.21.15.71:0/1556694476 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa73c032050 con 0x556c4ed9dbd0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa758e2b740 10 client.26653 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c4edbb240 con 0x556c4ed9dbd0 2024-09-05T12:38:36.581 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.577+0000 7fa738c00640 1 -- 172.21.15.71:0/1556694476 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa73c0035f0 con 0x556c4ed9dbd0 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa738c00640 1 -- 172.21.15.71:0/1556694476 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fa73c021030 con 0x556c4ed9dbd0 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa758e2b740 10 client.26653 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa758e2b740 10 client.26653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa758e2b740 20 client.26653 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4edbc6b0 0x556c4edbeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa758e2b740 4 client.26653 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c4ed10830 con 0x556c4edbc6b0 2024-09-05T12:38:36.582 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.578+0000 7fa74ea00640 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4edbc6b0 0x556c4edbeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.579+0000 7fa74ea00640 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4edbc6b0 0x556c4edbeaa0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a37e68a0f217566f server_cookie=7e59947cdce2acfb in_seq=0 out_seq=0 2024-09-05T12:38:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.579+0000 7fa738c00640 10 client.26653 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:36.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.579+0000 7fa738c00640 1 -- 172.21.15.71:0/1556694476 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c4ed10830 con 0x556c4edbc6b0 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.579+0000 7fa738c00640 10 client.26653 handle_command_reply: tid=0 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.579+0000 7fa758e2b740 1 client.26653 shutdown 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 msgr2=0x556c4eda9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4eda9e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa73c011eb0 tx=0x7fa73c0041e0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 shutdown_connections 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4edbc6b0 0x556c4edbeaa0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4edaa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9dbd0 0x556c4eda9e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1556694476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9d2b0 0x556c4eda98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa738c00640 0 client.26653 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 >> 172.21.15.71:0/1556694476 conn(0x556c4ed99920 msgr2=0x556c4ed9f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 shutdown_connections 2024-09-05T12:38:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 1 -- 172.21.15.71:0/1556694476 wait complete. 2024-09-05T12:38:36.585 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.580+0000 7fa758e2b740 20 client.26653 trim_cache size 0 max 16384 2024-09-05T12:38:36.585 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:36 smithi071 ceph-mon[29845]: pgmap v339: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-05T12:38:36.585 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:36 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2952364756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.596+0000 7fa758e2b740 1 Processor -- start 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.596+0000 7fa758e2b740 1 -- start start 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9d2b0 0x556c4edc4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edc47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4eda9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa758e2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501acc60 con 0x556c4ed9dbd0 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa758e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ae830 con 0x556c4edac980 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa758e2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ad3c0 con 0x556c4ed9d2b0 2024-09-05T12:38:36.598 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9d2b0 0x556c4edc4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edc47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9d2b0 0x556c4edc4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39646/0 (socket says 172.21.15.71:39646) 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74ea00640 1 -- 172.21.15.71:0/1113972588 learned_addr learned my addr 172.21.15.71:0/1113972588 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74f400640 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4eda9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74f400640 1 -- 172.21.15.71:0/1113972588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9d2b0 msgr2=0x556c4edc4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74f400640 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9d2b0 0x556c4edc4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74f400640 1 -- 172.21.15.71:0/1113972588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 msgr2=0x556c4edc47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74f400640 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edc47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.599 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.597+0000 7fa74f400640 1 -- 172.21.15.71:0/1113972588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c4edbf040 con 0x556c4edac980 2024-09-05T12:38:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.598+0000 7fa74f400640 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4eda9890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa74802ae10 tx=0x7fa748041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f15204f4bbf542 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.598+0000 7fa73b400640 1 -- 172.21.15.71:0/1113972588 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748003d00 con 0x556c4edac980 2024-09-05T12:38:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.598+0000 7fa73b400640 1 -- 172.21.15.71:0/1113972588 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa748036050 con 0x556c4edac980 2024-09-05T12:38:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.598+0000 7fa73b400640 1 -- 172.21.15.71:0/1113972588 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa748004620 con 0x556c4edac980 2024-09-05T12:38:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 -- 172.21.15.71:0/1113972588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 msgr2=0x556c4eda9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4eda9890 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa74802ae10 tx=0x7fa748041000 comp rx=0 tx=0).stop 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 -- 172.21.15.71:0/1113972588 shutdown_connections 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4edac980 0x556c4eda9890 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edc47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 --2- 172.21.15.71:0/1113972588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4ed9d2b0 0x556c4edc4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 -- 172.21.15.71:0/1113972588 >> 172.21.15.71:0/1113972588 conn(0x556c4ed99920 msgr2=0x556c4eda4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 -- 172.21.15.71:0/1113972588 shutdown_connections 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 -- 172.21.15.71:0/1113972588 wait complete. 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.599+0000 7fa758e2b740 1 -- start start 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 Processor -- start 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 0x556c4edb4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edb5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4edac980 0x556c4edb0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ae160 con 0x556c4ed9dbd0 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501ada90 con 0x556c4ed9d2b0 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7501aac60 con 0x556c4edac980 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa758e2b740 10 client.0 _ll_register_callbacks cb 0x7fa73c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 0x556c4edb4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa74e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edb5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa74ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 0x556c4edb4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35788/0 (socket says 172.21.15.71:35788) 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.600+0000 7fa74ea00640 1 -- 172.21.15.71:0/3781642169 learned_addr learned my addr 172.21.15.71:0/3781642169 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74f400640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4edac980 0x556c4edb0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74ea00640 1 -- 172.21.15.71:0/3781642169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4edac980 msgr2=0x556c4edb0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74ea00640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4edac980 0x556c4edb0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74ea00640 1 -- 172.21.15.71:0/3781642169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 msgr2=0x556c4edb5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:36.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74ea00640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edb5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74ea00640 1 -- 172.21.15.71:0/3781642169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c4ed9da00 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74f400640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4edac980 0x556c4edb0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa74ea00640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 0x556c4edb4d30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7400985b0 tx=0x7fa740005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0fa3ad23471396d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa738c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa738c00640 1 -- 172.21.15.71:0/3781642169 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74009b540 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa738c00640 1 -- 172.21.15.71:0/3781642169 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fa740098250 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa748036050 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa738c00640 1 -- 172.21.15.71:0/3781642169 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fa74009c2e0 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa758e2b740 10 client.26636 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.601+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556c4ee26cf0 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa738c00640 1 -- 172.21.15.71:0/3781642169 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa748036050 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa738c00640 1 -- 172.21.15.71:0/3781642169 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fa7400cd090 con 0x556c4ed9d2b0 2024-09-05T12:38:36.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa758e2b740 10 client.26636 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa758e2b740 10 client.26636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa758e2b740 20 client.26636 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa758e2b740 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4ee27220 0x556c4ee29610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa758e2b740 4 client.26636 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:38:36.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.602+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c4ed0ab80 con 0x556c4ee27220 2024-09-05T12:38:36.618 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.618+0000 7fa74e000640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4ee27220 0x556c4ee29610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.619+0000 7fa74e000640 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4ee27220 0x556c4ee29610 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60054db1f84bf48b server_cookie=e051881d903c2942 in_seq=0 out_seq=0 2024-09-05T12:38:36.619 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.619+0000 7fa738c00640 10 client.26636 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.648+0000 7fa738c00640 1 -- 172.21.15.71:0/3781642169 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c4ed0ab80 con 0x556c4ee27220 2024-09-05T12:38:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.648+0000 7fa738c00640 10 client.26636 handle_command_reply: tid=0 2024-09-05T12:38:36.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.649+0000 7fa758e2b740 1 client.26636 shutdown 2024-09-05T12:38:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.649+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 msgr2=0x556c4edb4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.649+0000 7fa758e2b740 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 0x556c4edb4d30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7400985b0 tx=0x7fa740005f10 comp rx=0 tx=0).stop 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 shutdown_connections 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556c4ee27220 0x556c4ee29610 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c4edac980 0x556c4edb0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c4ed9dbd0 0x556c4edb5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 --2- 172.21.15.71:0/3781642169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c4ed9d2b0 0x556c4edb4d30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa738c00640 0 client.26636 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 >> 172.21.15.71:0/3781642169 conn(0x556c4ed99920 msgr2=0x556c4ed981a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 shutdown_connections 2024-09-05T12:38:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 1 -- 172.21.15.71:0/3781642169 wait complete. 2024-09-05T12:38:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.650+0000 7fa758e2b740 20 client.26636 trim_cache size 0 max 16384 2024-09-05T12:38:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:38:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:38:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:38:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:38:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.651+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa7340a2cf0 msgr2=0x7fa7340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.651+0000 7fa74fe00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa7340a2cf0 0x7fa7340a51b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa748004af0 tx=0x7fa748005d80 comp rx=0 tx=0).stop 2024-09-05T12:38:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.651+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 msgr2=0x7fa7501981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.651+0000 7fa74fe00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 0x7fa7501981c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa74000ea60 tx=0x7fa74000ef20 comp rx=0 tx=0).stop 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.652+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 shutdown_connections 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.652+0000 7fa74fe00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa7340a2cf0 0x7fa7340a51b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.652+0000 7fa74fe00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7501078c0 0x7fa75019ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.652+0000 7fa74fe00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa750102250 0x7fa750198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.652+0000 7fa74fe00640 1 --2- 172.21.15.71:0/3831494563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7501018b0 0x7fa7501981c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.652+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 >> 172.21.15.71:0/3831494563 conn(0x7fa7500fd5d0 msgr2=0x7fa7500fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.653+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 shutdown_connections 2024-09-05T12:38:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:36.653+0000 7fa74fe00640 1 -- 172.21.15.71:0/3831494563 wait complete. 2024-09-05T12:38:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:36 smithi123 ceph-mon[34321]: pgmap v339: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-05T12:38:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:36 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2952364756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:38:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:36 smithi174 ceph-mon[34313]: pgmap v339: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-05T12:38:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:36 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2952364756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:38:37.114 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 139.3 --await-for 45.0' 2024-09-05T12:38:37.114 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 139.3 --await-for 45.0 2024-09-05T12:38:37.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 376432 3.34 MB/sec execute 453 sec latency 34.253 ms 2024-09-05T12:38:37.577 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:38:38.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 -- 172.21.15.71:0/947652094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 msgr2=0x7f570010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 --2- 172.21.15.71:0/947652094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570010e460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009ca0 tx=0x7f56f802a950 comp rx=0 tx=0).stop 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 -- 172.21.15.71:0/947652094 shutdown_connections 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 --2- 172.21.15.71:0/947652094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570010e460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 --2- 172.21.15.71:0/947652094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57001034b0 0x7f570010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 --2- 172.21.15.71:0/947652094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5700102b90 0x7f5700102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 -- 172.21.15.71:0/947652094 >> 172.21.15.71:0/947652094 conn(0x7f57000fc610 msgr2=0x7f57000fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 -- 172.21.15.71:0/947652094 shutdown_connections 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.207+0000 7f5704a00640 1 -- 172.21.15.71:0/947652094 wait complete. 2024-09-05T12:38:38.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 Processor -- start 2024-09-05T12:38:38.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 -- start start 2024-09-05T12:38:38.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 0x7f5700198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57001034b0 0x7f5700198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570019cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5700111dd0 con 0x7f57001034b0 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5700111c50 con 0x7f570010bfc0 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.208+0000 7f5704a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5700111f50 con 0x7f5700102b90 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 0x7f5700198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 0x7f5700198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39662/0 (socket says 172.21.15.71:39662) 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57001034b0 0x7f5700198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 -- 172.21.15.71:0/3529781487 learned_addr learned my addr 172.21.15.71:0/3529781487 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56ff200640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570019cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 -- 172.21.15.71:0/3529781487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 msgr2=0x7f570019cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570019cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 -- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57001034b0 msgr2=0x7f5700198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.212 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57001034b0 0x7f5700198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 -- 172.21.15.71:0/3529781487 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f570019d1e0 con 0x7f5700102b90 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56ff200640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56fe800640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 0x7f5700198230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f56f000e9f0 tx=0x7f56f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.209+0000 7f56f7400640 1 -- 172.21.15.71:0/3529781487 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f000cd80 con 0x7f5700102b90 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.210+0000 7f56f7400640 1 -- 172.21.15.71:0/3529781487 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f0010910 con 0x7f5700102b90 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.210+0000 7f56f7400640 1 -- 172.21.15.71:0/3529781487 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0018990 con 0x7f5700102b90 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.210+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f570019d4d0 con 0x7f5700102b90 2024-09-05T12:38:38.213 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.210+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57001a4db0 con 0x7f5700102b90 2024-09-05T12:38:38.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f56f7400640 1 -- 172.21.15.71:0/3529781487 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f56f0010ac0 con 0x7f5700102b90 2024-09-05T12:38:38.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 Processor -- start 2024-09-05T12:38:38.214 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 -- start start 2024-09-05T12:38:38.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1370610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1372f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b6e12c7530 con 0x55b6e1361d20 2024-09-05T12:38:38.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.213+0000 7f5708024740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6e126e860 con 0x55b6e1370b50 2024-09-05T12:38:38.215 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f5708024740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6e12c34f0 con 0x55b6e1361400 2024-09-05T12:38:38.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56ff200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56fe800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1370610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56ff200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13617e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39678/0 (socket says 172.21.15.71:39678) 2024-09-05T12:38:38.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56ff200640 1 -- 172.21.15.71:0/1388140293 learned_addr learned my addr 172.21.15.71:0/1388140293 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:38.216 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56fde00640 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1372f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56ff200640 1 -- 172.21.15.71:0/1388140293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 msgr2=0x55b6e1372f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56ff200640 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1372f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.214+0000 7f56ff200640 1 -- 172.21.15.71:0/1388140293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 msgr2=0x55b6e1370610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.215+0000 7f56ff200640 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1370610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.215+0000 7f56ff200640 1 -- 172.21.15.71:0/1388140293 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e1038b90 con 0x55b6e1361400 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.215+0000 7f56fe800640 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1370610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.215+0000 7f56f7400640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f56e40a2d70 0x7f56e40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.215+0000 7f56f7400640 1 -- 172.21.15.71:0/3529781487 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f56f00cfab0 con 0x7f5700102b90 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.215+0000 7f56fe800640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f56e40a2d70 0x7f56e40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.216+0000 7f56ff200640 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13617e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009ca0 tx=0x7f56f8005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f2526ee03aecf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.216+0000 7f56eb400640 1 -- 172.21.15.71:0/1388140293 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8048070 con 0x55b6e1361400 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.216+0000 7f56eb400640 1 -- 172.21.15.71:0/1388140293 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f8003e50 con 0x55b6e1361400 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.216+0000 7f56eb400640 1 -- 172.21.15.71:0/1388140293 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8034040 con 0x55b6e1361400 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 -- 172.21.15.71:0/1388140293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 msgr2=0x55b6e13617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f56fe800640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f56e40a2d70 0x7f56e40a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55b6e1362480 tx=0x7f56f00d3560 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13617e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009ca0 tx=0x7f56f8005f80 comp rx=0 tx=0).stop 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 -- 172.21.15.71:0/1388140293 shutdown_connections 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1372f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1370610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 --2- 172.21.15.71:0/1388140293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13617e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 -- 172.21.15.71:0/1388140293 >> 172.21.15.71:0/1388140293 conn(0x55b6e135daf0 msgr2=0x55b6e135ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:38.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 -- 172.21.15.71:0/1388140293 shutdown_connections 2024-09-05T12:38:38.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.217+0000 7f5708024740 1 -- 172.21.15.71:0/1388140293 wait complete. 2024-09-05T12:38:38.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 -- start start 2024-09-05T12:38:38.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 Processor -- start 2024-09-05T12:38:38.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361400 0x55b6e136db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e136e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1370b50 0x55b6e136e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b6e1373720 con 0x55b6e1361400 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6e1281e60 con 0x55b6e1361d20 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6e13738a0 con 0x55b6e1370b50 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f5708024740 10 client.0 _ll_register_callbacks cb 0x55b6e1286cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.218+0000 7f56fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e136e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e136e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35804/0 (socket says 172.21.15.71:35804) 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56ff200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361400 0x55b6e136db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 -- 172.21.15.71:0/2265456896 learned_addr learned my addr 172.21.15.71:0/2265456896 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56ff200640 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1370b50 0x55b6e136e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 -- 172.21.15.71:0/2265456896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1370b50 msgr2=0x55b6e136e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1370b50 0x55b6e136e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 -- 172.21.15.71:0/2265456896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361400 msgr2=0x55b6e136db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361400 0x55b6e136db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 -- 172.21.15.71:0/2265456896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e128d990 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56fde00640 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e136e060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f56ec012700 tx=0x7f56ec012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c4bf4faec52966a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.219+0000 7f56dfe00640 1 -- 172.21.15.71:0/2265456896 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec004050 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f8003e50 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f56dfe00640 1 -- 172.21.15.71:0/2265456896 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f56ec0041f0 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f56dfe00640 1 -- 172.21.15.71:0/2265456896 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f56ec025050 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f5708024740 10 client.26648 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b6e137f4c0 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f56dfe00640 1 -- 172.21.15.71:0/2265456896 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f8003e50 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f56dfe00640 1 -- 172.21.15.71:0/2265456896 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f56ec003a30 con 0x55b6e1361d20 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f5708024740 10 client.26648 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:38.224 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f5708024740 10 client.26648 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.220+0000 7f5708024740 20 client.26648 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.221+0000 7f5708024740 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13808d0 0x55b6e1382d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.221+0000 7f5708024740 4 client.26648 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.221+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6e126a310 con 0x55b6e13808d0 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.221+0000 7f56ff200640 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13808d0 0x55b6e1382d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.221+0000 7f56ff200640 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13808d0 0x55b6e1382d30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24de47a0671d110 server_cookie=717f340ce669b12 in_seq=0 out_seq=0 2024-09-05T12:38:38.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.221+0000 7f56dfe00640 10 client.26648 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:38.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.222+0000 7f56dfe00640 1 -- 172.21.15.71:0/2265456896 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6e126a310 con 0x55b6e13808d0 2024-09-05T12:38:38.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.222+0000 7f56dfe00640 10 client.26648 handle_command_reply: tid=0 2024-09-05T12:38:38.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.222+0000 7f5708024740 1 client.26648 shutdown 2024-09-05T12:38:38.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 msgr2=0x55b6e136e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e136e060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f56ec012700 tx=0x7f56ec012bc0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.226 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 shutdown_connections 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13808d0 0x55b6e1382d30 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1370b50 0x55b6e136e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e136e060 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 --2- 172.21.15.71:0/2265456896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361400 0x55b6e136db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 >> 172.21.15.71:0/2265456896 conn(0x55b6e135daf0 msgr2=0x55b6e135cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f56dfe00640 0 client.26648 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 shutdown_connections 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 1 -- 172.21.15.71:0/2265456896 wait complete. 2024-09-05T12:38:38.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.223+0000 7f5708024740 20 client.26648 trim_cache size 0 max 16384 2024-09-05T12:38:38.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 Processor -- start 2024-09-05T12:38:38.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 -- start start 2024-09-05T12:38:38.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e135cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1388640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1388b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57001acbd0 con 0x55b6e1361d20 2024-09-05T12:38:38.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ae7a0 con 0x55b6e1370b50 2024-09-05T12:38:38.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.244+0000 7f5708024740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ad330 con 0x55b6e1361400 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1388640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1388640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39976/0 (socket says 172.21.15.71:39976) 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 -- 172.21.15.71:0/3289521106 learned_addr learned my addr 172.21.15.71:0/3289521106 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56ff200640 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e135cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56ff200640 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1388b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 -- 172.21.15.71:0/3289521106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 msgr2=0x55b6e135cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e135cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.248 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 -- 172.21.15.71:0/3289521106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 msgr2=0x55b6e1388b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1388b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 -- 172.21.15.71:0/3289521106 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6e1359530 con 0x55b6e1361d20 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56ff200640 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1388b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.245+0000 7f56fde00640 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1388640 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f56ec003dd0 tx=0x7f56ec010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4998b37669fa944 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.246+0000 7f56eaa00640 1 -- 172.21.15.71:0/3289521106 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec00f920 con 0x55b6e1361d20 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.246+0000 7f56eaa00640 1 -- 172.21.15.71:0/3289521106 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ec02e070 con 0x55b6e1361d20 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.246+0000 7f56eaa00640 1 -- 172.21.15.71:0/3289521106 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec005580 con 0x55b6e1361d20 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 -- 172.21.15.71:0/3289521106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 msgr2=0x55b6e1388640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1388640 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f56ec003dd0 tx=0x7f56ec010690 comp rx=0 tx=0).stop 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 -- 172.21.15.71:0/3289521106 shutdown_connections 2024-09-05T12:38:38.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1370b50 0x55b6e1388b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1361d20 0x55b6e1388640 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 --2- 172.21.15.71:0/3289521106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e135cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 -- 172.21.15.71:0/3289521106 >> 172.21.15.71:0/3289521106 conn(0x55b6e135daf0 msgr2=0x55b6e135c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:38.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 -- 172.21.15.71:0/3289521106 shutdown_connections 2024-09-05T12:38:38.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 -- 172.21.15.71:0/3289521106 wait complete. 2024-09-05T12:38:38.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 -- start start 2024-09-05T12:38:38.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.247+0000 7f5708024740 1 Processor -- start 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e1375ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1370b50 0x55b6e1376000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ae0d0 con 0x55b6e1370b50 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57001ada00 con 0x55b6e1361d20 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57001aabd0 con 0x55b6e1361400 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f5708024740 10 client.0 _ll_register_callbacks cb 0x7f56ec009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f56ff200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f56fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e1375ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f56ff200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13792d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:39710/0 (socket says 172.21.15.71:39710) 2024-09-05T12:38:38.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f56fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e1375ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:35818/0 (socket says 172.21.15.71:35818) 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f56ff200640 1 -- 172.21.15.71:0/164941555 learned_addr learned my addr 172.21.15.71:0/164941555 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.248+0000 7f56ff200640 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1370b50 0x55b6e1376000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56fde00640 1 -- 172.21.15.71:0/164941555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 msgr2=0x55b6e13792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56fde00640 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13792d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56fde00640 1 -- 172.21.15.71:0/164941555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1370b50 msgr2=0x55b6e1376000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56fde00640 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1370b50 0x55b6e1376000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56fde00640 1 -- 172.21.15.71:0/164941555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6e1376850 con 0x55b6e1361d20 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56fde00640 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e1375ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55b6e1361300 tx=0x7f56ec005380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b661fad8dee047 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56dfe00640 1 -- 172.21.15.71:0/164941555 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ec036020 con 0x55b6e1361d20 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ec02e070 con 0x55b6e1361d20 2024-09-05T12:38:38.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.249+0000 7f56dfe00640 1 -- 172.21.15.71:0/164941555 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f56ec022040 con 0x55b6e1361d20 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f56dfe00640 1 -- 172.21.15.71:0/164941555 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f56ec00f920 con 0x55b6e1361d20 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 10 client.26654 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55b6e13ec630 con 0x55b6e1361d20 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f56dfe00640 1 -- 172.21.15.71:0/164941555 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ec02e070 con 0x55b6e1361d20 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f56dfe00640 1 -- 172.21.15.71:0/164941555 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f56ec003a00 con 0x55b6e1361d20 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 10 client.26654 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 10 client.26654 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 20 client.26654 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13ecd60 0x55b6e13ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 4 client.26654 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.3, "await_for": 45.0}] 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.3, "await_for": 45.0}) -- 0x55b6e12d4c60 con 0x55b6e13ecd60 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.250+0000 7f56ff200640 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13ecd60 0x55b6e13ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.251+0000 7f56ff200640 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13ecd60 0x55b6e13ef150 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d50f27ba57c4e0a server_cookie=2390025a78d5a570 in_seq=0 out_seq=0 2024-09-05T12:38:38.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.251+0000 7f56dfe00640 10 client.26654 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:38.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.34 MB/sec execute 454 sec latency 15.282 ms 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f56dfe00640 1 -- 172.21.15.71:0/164941555 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b6e12d4c60 con 0x55b6e13ecd60 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f56dfe00640 10 client.26654 handle_command_reply: tid=0 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 client.26654 shutdown 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 msgr2=0x55b6e1375ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e1375ac0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55b6e1361300 tx=0x7f56ec005380 comp rx=0 tx=0).stop 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 shutdown_connections 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b6e13ecd60 0x55b6e13ef150 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b6e1370b50 0x55b6e1376000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f56dfe00640 0 client.26654 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:38.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6e1361d20 0x55b6e1375ac0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 --2- 172.21.15.71:0/164941555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6e1361400 0x55b6e13792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.306+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 >> 172.21.15.71:0/164941555 conn(0x55b6e135daf0 msgr2=0x55b6e136e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.307+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 shutdown_connections 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.307+0000 7f5708024740 1 -- 172.21.15.71:0/164941555 wait complete. 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.307+0000 7f5708024740 20 client.26654 trim_cache size 0 max 16384 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 49, 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:38:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 34, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "fed88741": { 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 34, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 139.3, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:38:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:38.311 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f56e40a2d70 msgr2=0x7f56e40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f56e40a2d70 0x7f56e40a5230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55b6e1362480 tx=0x7f56f00d3560 comp rx=0 tx=0).stop 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 msgr2=0x7f5700198230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 0x7f5700198230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f56f000e9f0 tx=0x7f56f000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 shutdown_connections 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f570010bfc0 0x7f570019cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f56e40a2d70 0x7f56e40a5230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57001034b0 0x7f5700198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 --2- 172.21.15.71:0/3529781487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5700102b90 0x7f5700198230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 >> 172.21.15.71:0/3529781487 conn(0x7f57000fc610 msgr2=0x7f57000fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 shutdown_connections 2024-09-05T12:38:38.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:38.308+0000 7f5704a00640 1 -- 172.21.15.71:0/3529781487 wait complete. 2024-09-05T12:38:38.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:38 smithi071 ceph-mon[29845]: pgmap v340: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-09-05T12:38:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:38 smithi123 ceph-mon[34321]: pgmap v340: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-09-05T12:38:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:38 smithi174 ceph-mon[34313]: pgmap v340: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-09-05T12:38:39.223 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fed88741', quiesce took 4.3 seconds. Will release after: 15.0 2024-09-05T12:38:39.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.33 MB/sec execute 455 sec latency 1009.309 ms 2024-09-05T12:38:40.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.33 MB/sec execute 456 sec latency 2009.431 ms 2024-09-05T12:38:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:40 smithi071 ceph-mon[29845]: pgmap v341: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-05T12:38:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:40 smithi123 ceph-mon[34321]: pgmap v341: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-05T12:38:40.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:40 smithi174 ceph-mon[34313]: pgmap v341: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-05T12:38:41.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.32 MB/sec execute 457 sec latency 3009.561 ms 2024-09-05T12:38:42.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.31 MB/sec execute 458 sec latency 4009.653 ms 2024-09-05T12:38:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:42 smithi071 ceph-mon[29845]: pgmap v342: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-05T12:38:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:42 smithi123 ceph-mon[34321]: pgmap v342: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-05T12:38:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:42 smithi174 ceph-mon[34313]: pgmap v342: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-05T12:38:43.274 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.31 MB/sec execute 459 sec latency 5009.795 ms 2024-09-05T12:38:44.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.30 MB/sec execute 460 sec latency 6009.882 ms 2024-09-05T12:38:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:44 smithi071 ceph-mon[29845]: pgmap v343: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-05T12:38:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:44 smithi123 ceph-mon[34321]: pgmap v343: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-05T12:38:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:44 smithi174 ceph-mon[34313]: pgmap v343: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-09-05T12:38:45.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.29 MB/sec execute 461 sec latency 7009.989 ms 2024-09-05T12:38:46.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.28 MB/sec execute 462 sec latency 8010.089 ms 2024-09-05T12:38:46.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:46 smithi071 ceph-mon[29845]: pgmap v344: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-09-05T12:38:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:46 smithi123 ceph-mon[34321]: pgmap v344: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-09-05T12:38:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:46 smithi174 ceph-mon[34313]: pgmap v344: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-09-05T12:38:47.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.28 MB/sec execute 463 sec latency 9010.214 ms 2024-09-05T12:38:48.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.27 MB/sec execute 464 sec latency 10010.324 ms 2024-09-05T12:38:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:48 smithi123 ceph-mon[34321]: pgmap v345: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-09-05T12:38:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:48 smithi174 ceph-mon[34313]: pgmap v345: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-09-05T12:38:48.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:48 smithi071 ceph-mon[29845]: pgmap v345: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-09-05T12:38:49.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.26 MB/sec execute 465 sec latency 11010.450 ms 2024-09-05T12:38:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 -- 172.21.15.71:0/3062635491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190076720 msgr2=0x7f9190076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 --2- 172.21.15.71:0/3062635491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190076720 0x7f9190076b00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002a410 comp rx=0 tx=0).stop 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 -- 172.21.15.71:0/3062635491 shutdown_connections 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 --2- 172.21.15.71:0/3062635491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9190112980 0x7f9190114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 --2- 172.21.15.71:0/3062635491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9190077040 0x7f9190112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 --2- 172.21.15.71:0/3062635491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190076720 0x7f9190076b00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 -- 172.21.15.71:0/3062635491 >> 172.21.15.71:0/3062635491 conn(0x7f91900ff9e0 msgr2=0x7f9190101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 -- 172.21.15.71:0/3062635491 shutdown_connections 2024-09-05T12:38:49.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.775+0000 7f9196c00640 1 -- 172.21.15.71:0/3062635491 wait complete. 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.776+0000 7f9196c00640 1 Processor -- start 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.776+0000 7f9196c00640 1 -- start start 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9196c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9190076720 0x7f91901a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9196c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9190077040 0x7f91901a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9196c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 0x7f919019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9196c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9190118550 con 0x7f9190112980 2024-09-05T12:38:49.779 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9196c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91901183d0 con 0x7f9190077040 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9196c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91901186d0 con 0x7f9190076720 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 0x7f919019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 0x7f919019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41582/0 (socket says 172.21.15.71:41582) 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 -- 172.21.15.71:0/782809929 learned_addr learned my addr 172.21.15.71:0/782809929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f918fe00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9190076720 0x7f91901a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f918f400640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9190077040 0x7f91901a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 -- 172.21.15.71:0/782809929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9190076720 msgr2=0x7f91901a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9190076720 0x7f91901a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 -- 172.21.15.71:0/782809929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9190077040 msgr2=0x7f91901a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9190077040 0x7f91901a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.777+0000 7f9194c00640 1 -- 172.21.15.71:0/782809929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919019ef70 con 0x7f9190112980 2024-09-05T12:38:49.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.778+0000 7f9194c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 0x7f919019e7a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f918800eb40 tx=0x7f918800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.778+0000 7f918cc00640 1 -- 172.21.15.71:0/782809929 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188004b20 con 0x7f9190112980 2024-09-05T12:38:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.778+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919019f260 con 0x7f9190112980 2024-09-05T12:38:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.778+0000 7f918cc00640 1 -- 172.21.15.71:0/782809929 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9188004cc0 con 0x7f9190112980 2024-09-05T12:38:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.778+0000 7f918cc00640 1 -- 172.21.15.71:0/782809929 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188010850 con 0x7f9190112980 2024-09-05T12:38:49.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.778+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901ab480 con 0x7f9190112980 2024-09-05T12:38:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.781+0000 7f9199c60740 1 Processor -- start 2024-09-05T12:38:49.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.781+0000 7f918cc00640 1 -- 172.21.15.71:0/782809929 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9188005020 con 0x7f9190112980 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.781+0000 7f9199c60740 1 -- start start 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f9199c60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f9199c60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe47b00 0x561f7fe51e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f9199c60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe523a0 0x561f7fe54790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f9199c60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561f7fdad100 con 0x561f7fe471e0 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f9199c60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f7fd551b0 con 0x561f7fe523a0 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f9199c60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f7fda90c0 con 0x561f7fe47b00 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f918f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f918fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe47b00 0x561f7fe51e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f918f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41596/0 (socket says 172.21.15.71:41596) 2024-09-05T12:38:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f918f400640 1 -- 172.21.15.71:0/2348317027 learned_addr learned my addr 172.21.15.71:0/2348317027 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:49.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.782+0000 7f918f400640 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe523a0 0x561f7fe54790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918f400640 1 -- 172.21.15.71:0/2348317027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe47b00 msgr2=0x561f7fe51e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918f400640 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe47b00 0x561f7fe51e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918f400640 1 -- 172.21.15.71:0/2348317027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe523a0 msgr2=0x561f7fe54790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918f400640 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe523a0 0x561f7fe54790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918f400640 1 -- 172.21.15.71:0/2348317027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7fac0210 con 0x561f7fe471e0 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918f400640 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe475c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f917c009fc0 tx=0x7f917c009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564b72cb8d8eb070 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918cc00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91740a2e20 0x7f91740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f9185200640 1 -- 172.21.15.71:0/2348317027 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c004050 con 0x561f7fe471e0 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f9185200640 1 -- 172.21.15.71:0/2348317027 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917c005500 con 0x561f7fe471e0 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f9185200640 1 -- 172.21.15.71:0/2348317027 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917c004d00 con 0x561f7fe471e0 2024-09-05T12:38:49.786 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918cc00640 1 -- 172.21.15.71:0/782809929 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f91880cff40 con 0x7f9190112980 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.783+0000 7f918fe00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91740a2e20 0x7f91740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 -- 172.21.15.71:0/2348317027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 msgr2=0x561f7fe475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe475c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f917c009fc0 tx=0x7f917c009830 comp rx=0 tx=0).stop 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 -- 172.21.15.71:0/2348317027 shutdown_connections 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe523a0 0x561f7fe54790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe47b00 0x561f7fe51e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 --2- 172.21.15.71:0/2348317027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe475c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 -- 172.21.15.71:0/2348317027 >> 172.21.15.71:0/2348317027 conn(0x561f7fe43750 msgr2=0x561f7fe43b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 -- 172.21.15.71:0/2348317027 shutdown_connections 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f918fe00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91740a2e20 0x7f91740a52e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f9180005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.784+0000 7f9199c60740 1 -- 172.21.15.71:0/2348317027 wait complete. 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.785+0000 7f9199c60740 1 -- start start 2024-09-05T12:38:49.787 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.785+0000 7f9199c60740 1 Processor -- start 2024-09-05T12:38:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe4f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe4f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.791 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561f7fe54fd0 con 0x561f7fe471e0 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f7fd36090 con 0x561f7fe47b00 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f7fe55150 con 0x561f7fe523a0 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9199c60740 10 client.0 _ll_register_callbacks cb 0x561f7fd8af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe4f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe4f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41610/0 (socket says 172.21.15.71:41610) 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 -- 172.21.15.71:0/3260895079 learned_addr learned my addr 172.21.15.71:0/3260895079 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918f400640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe4f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f9194c00640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 -- 172.21.15.71:0/3260895079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 msgr2=0x561f7fe4fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe4fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 -- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 msgr2=0x561f7fe4f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe4f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.786+0000 7f918fe00640 1 -- 172.21.15.71:0/3260895079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7fb8c620 con 0x561f7fe471e0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9194c00640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f918fe00640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe4f3b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d44c0f804a10c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9179600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9179600640 1 -- 172.21.15.71:0/3260895079 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918004d020 con 0x561f7fe471e0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9179600640 1 -- 172.21.15.71:0/3260895079 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f9180005510 con 0x561f7fe471e0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f917c005500 con 0x561f7fe471e0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9179600640 1 -- 172.21.15.71:0/3260895079 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9180058050 con 0x561f7fe471e0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9199c60740 10 client.16626 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.787+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561f7fe60bf0 con 0x561f7fe471e0 2024-09-05T12:38:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9179600640 1 -- 172.21.15.71:0/3260895079 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f917c005500 con 0x561f7fe471e0 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9179600640 1 -- 172.21.15.71:0/3260895079 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f9180032990 con 0x561f7fe471e0 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9199c60740 10 client.16626 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9199c60740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9199c60740 20 client.16626 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9199c60740 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7fe62100 0x561f7fe644f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9199c60740 4 client.16626 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.788+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7fdba830 con 0x561f7fe62100 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.789+0000 7f918f400640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7fe62100 0x561f7fe644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.789+0000 7f918f400640 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7fe62100 0x561f7fe644f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c54b76162540bf39 server_cookie=40ef033b13bf6942 in_seq=0 out_seq=0 2024-09-05T12:38:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.789+0000 7f9179600640 10 client.16626 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9179600640 1 -- 172.21.15.71:0/3260895079 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f7fdba830 con 0x561f7fe62100 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9179600640 10 client.16626 handle_command_reply: tid=0 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 client.16626 shutdown 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 msgr2=0x561f7fe4f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe4f3b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002a410 comp rx=0 tx=0).stop 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 shutdown_connections 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7fe62100 0x561f7fe644f0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe4f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 --2- 172.21.15.71:0/3260895079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe4f3b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 >> 172.21.15.71:0/3260895079 conn(0x561f7fe43750 msgr2=0x561f7fe494b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9179600640 0 client.16626 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 shutdown_connections 2024-09-05T12:38:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.790+0000 7f9199c60740 1 -- 172.21.15.71:0/3260895079 wait complete. 2024-09-05T12:38:49.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.791+0000 7f9199c60740 20 client.16626 trim_cache size 0 max 16384 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 Processor -- start 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 -- start start 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe69e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe50b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b5480 con 0x561f7fe471e0 2024-09-05T12:38:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b7050 con 0x561f7fe47b00 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.805+0000 7f9199c60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b5be0 con 0x561f7fe523a0 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe69e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe69e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41624/0 (socket says 172.21.15.71:41624) 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 -- 172.21.15.71:0/1285694939 learned_addr learned my addr 172.21.15.71:0/1285694939 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918f400640 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 -- 172.21.15.71:0/1285694939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 msgr2=0x561f7fe50b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe50b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 -- 172.21.15.71:0/1285694939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 msgr2=0x561f7fe6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 -- 172.21.15.71:0/1285694939 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7fe44800 con 0x561f7fe471e0 2024-09-05T12:38:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.806+0000 7f918fe00640 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe69e00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9180003850 tx=0x7f9180003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e08a5b47b837b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:49.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f917be00640 1 -- 172.21.15.71:0/1285694939 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918004d070 con 0x561f7fe471e0 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f917be00640 1 -- 172.21.15.71:0/1285694939 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9180060020 con 0x561f7fe471e0 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f917be00640 1 -- 172.21.15.71:0/1285694939 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180035040 con 0x561f7fe471e0 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 -- 172.21.15.71:0/1285694939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 msgr2=0x561f7fe69e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe69e00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9180003850 tx=0x7f9180003880 comp rx=0 tx=0).stop 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 -- 172.21.15.71:0/1285694939 shutdown_connections 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe50b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 --2- 172.21.15.71:0/1285694939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe69e00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 -- 172.21.15.71:0/1285694939 >> 172.21.15.71:0/1285694939 conn(0x561f7fe43750 msgr2=0x561f7fe43b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:49.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 -- 172.21.15.71:0/1285694939 shutdown_connections 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.807+0000 7f9199c60740 1 -- 172.21.15.71:0/1285694939 wait complete. 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 -- start start 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 Processor -- start 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe55710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b6980 con 0x561f7fe471e0 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b62b0 con 0x561f7fe47b00 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91901b3480 con 0x561f7fe523a0 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.808+0000 7f9199c60740 10 client.0 _ll_register_callbacks cb 0x7f9180040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe5a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41636/0 (socket says 172.21.15.71:41636) 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 -- 172.21.15.71:0/1776335540 learned_addr learned my addr 172.21.15.71:0/1776335540 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:49.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918f400640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f9194c00640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe55710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 -- 172.21.15.71:0/1776335540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 msgr2=0x561f7fe55710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe55710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 -- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 msgr2=0x561f7fe5a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe5a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 -- 172.21.15.71:0/1776335540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7fe55f40 con 0x561f7fe471e0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f918fe00640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe5a2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9180032ed0 tx=0x7f91800055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f41962f98d268c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f9179600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.809+0000 7f9179600640 1 -- 172.21.15.71:0/1776335540 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918006a060 con 0x561f7fe471e0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9179600640 1 -- 172.21.15.71:0/1776335540 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f9180003bb0 con 0x561f7fe471e0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9180060020 con 0x561f7fe471e0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9179600640 1 -- 172.21.15.71:0/1776335540 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9180035070 con 0x561f7fe471e0 2024-09-05T12:38:49.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 10 client.16632 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561f7fecc840 con 0x561f7fe471e0 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9179600640 1 -- 172.21.15.71:0/1776335540 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9180060020 con 0x561f7fe471e0 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9179600640 1 -- 172.21.15.71:0/1776335540 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f91800048a0 con 0x561f7fe471e0 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 10 client.16632 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 20 client.16632 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7feccea0 0x561f7fecf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 4 client.16632 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.810+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f7fdb4b80 con 0x561f7feccea0 2024-09-05T12:38:49.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.811+0000 7f918f400640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7feccea0 0x561f7fecf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.811+0000 7f9179600640 10 client.16632 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:49.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.811+0000 7f918f400640 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7feccea0 0x561f7fecf290 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1156782242a3469 server_cookie=42722a86a0c46922 in_seq=0 out_seq=0 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.839+0000 7f9179600640 1 -- 172.21.15.71:0/1776335540 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f7fdb4b80 con 0x561f7feccea0 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.839+0000 7f9179600640 10 client.16632 handle_command_reply: tid=0 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.839+0000 7f9199c60740 1 client.16632 shutdown 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.839+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 msgr2=0x561f7fe5a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.839+0000 7f9199c60740 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe5a2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9180032ed0 tx=0x7f91800055c0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 shutdown_connections 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f7feccea0 0x561f7fecf290 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f7fe523a0 0x561f7fe55710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9179600640 0 client.16632 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f7fe47b00 0x561f7fe5a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 --2- 172.21.15.71:0/1776335540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f7fe471e0 0x561f7fe5a2f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 >> 172.21.15.71:0/1776335540 conn(0x561f7fe43750 msgr2=0x561f7fe48c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 shutdown_connections 2024-09-05T12:38:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 1 -- 172.21.15.71:0/1776335540 wait complete. 2024-09-05T12:38:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9199c60740 20 client.16632 trim_cache size 0 max 16384 2024-09-05T12:38:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:38:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:38:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:38:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91740a2e20 msgr2=0x7f91740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.840+0000 7f9196c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91740a2e20 0x7f91740a52e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f9180005b50 comp rx=0 tx=0).stop 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 msgr2=0x7f919019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 0x7f919019e7a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f918800eb40 tx=0x7f918800c2d0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 shutdown_connections 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f91740a2e20 0x7f91740a52e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9190112980 0x7f919019e7a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9190077040 0x7f91901a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 --2- 172.21.15.71:0/782809929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9190076720 0x7f91901a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 >> 172.21.15.71:0/782809929 conn(0x7f91900ff9e0 msgr2=0x7f9190101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 shutdown_connections 2024-09-05T12:38:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:38:49.841+0000 7f9196c00640 1 -- 172.21.15.71:0/782809929 wait complete. 2024-09-05T12:38:50.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.26 MB/sec execute 466 sec latency 12010.547 ms 2024-09-05T12:38:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:50 smithi123 ceph-mon[34321]: pgmap v346: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 920 KiB/s wr, 96 op/s 2024-09-05T12:38:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:50 smithi174 ceph-mon[34313]: pgmap v346: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 920 KiB/s wr, 96 op/s 2024-09-05T12:38:50.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:50 smithi071 ceph-mon[29845]: pgmap v346: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 920 KiB/s wr, 96 op/s 2024-09-05T12:38:51.275 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.25 MB/sec execute 467 sec latency 13010.663 ms 2024-09-05T12:38:52.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.24 MB/sec execute 468 sec latency 14010.718 ms 2024-09-05T12:38:52.560 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:52 smithi071 ceph-mon[29845]: pgmap v347: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-09-05T12:38:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:52 smithi123 ceph-mon[34321]: pgmap v347: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-09-05T12:38:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:52 smithi174 ceph-mon[34313]: pgmap v347: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 735 KiB/s rd, 1.1 MiB/s wr, 120 op/s 2024-09-05T12:38:53.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.24 MB/sec execute 469 sec latency 15011.151 ms 2024-09-05T12:38:54.224 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:38:54.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.23 MB/sec execute 470 sec latency 16011.243 ms 2024-09-05T12:38:54.609 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:38:54.641 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:54 smithi071 ceph-mon[29845]: pgmap v348: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 579 KiB/s wr, 50 op/s 2024-09-05T12:38:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:54 smithi123 ceph-mon[34321]: pgmap v348: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 579 KiB/s wr, 50 op/s 2024-09-05T12:38:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:54 smithi174 ceph-mon[34313]: pgmap v348: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 579 KiB/s wr, 50 op/s 2024-09-05T12:38:55.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.22 MB/sec execute 471 sec latency 17011.328 ms 2024-09-05T12:38:55.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 -- 172.21.15.71:0/3613483892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 msgr2=0x7f5470110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:55.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 --2- 172.21.15.71:0/3613483892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f5470110ca0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5468009ca0 tx=0x7f546802a8c0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 -- 172.21.15.71:0/3613483892 shutdown_connections 2024-09-05T12:38:55.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 --2- 172.21.15.71:0/3613483892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f5470110ca0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 --2- 172.21.15.71:0/3613483892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54701049a0 0x7f54701089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 --2- 172.21.15.71:0/3613483892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5470103ff0 0x7f54701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 -- 172.21.15.71:0/3613483892 >> 172.21.15.71:0/3613483892 conn(0x7f54700ffa90 msgr2=0x7f5470101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:55.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 -- 172.21.15.71:0/3613483892 shutdown_connections 2024-09-05T12:38:55.386 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.383+0000 7f5475000640 1 -- 172.21.15.71:0/3613483892 wait complete. 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.384+0000 7f5475000640 1 Processor -- start 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.384+0000 7f5475000640 1 -- start start 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.384+0000 7f5475000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5470103ff0 0x7f547019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.384+0000 7f5475000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54701049a0 0x7f547019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f5475000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f547019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f5475000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f54701141a0 con 0x7f5470109120 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f5475000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5470114020 con 0x7f54701049a0 2024-09-05T12:38:55.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f5475000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5470114320 con 0x7f5470103ff0 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f547019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5470103ff0 0x7f547019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f547019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48948/0 (socket says 172.21.15.71:48948) 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546e800640 1 -- 172.21.15.71:0/1532817203 learned_addr learned my addr 172.21.15.71:0/1532817203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546d400640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54701049a0 0x7f547019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546de00640 1 -- 172.21.15.71:0/1532817203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54701049a0 msgr2=0x7f547019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546de00640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54701049a0 0x7f547019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546de00640 1 -- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 msgr2=0x7f547019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546de00640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f547019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.385+0000 7f546de00640 1 -- 172.21.15.71:0/1532817203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547019f590 con 0x7f5470103ff0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.386+0000 7f546de00640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5470103ff0 0x7f547019a5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f546000e9f0 tx=0x7f546000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.386+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546000ccc0 con 0x7f5470103ff0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.386+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547019f880 con 0x7f5470103ff0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.386+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5460010910 con 0x7f5470103ff0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.386+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460018960 con 0x7f5470103ff0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.386+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54701a7160 con 0x7f5470103ff0 2024-09-05T12:38:55.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.387+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54701043d0 con 0x7f5470103ff0 2024-09-05T12:38:55.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.389+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5460010ac0 con 0x7f5470103ff0 2024-09-05T12:38:55.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.390+0000 7f5466a00640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54540a2d70 0x7f54540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:55.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.391+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7f54600cffe0 con 0x7f5470103ff0 2024-09-05T12:38:55.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.391+0000 7f546d400640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54540a2d70 0x7f54540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:55.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.391+0000 7f546d400640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54540a2d70 0x7f54540a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f547010ab00 tx=0x7f545c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:55.395 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.393+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f546009c1c0 con 0x7f5470103ff0 2024-09-05T12:38:55.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.605+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54701a7390 con 0x7f5470103ff0 2024-09-05T12:38:55.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.606+0000 7f5466a00640 1 -- 172.21.15.71:0/1532817203 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11976 (secure 0 0 0) 0x7f54600a04c0 con 0x7f5470103ff0 2024-09-05T12:38:55.608 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:38:55.608 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":49,"btime":"2024-09-05T12:38:27:359285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:38:26.374701+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:38:55.608 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 49 2024-09-05T12:38:55.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54540a2d70 msgr2=0x7f54540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:55.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54540a2d70 0x7f54540a5230 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f547010ab00 tx=0x7f545c005f90 comp rx=0 tx=0).stop 2024-09-05T12:38:55.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5470103ff0 msgr2=0x7f547019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5470103ff0 0x7f547019a5e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f546000e9f0 tx=0x7f546000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 shutdown_connections 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f54540a2d70 0x7f54540a5230 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5470109120 0x7f547019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54701049a0 0x7f547019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 --2- 172.21.15.71:0/1532817203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5470103ff0 0x7f547019a5e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.608+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 >> 172.21.15.71:0/1532817203 conn(0x7f54700ffa90 msgr2=0x7f54701005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.609+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 shutdown_connections 2024-09-05T12:38:55.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:55.609+0000 7f5475000640 1 -- 172.21.15.71:0/1532817203 wait complete. 2024-09-05T12:38:56.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.21 MB/sec execute 472 sec latency 18011.463 ms 2024-09-05T12:38:56.400 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id fed88741 --release --await' 2024-09-05T12:38:56.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id fed88741 --release --await 2024-09-05T12:38:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:56 smithi071 ceph-mon[29845]: pgmap v349: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 575 KiB/s wr, 47 op/s 2024-09-05T12:38:56.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:56 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1532817203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:38:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:56 smithi123 ceph-mon[34321]: pgmap v349: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 575 KiB/s wr, 47 op/s 2024-09-05T12:38:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:56 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1532817203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:38:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:56 smithi174 ceph-mon[34313]: pgmap v349: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 575 KiB/s wr, 47 op/s 2024-09-05T12:38:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:56 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1532817203' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:38:56.786 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:38:57.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377568 3.21 MB/sec execute 473 sec latency 19011.627 ms 2024-09-05T12:38:57.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.397+0000 7fd20a800640 1 -- 172.21.15.71:0/2598746666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041049a0 msgr2=0x7fd2041089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.397+0000 7fd20a800640 1 --2- 172.21.15.71:0/2598746666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041049a0 0x7fd2041089f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009900 tx=0x7fd1f002a410 comp rx=0 tx=0).stop 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 -- 172.21.15.71:0/2598746666 shutdown_connections 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 --2- 172.21.15.71:0/2598746666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204109120 0x7fd204110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 --2- 172.21.15.71:0/2598746666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2041049a0 0x7fd2041089f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 --2- 172.21.15.71:0/2598746666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd204103ff0 0x7fd2041043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 -- 172.21.15.71:0/2598746666 >> 172.21.15.71:0/2598746666 conn(0x7fd2040ffa90 msgr2=0x7fd204101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 -- 172.21.15.71:0/2598746666 shutdown_connections 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.398+0000 7fd20a800640 1 -- 172.21.15.71:0/2598746666 wait complete. 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 Processor -- start 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 -- start start 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 0x7fd20419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2041049a0 0x7fd20419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204109120 0x7fd20419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd204114140 con 0x7fd204103ff0 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd204113fc0 con 0x7fd204109120 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd20a800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041142c0 con 0x7fd2041049a0 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd203400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 0x7fd20419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd203400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 0x7fd20419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48970/0 (socket says 172.21.15.71:48970) 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd203400640 1 -- 172.21.15.71:0/3338348923 learned_addr learned my addr 172.21.15.71:0/3338348923 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd202a00640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2041049a0 0x7fd20419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.399+0000 7fd203e00640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204109120 0x7fd20419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd203400640 1 -- 172.21.15.71:0/3338348923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2041049a0 msgr2=0x7fd20419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd203400640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2041049a0 0x7fd20419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd203400640 1 -- 172.21.15.71:0/3338348923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204109120 msgr2=0x7fd20419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd203400640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204109120 0x7fd20419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd203400640 1 -- 172.21.15.71:0/3338348923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20419f630 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd203400640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 0x7fd20419a680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400e9f0 tx=0x7fd1f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd1fbe00640 1 -- 172.21.15.71:0/3338348923 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f400cd80 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd1fbe00640 1 -- 172.21.15.71:0/3338348923 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f4005500 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd1fbe00640 1 -- 172.21.15.71:0/3338348923 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4010690 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20419f920 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.400+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2041a7200 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.403+0000 7fd1fbe00640 1 -- 172.21.15.71:0/3338348923 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd1f40046c0 con 0x7fd204103ff0 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.403+0000 7fd20d443740 1 Processor -- start 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 -- start start 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf508e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56109bea9650 con 0x56109bf43310 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56109be50f10 con 0x56109bf4e4f0 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd20d443740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56109bea5610 con 0x56109bf43c30 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf436f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48982/0 (socket says 172.21.15.71:48982) 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd203e00640 1 -- 172.21.15.71:0/24184853 learned_addr learned my addr 172.21.15.71:0/24184853 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd203400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.404+0000 7fd202a00640 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203e00640 1 -- 172.21.15.71:0/24184853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 msgr2=0x56109bf4dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203e00640 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203e00640 1 -- 172.21.15.71:0/24184853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 msgr2=0x56109bf508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203e00640 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203e00640 1 -- 172.21.15.71:0/24184853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109bbb8580 con 0x56109bf43310 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203400640 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd1fbe00640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1e80a2cf0 0x7fd1e80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd1fbe00640 1 -- 172.21.15.71:0/3338348923 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fd1f40cfc20 con 0x7fd204103ff0 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203e00640 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf436f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009f50 tx=0x7fd1fc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae2b7baa990ec02 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd203400640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1e80a2cf0 0x7fd1e80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd1f8c00640 1 -- 172.21.15.71:0/24184853 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc004d20 con 0x56109bf43310 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd1f8c00640 1 -- 172.21.15.71:0/24184853 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc004050 con 0x56109bf43310 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.405+0000 7fd1f8c00640 1 -- 172.21.15.71:0/24184853 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc005280 con 0x56109bf43310 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 -- 172.21.15.71:0/24184853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 msgr2=0x56109bf436f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd203400640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1e80a2cf0 0x7fd1e80a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c6a0 tx=0x7fd1f40d36d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf436f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009f50 tx=0x7fd1fc0118d0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 -- 172.21.15.71:0/24184853 shutdown_connections 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 --2- 172.21.15.71:0/24184853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf436f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 -- 172.21.15.71:0/24184853 >> 172.21.15.71:0/24184853 conn(0x56109bf3f8a0 msgr2=0x56109bf3fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 -- 172.21.15.71:0/24184853 shutdown_connections 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.406+0000 7fd20d443740 1 -- 172.21.15.71:0/24184853 wait complete. 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 -- start start 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 Processor -- start 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf4b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56109bf510c0 con 0x56109bf43310 2024-09-05T12:38:57.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56109be64350 con 0x56109bf4e4f0 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56109bf51240 con 0x56109bf43c30 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.407+0000 7fd20d443740 10 client.0 _ll_register_callbacks cb 0x56109be87290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf4b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48996/0 (socket says 172.21.15.71:48996) 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 -- 172.21.15.71:0/283669434 learned_addr learned my addr 172.21.15.71:0/283669434 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd202a00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 -- 172.21.15.71:0/283669434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 msgr2=0x56109bf4b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.441 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 -- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 msgr2=0x56109bf4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf4bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 -- 172.21.15.71:0/283669434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109bc82960 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd203e00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf4b460 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009250 tx=0x7fd1fc0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e82163222f80c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.408+0000 7fd1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd1ed600640 1 -- 172.21.15.71:0/283669434 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc02b070 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd1ed600640 1 -- 172.21.15.71:0/283669434 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fd1fc025450 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1fc004050 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd1ed600640 1 -- 172.21.15.71:0/283669434 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc003e40 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 10 client.16650 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56109bf5cd50 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd1ed600640 1 -- 172.21.15.71:0/283669434 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1fc004050 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd1ed600640 1 -- 172.21.15.71:0/283669434 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fd1fc00f510 con 0x56109bf43310 2024-09-05T12:38:57.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 10 client.16650 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 10 client.16650 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 20 client.16650 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bf5e1b0 0x56109bf605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 4 client.16650 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.409+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56109beb6bc0 con 0x56109bf5e1b0 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.410+0000 7fd202a00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bf5e1b0 0x56109bf605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.410+0000 7fd202a00640 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bf5e1b0 0x56109bf605a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274389bee90339c9 server_cookie=11223c8d78e23ca1 in_seq=0 out_seq=0 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.410+0000 7fd1ed600640 10 client.16650 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.411+0000 7fd1ed600640 1 -- 172.21.15.71:0/283669434 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56109beb6bc0 con 0x56109bf5e1b0 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.411+0000 7fd1ed600640 10 client.16650 handle_command_reply: tid=0 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 client.16650 shutdown 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 msgr2=0x56109bf4b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf4b460 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009250 tx=0x7fd1fc0096c0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 shutdown_connections 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bf5e1b0 0x56109bf605a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf4e4f0 0x56109bf4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf43c30 0x56109bf4b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 --2- 172.21.15.71:0/283669434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf4b460 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd1ed600640 0 client.16650 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 >> 172.21.15.71:0/283669434 conn(0x56109bf3f8a0 msgr2=0x56109bf3eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 shutdown_connections 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 1 -- 172.21.15.71:0/283669434 wait complete. 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.412+0000 7fd20d443740 20 client.16650 trim_cache size 0 max 16384 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 Processor -- start 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 -- start start 2024-09-05T12:38:57.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf65c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf66160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b1200 con 0x56109bf43310 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b2dd0 con 0x56109bf43c30 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.430+0000 7fd20d443740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b1960 con 0x56109bf4e4f0 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf65c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf66160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf65c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49004/0 (socket says 172.21.15.71:49004) 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd203e00640 1 -- 172.21.15.71:0/636570213 learned_addr learned my addr 172.21.15.71:0/636570213 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd203e00640 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 -- 172.21.15.71:0/636570213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 msgr2=0x56109bf486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 -- 172.21.15.71:0/636570213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 msgr2=0x56109bf65c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf65c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 -- 172.21.15.71:0/636570213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56109bf508e0 con 0x56109bf43c30 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd203e00640 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.431+0000 7fd202a00640 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf66160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005940 tx=0x7fd1f0009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f329fe614753383f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd1efe00640 1 -- 172.21.15.71:0/636570213 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0035040 con 0x56109bf43c30 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd1efe00640 1 -- 172.21.15.71:0/636570213 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f0005020 con 0x56109bf43c30 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd1efe00640 1 -- 172.21.15.71:0/636570213 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0004c30 con 0x56109bf43c30 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 -- 172.21.15.71:0/636570213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 msgr2=0x56109bf66160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf66160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005940 tx=0x7fd1f0009680 comp rx=0 tx=0).stop 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 -- 172.21.15.71:0/636570213 shutdown_connections 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf66160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 --2- 172.21.15.71:0/636570213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf65c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.432+0000 7fd20d443740 1 -- 172.21.15.71:0/636570213 >> 172.21.15.71:0/636570213 conn(0x56109bf3f8a0 msgr2=0x56109bf4d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:57.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.433+0000 7fd20d443740 1 -- 172.21.15.71:0/636570213 shutdown_connections 2024-09-05T12:38:57.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.433+0000 7fd20d443740 1 -- 172.21.15.71:0/636570213 wait complete. 2024-09-05T12:38:57.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.433+0000 7fd20d443740 1 -- start start 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.433+0000 7fd20d443740 1 Processor -- start 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf52240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf52780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf52cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b2700 con 0x56109bf43310 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041b2030 con 0x56109bf43c30 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041af200 con 0x56109bf4e4f0 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd20d443740 10 client.0 _ll_register_callbacks cb 0x7fd1f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf52240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf52240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49016/0 (socket says 172.21.15.71:49016) 2024-09-05T12:38:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 -- 172.21.15.71:0/1293392266 learned_addr learned my addr 172.21.15.71:0/1293392266 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd202a00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf52780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf52cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 -- 172.21.15.71:0/1293392266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 msgr2=0x56109bf52cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf52cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 -- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 msgr2=0x56109bf52780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf52780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.434+0000 7fd203e00640 1 -- 172.21.15.71:0/1293392266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56109bfc7c40 con 0x56109bf43310 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd203e00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf52240 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc002230 tx=0x7fd1fc005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765e4b73f7dfed31 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd1ed600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd1ed600640 1 -- 172.21.15.71:0/1293392266 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc02b040 con 0x56109bf43310 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd1ed600640 1 -- 172.21.15.71:0/1293392266 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7fd1fc025450 con 0x56109bf43310 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1f0005020 con 0x56109bf43310 2024-09-05T12:38:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd1ed600640 1 -- 172.21.15.71:0/1293392266 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fd1fc042050 con 0x56109bf43310 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd20d443740 10 client.16656 fetch_fsmap learned FSMap version 49 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56109bfc8f00 con 0x56109bf43310 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.435+0000 7fd1ed600640 1 -- 172.21.15.71:0/1293392266 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1f0005020 con 0x56109bf43310 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd1ed600640 1 -- 172.21.15.71:0/1293392266 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fd1fc03b040 con 0x56109bf43310 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd20d443740 10 client.16656 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd20d443740 10 client.16656 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd20d443740 20 client.16656 populate_metadata read hostname 'smithi071' 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd20d443740 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bfc9630 0x56109bfcba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd20d443740 4 client.16656 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fed88741", "release": true, "await": true}] 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fed88741", "release": true, "await": true}) -- 0x56109bf45c10 con 0x56109bfc9630 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd202a00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bfc9630 0x56109bfcba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.436+0000 7fd202a00640 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bfc9630 0x56109bfcba50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1aabb31b93f0d9 server_cookie=9d28ffd968c8ae1f in_seq=0 out_seq=0 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.437+0000 7fd1ed600640 10 client.16656 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:57.452 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd1ed600640 1 -- 172.21.15.71:0/1293392266 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56109bf45c10 con 0x56109bfc9630 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd1ed600640 10 client.16656 handle_command_reply: tid=0 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 client.16656 shutdown 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 msgr2=0x56109bf52240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf52240 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc002230 tx=0x7fd1fc005cf0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 shutdown_connections 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56109bfc9630 0x56109bfcba50 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56109bf4e4f0 0x56109bf52cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56109bf43c30 0x56109bf52780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 --2- 172.21.15.71:0/1293392266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56109bf43310 0x56109bf52240 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 >> 172.21.15.71:0/1293392266 conn(0x56109bf3f8a0 msgr2=0x56109bf49c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd1ed600640 0 client.16656 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 shutdown_connections 2024-09-05T12:38:57.453 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 1 -- 172.21.15.71:0/1293392266 wait complete. 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.443+0000 7fd20d443740 20 client.16656 trim_cache size 0 max 16384 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 49, 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 36, 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "fed88741": { 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 36, 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:38:57.454 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 139.3, 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:57.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:57.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:57.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:38:57.456 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1e80a2cf0 msgr2=0x7fd1e80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1e80a2cf0 0x7fd1e80a51b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400c6a0 tx=0x7fd1f40d36d0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 msgr2=0x7fd20419a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 0x7fd20419a680 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd1f400e9f0 tx=0x7fd1f400edf0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 shutdown_connections 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd204109120 0x7fd20419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2041049a0 0x7fd20419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd1e80a2cf0 0x7fd1e80a51b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.457 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 --2- 172.21.15.71:0/3338348923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd204103ff0 0x7fd20419a680 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:38:57.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.444+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 >> 172.21.15.71:0/3338348923 conn(0x7fd2040ffa90 msgr2=0x7fd204100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:38:57.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.445+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 shutdown_connections 2024-09-05T12:38:57.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:38:57.445+0000 7fd20a800640 1 -- 172.21.15.71:0/3338348923 wait complete. 2024-09-05T12:38:58.276 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377666 3.20 MB/sec execute 474 sec latency 19183.080 ms 2024-09-05T12:38:58.343 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fed88741', total seconds elapsed: 23.4 2024-09-05T12:38:58.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:38:58 smithi071 ceph-mon[29845]: pgmap v350: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 835 KiB/s wr, 64 op/s 2024-09-05T12:38:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:38:58 smithi123 ceph-mon[34321]: pgmap v350: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 835 KiB/s wr, 64 op/s 2024-09-05T12:38:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:38:58 smithi174 ceph-mon[34313]: pgmap v350: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 835 KiB/s wr, 64 op/s 2024-09-05T12:38:59.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377666 3.19 MB/sec execute 475 sec latency 1755.632 ms 2024-09-05T12:39:00.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377666 3.19 MB/sec execute 476 sec latency 2755.698 ms 2024-09-05T12:39:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:00 smithi071 ceph-mon[29845]: pgmap v351: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 555 KiB/s wr, 42 op/s 2024-09-05T12:39:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:00 smithi174 ceph-mon[34313]: pgmap v351: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 555 KiB/s wr, 42 op/s 2024-09-05T12:39:00.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:00 smithi123 ceph-mon[34321]: pgmap v351: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 555 KiB/s wr, 42 op/s 2024-09-05T12:39:01.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 377666 3.18 MB/sec execute 477 sec latency 3755.753 ms 2024-09-05T12:39:02.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 378279 3.18 MB/sec execute 478 sec latency 4443.174 ms 2024-09-05T12:39:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:02 smithi174 ceph-mon[34313]: pgmap v352: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 832 KiB/s wr, 63 op/s 2024-09-05T12:39:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:02 smithi071 ceph-mon[29845]: pgmap v352: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 832 KiB/s wr, 63 op/s 2024-09-05T12:39:02.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:02 smithi123 ceph-mon[34321]: pgmap v352: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 832 KiB/s wr, 63 op/s 2024-09-05T12:39:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 -- 172.21.15.71:0/1101440245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 msgr2=0x7faa881001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 --2- 172.21.15.71:0/1101440245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 0x7faa881001d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7faa78009930 tx=0x7faa7802a4d0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 -- 172.21.15.71:0/1101440245 shutdown_connections 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 --2- 172.21.15.71:0/1101440245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 0x7faa88110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 --2- 172.21.15.71:0/1101440245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa88109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 --2- 172.21.15.71:0/1101440245 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 0x7faa881001d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 -- 172.21.15.71:0/1101440245 >> 172.21.15.71:0/1101440245 conn(0x7faa880fba30 msgr2=0x7faa880fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 -- 172.21.15.71:0/1101440245 shutdown_connections 2024-09-05T12:39:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.960+0000 7faa8d600640 1 -- 172.21.15.71:0/1101440245 wait complete. 2024-09-05T12:39:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.961+0000 7faa8d600640 1 Processor -- start 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.961+0000 7faa8d600640 1 -- start start 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa8d600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 0x7faa8819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa8d600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa8819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa8d600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 0x7faa8819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa8d600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faa88114110 con 0x7faa880ffdf0 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa8d600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa88113f90 con 0x7faa8810a2c0 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa8d600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa88114290 con 0x7faa88100710 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa8819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa8819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34694/0 (socket says 172.21.15.71:34694) 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa86000640 1 -- 172.21.15.71:0/3534217164 learned_addr learned my addr 172.21.15.71:0/3534217164 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa87400640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 0x7faa8819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa86a00640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 0x7faa8819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa86000640 1 -- 172.21.15.71:0/3534217164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 msgr2=0x7faa8819eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.962+0000 7faa86000640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 0x7faa8819eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa86000640 1 -- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 msgr2=0x7faa8819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa86000640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 0x7faa8819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa86000640 1 -- 172.21.15.71:0/3534217164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8819f5f0 con 0x7faa88100710 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa87400640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 0x7faa8819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa86000640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa8819ab40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e9f0 tx=0x7faa7400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa7f400640 1 -- 172.21.15.71:0/3534217164 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7400cd80 con 0x7faa88100710 2024-09-05T12:39:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa7f400640 1 -- 172.21.15.71:0/3534217164 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa74005500 con 0x7faa88100710 2024-09-05T12:39:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8819f8e0 con 0x7faa88100710 2024-09-05T12:39:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa7f400640 1 -- 172.21.15.71:0/3534217164 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74010690 con 0x7faa88100710 2024-09-05T12:39:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.963+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa881a7170 con 0x7faa88100710 2024-09-05T12:39:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.966+0000 7faa7f400640 1 -- 172.21.15.71:0/3534217164 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7faa740046c0 con 0x7faa88100710 2024-09-05T12:39:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 Processor -- start 2024-09-05T12:39:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 -- start start 2024-09-05T12:39:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 0x5612db1b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.970 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b7b00 0x5612db1c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5612db11d100 con 0x5612db1b7b00 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612db0c51b0 con 0x5612db1c23a0 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa905c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612db1190c0 con 0x5612db1b71e0 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa87400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 0x5612db1b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa86000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa87400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 0x5612db1b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34706/0 (socket says 172.21.15.71:34706) 2024-09-05T12:39:02.971 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa87400640 1 -- 172.21.15.71:0/3251488380 learned_addr learned my addr 172.21.15.71:0/3251488380 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.967+0000 7faa86a00640 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b7b00 0x5612db1c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa87400640 1 -- 172.21.15.71:0/3251488380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 msgr2=0x5612db1c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa87400640 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa87400640 1 -- 172.21.15.71:0/3251488380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b7b00 msgr2=0x5612db1c1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa87400640 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b7b00 0x5612db1c1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa87400640 1 -- 172.21.15.71:0/3251488380 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612dae30210 con 0x5612db1b71e0 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa7f400640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faa6c0a2e20 0x7faa6c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.972 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa86000640 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa7f400640 1 -- 172.21.15.71:0/3534217164 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7faa740cfbc0 con 0x7faa88100710 2024-09-05T12:39:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa86a00640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faa6c0a2e20 0x7faa6c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.968+0000 7faa87400640 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 0x5612db1b75c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa80011790 tx=0x7faa80011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf00e5892bba875c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.969+0000 7faa73400640 1 -- 172.21.15.71:0/3251488380 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa80003c30 con 0x5612db1b71e0 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.969+0000 7faa73400640 1 -- 172.21.15.71:0/3251488380 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa8000fe80 con 0x5612db1b71e0 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.969+0000 7faa73400640 1 -- 172.21.15.71:0/3251488380 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa800193e0 con 0x5612db1b71e0 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.969+0000 7faa86a00640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faa6c0a2e20 0x7faa6c0a52e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa78006010 tx=0x7faa78000bc0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.969+0000 7faa905c1740 1 -- 172.21.15.71:0/3251488380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 msgr2=0x5612db1b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.969+0000 7faa905c1740 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 0x5612db1b75c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7faa80011790 tx=0x7faa80011c50 comp rx=0 tx=0).stop 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.970+0000 7faa905c1740 1 -- 172.21.15.71:0/3251488380 shutdown_connections 2024-09-05T12:39:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.970+0000 7faa905c1740 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.970+0000 7faa905c1740 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b7b00 0x5612db1c1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.970+0000 7faa905c1740 1 --2- 172.21.15.71:0/3251488380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b71e0 0x5612db1b75c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.970+0000 7faa905c1740 1 -- 172.21.15.71:0/3251488380 >> 172.21.15.71:0/3251488380 conn(0x5612db1b3750 msgr2=0x5612db1b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.970+0000 7faa905c1740 1 -- 172.21.15.71:0/3251488380 shutdown_connections 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.971+0000 7faa905c1740 1 -- 172.21.15.71:0/3251488380 wait complete. 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.971+0000 7faa905c1740 1 -- start start 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.971+0000 7faa905c1740 1 Processor -- start 2024-09-05T12:39:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1bf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5612db1c4fd0 con 0x5612db1c23a0 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612db0a6090 con 0x5612db1b71e0 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612db1c5150 con 0x5612db1b7b00 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa905c1740 10 client.0 _ll_register_callbacks cb 0x5612db0faf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1bf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1bf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34716/0 (socket says 172.21.15.71:34716) 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 -- 172.21.15.71:0/1442353299 learned_addr learned my addr 172.21.15.71:0/1442353299 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa87400640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 -- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 msgr2=0x5612db1bf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1bf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 -- 172.21.15.71:0/1442353299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 msgr2=0x5612db1bfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1bfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86000640 1 -- 172.21.15.71:0/1442353299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612daefc620 con 0x5612db1b7b00 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa87400640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:39:02.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.972+0000 7faa86a00640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1bf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa86000640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1bf8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa7409b9f0 tx=0x7faa74097f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af5e86a8103fb2e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa67e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa67e00640 1 -- 172.21.15.71:0/1442353299 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7409b650 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5612db12a830 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa67e00640 1 -- 172.21.15.71:0/1442353299 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7faa740a46d0 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa67e00640 1 -- 172.21.15.71:0/1442353299 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faa740cd3b0 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa905c1740 10 client.26722 fetch_fsmap learned FSMap version 49 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5612db1d0c50 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.973+0000 7faa67e00640 1 -- 172.21.15.71:0/1442353299 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5612db12a830 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa67e00640 1 -- 172.21.15.71:0/1442353299 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7faa7409b1d0 con 0x5612db1b7b00 2024-09-05T12:39:02.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa905c1740 10 client.26722 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa905c1740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa905c1740 20 client.26722 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa905c1740 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db1d2110 0x5612db1d4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa905c1740 4 client.26722 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612db124b80 con 0x5612db1d2110 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa86a00640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db1d2110 0x5612db1d4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa86a00640 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db1d2110 0x5612db1d4500 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97441a641333339c server_cookie=164a312e689dded in_seq=0 out_seq=0 2024-09-05T12:39:02.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.974+0000 7faa67e00640 10 client.26722 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa67e00640 1 -- 172.21.15.71:0/1442353299 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5612db124b80 con 0x5612db1d2110 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa67e00640 10 client.26722 handle_command_reply: tid=0 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa905c1740 1 client.26722 shutdown 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 msgr2=0x5612db1bf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa905c1740 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1bf8d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7faa7409b9f0 tx=0x7faa74097f90 comp rx=0 tx=0).stop 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 shutdown_connections 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.975+0000 7faa905c1740 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db1d2110 0x5612db1d4500 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1bfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa67e00640 0 client.26722 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:02.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1bf8d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 1 --2- 172.21.15.71:0/1442353299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1bf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 >> 172.21.15.71:0/1442353299 conn(0x5612db1b3750 msgr2=0x5612db1b94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 shutdown_connections 2024-09-05T12:39:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 1 -- 172.21.15.71:0/1442353299 wait complete. 2024-09-05T12:39:02.986 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.976+0000 7faa905c1740 20 client.26722 trim_cache size 0 max 16384 2024-09-05T12:39:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.993+0000 7faa905c1740 1 Processor -- start 2024-09-05T12:39:02.995 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.993+0000 7faa905c1740 1 -- start start 2024-09-05T12:39:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa905c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa905c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa905c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa905c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faa881b1130 con 0x5612db1c23a0 2024-09-05T12:39:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa905c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa881b2d00 con 0x5612db1b71e0 2024-09-05T12:39:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa905c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa881b1890 con 0x5612db1b7b00 2024-09-05T12:39:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa87400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1c1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44924/0 (socket says 172.21.15.71:44924) 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 -- 172.21.15.71:0/3503597218 learned_addr learned my addr 172.21.15.71:0/3503597218 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 -- 172.21.15.71:0/3503597218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 msgr2=0x5612db1d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1d9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 -- 172.21.15.71:0/3503597218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 msgr2=0x5612db1da2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1da2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.994+0000 7faa86a00640 1 -- 172.21.15.71:0/3503597218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612db1d4870 con 0x5612db1b71e0 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.995+0000 7faa86a00640 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1c1180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faa78054fd0 tx=0x7faa7803f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c5c39114400ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.995+0000 7faa72a00640 1 -- 172.21.15.71:0/3503597218 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78035040 con 0x5612db1b71e0 2024-09-05T12:39:03.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.995+0000 7faa72a00640 1 -- 172.21.15.71:0/3503597218 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78004c60 con 0x5612db1b71e0 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.995+0000 7faa72a00640 1 -- 172.21.15.71:0/3503597218 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78032dd0 con 0x5612db1b71e0 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 -- 172.21.15.71:0/3503597218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 msgr2=0x5612db1c1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1c1180 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7faa78054fd0 tx=0x7faa7803f000 comp rx=0 tx=0).stop 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 -- 172.21.15.71:0/3503597218 shutdown_connections 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1c23a0 0x5612db1da2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 --2- 172.21.15.71:0/3503597218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1b71e0 0x5612db1c1180 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 -- 172.21.15.71:0/3503597218 >> 172.21.15.71:0/3503597218 conn(0x5612db1b3750 msgr2=0x5612db1c0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 -- 172.21.15.71:0/3503597218 shutdown_connections 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 -- 172.21.15.71:0/3503597218 wait complete. 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 -- start start 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.996+0000 7faa905c1740 1 Processor -- start 2024-09-05T12:39:03.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b71e0 0x5612db1ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faa881b2630 con 0x5612db1b71e0 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa881b1f60 con 0x5612db1c23a0 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa881af130 con 0x5612db1b7b00 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa905c1740 10 client.0 _ll_register_callbacks cb 0x7faa740d3360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b71e0 0x5612db1ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34730/0 (socket says 172.21.15.71:34730) 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa87400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44926/0 (socket says 172.21.15.71:44926) 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 -- 172.21.15.71:0/3610453027 learned_addr learned my addr 172.21.15.71:0/3610453027 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 -- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 msgr2=0x5612db1c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:03.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 -- 172.21.15.71:0/3610453027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b71e0 msgr2=0x5612db1ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b71e0 0x5612db1ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa86000640 1 -- 172.21.15.71:0/3610453027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612db1c69a0 con 0x5612db1b7b00 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.997+0000 7faa87400640 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa86000640 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1c5c30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa7409b5e0 tx=0x7faa740cff60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0a8407329c6b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa67e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa67e00640 1 -- 172.21.15.71:0/3610453027 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7409bac0 con 0x5612db1b7b00 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa78004c60 con 0x5612db1b7b00 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa67e00640 1 -- 172.21.15.71:0/3610453027 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9438+0+0 (secure 0 0 0) 0x7faa740a4bb0 con 0x5612db1b7b00 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa67e00640 1 -- 172.21.15.71:0/3610453027 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faa7409c3f0 con 0x5612db1b7b00 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa905c1740 10 client.26728 fetch_fsmap learned FSMap version 49 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.998+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5612db1c69a0 con 0x5612db1b7b00 2024-09-05T12:39:03.004 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa67e00640 1 -- 172.21.15.71:0/3610453027 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa78004c60 con 0x5612db1b7b00 2024-09-05T12:39:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa67e00640 1 -- 172.21.15.71:0/3610453027 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7faa740a1230 con 0x5612db1b7b00 2024-09-05T12:39:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa905c1740 10 client.26728 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:39:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa905c1740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:03.005 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa905c1740 20 client.26728 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa905c1740 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db23d0f0 0x5612db23f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa905c1740 4 client.26728 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:39:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612db0a17c0 con 0x5612db23d0f0 2024-09-05T12:39:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:02.999+0000 7faa86a00640 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db23d0f0 0x5612db23f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.000+0000 7faa86a00640 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db23d0f0 0x5612db23f510 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2111396bdf5d40ca server_cookie=bf8179261c6e8a42 in_seq=0 out_seq=0 2024-09-05T12:39:03.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.000+0000 7faa67e00640 10 client.26728 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.028+0000 7faa67e00640 1 -- 172.21.15.71:0/3610453027 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612db0a17c0 con 0x5612db23d0f0 2024-09-05T12:39:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.028+0000 7faa67e00640 10 client.26728 handle_command_reply: tid=0 2024-09-05T12:39:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.028+0000 7faa905c1740 1 client.26728 shutdown 2024-09-05T12:39:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.028+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 msgr2=0x5612db1c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.028+0000 7faa905c1740 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1c5c30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7faa7409b5e0 tx=0x7faa740cff60 comp rx=0 tx=0).stop 2024-09-05T12:39:03.031 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 shutdown_connections 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5612db23d0f0 0x5612db23f510 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612db1c23a0 0x5612db1c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612db1b7b00 0x5612db1c5c30 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa67e00640 0 client.26728 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 --2- 172.21.15.71:0/3610453027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612db1b71e0 0x5612db1ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 >> 172.21.15.71:0/3610453027 conn(0x5612db1b3750 msgr2=0x5612db1bfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 shutdown_connections 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 1 -- 172.21.15.71:0/3610453027 wait complete. 2024-09-05T12:39:03.032 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa905c1740 20 client.26728 trim_cache size 0 max 16384 2024-09-05T12:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:39:03.033 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faa6c0a2e20 msgr2=0x7faa6c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa8d600640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faa6c0a2e20 0x7faa6c0a52e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faa78006010 tx=0x7faa78000bc0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.029+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 msgr2=0x7faa8819ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa8819ab40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faa7400e9f0 tx=0x7faa7400edf0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 shutdown_connections 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa8810a2c0 0x7faa8819eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa88100710 0x7faa8819ab40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7faa6c0a2e20 0x7faa6c0a52e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 --2- 172.21.15.71:0/3534217164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa880ffdf0 0x7faa8819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 >> 172.21.15.71:0/3534217164 conn(0x7faa880fba30 msgr2=0x7faa88108d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:03.034 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 shutdown_connections 2024-09-05T12:39:03.035 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:03.030+0000 7faa8d600640 1 -- 172.21.15.71:0/3534217164 wait complete. 2024-09-05T12:39:03.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 378582 3.17 MB/sec execute 479 sec latency 66.502 ms 2024-09-05T12:39:04.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380030 3.19 MB/sec execute 480 sec latency 10.724 ms 2024-09-05T12:39:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:04 smithi174 ceph-mon[34313]: pgmap v353: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 595 KiB/s wr, 39 op/s 2024-09-05T12:39:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:04 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:39:04.881 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:04 smithi071 ceph-mon[29845]: pgmap v353: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 595 KiB/s wr, 39 op/s 2024-09-05T12:39:04.882 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:04 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:39:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:04 smithi123 ceph-mon[34321]: pgmap v353: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 595 KiB/s wr, 39 op/s 2024-09-05T12:39:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:04 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:39:05.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 380742 3.19 MB/sec execute 481 sec latency 323.395 ms 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: pgmap v354: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 536 KiB/s wr, 37 op/s 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:39:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:05 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:05.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: pgmap v354: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 536 KiB/s wr, 37 op/s 2024-09-05T12:39:05.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:39:05.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:39:05.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:05 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: pgmap v354: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 536 KiB/s wr, 37 op/s 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:39:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:39:06.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:05 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:39:06.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 381255 3.19 MB/sec execute 482 sec latency 568.753 ms 2024-09-05T12:39:07.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 382729 3.20 MB/sec execute 483 sec latency 680.646 ms 2024-09-05T12:39:08.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:07 smithi071 ceph-mon[29845]: pgmap v355: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-05T12:39:08.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:07 smithi123 ceph-mon[34321]: pgmap v355: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-05T12:39:08.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:07 smithi174 ceph-mon[34313]: pgmap v355: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-09-05T12:39:08.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 383771 3.20 MB/sec execute 484 sec latency 31.210 ms 2024-09-05T12:39:09.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:08 smithi071 ceph-mon[29845]: osdmap e107: 12 total, 12 up, 12 in 2024-09-05T12:39:09.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:08 smithi123 ceph-mon[34321]: osdmap e107: 12 total, 12 up, 12 in 2024-09-05T12:39:09.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:08 smithi174 ceph-mon[34313]: osdmap e107: 12 total, 12 up, 12 in 2024-09-05T12:39:09.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 384796 3.21 MB/sec execute 485 sec latency 311.221 ms 2024-09-05T12:39:10.043 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:09 smithi071 ceph-mon[29845]: pgmap v357: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-09-05T12:39:10.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:09 smithi123 ceph-mon[34321]: pgmap v357: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-09-05T12:39:10.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:09 smithi174 ceph-mon[34313]: pgmap v357: 129 pgs: 129 active+clean; 126 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-09-05T12:39:10.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 384796 3.20 MB/sec execute 486 sec latency 1311.265 ms 2024-09-05T12:39:11.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 385570 3.20 MB/sec execute 487 sec latency 1829.784 ms 2024-09-05T12:39:12.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:11 smithi071 ceph-mon[29845]: pgmap v358: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:12.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:11 smithi123 ceph-mon[34321]: pgmap v358: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:12.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:11 smithi174 ceph-mon[34313]: pgmap v358: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:12.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 387069 3.22 MB/sec execute 488 sec latency 14.622 ms 2024-09-05T12:39:13.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 388316 3.22 MB/sec execute 489 sec latency 21.168 ms 2024-09-05T12:39:14.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:13 smithi071 ceph-mon[29845]: pgmap v359: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:14.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:13 smithi123 ceph-mon[34321]: pgmap v359: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:14.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:13 smithi174 ceph-mon[34313]: pgmap v359: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:14.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 388368 3.21 MB/sec execute 490 sec latency 959.630 ms 2024-09-05T12:39:15.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 388368 3.21 MB/sec execute 491 sec latency 1959.695 ms 2024-09-05T12:39:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 -- 172.21.15.71:0/1775518711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c109080 msgr2=0x7fca1c110bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 --2- 172.21.15.71:0/1775518711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c109080 0x7fca1c110bd0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fca14009ca0 tx=0x7fca1402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 -- 172.21.15.71:0/1775518711 shutdown_connections 2024-09-05T12:39:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 --2- 172.21.15.71:0/1775518711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c109080 0x7fca1c110bd0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 --2- 172.21.15.71:0/1775518711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c1048e0 0x7fca1c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 --2- 172.21.15.71:0/1775518711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c103f30 0x7fca1c104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 -- 172.21.15.71:0/1775518711 >> 172.21.15.71:0/1775518711 conn(0x7fca1c0ff980 msgr2=0x7fca1c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 -- 172.21.15.71:0/1775518711 shutdown_connections 2024-09-05T12:39:16.157 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.154+0000 7fca22800640 1 -- 172.21.15.71:0/1775518711 wait complete. 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.155+0000 7fca22800640 1 Processor -- start 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.155+0000 7fca22800640 1 -- start start 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.155+0000 7fca22800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 0x7fca1c19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca22800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c109080 0x7fca1c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca22800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c19edf0 0x7fca1c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca22800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1140c0 con 0x7fca1c103f30 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca22800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c113f40 con 0x7fca1c109080 2024-09-05T12:39:16.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca22800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c114240 con 0x7fca1c19edf0 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 0x7fca1c19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 0x7fca1c19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38070/0 (socket says 172.21.15.71:38070) 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 -- 172.21.15.71:0/1907002400 learned_addr learned my addr 172.21.15.71:0/1907002400 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca12a00640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c109080 0x7fca1c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1be00640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c19edf0 0x7fca1c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 -- 172.21.15.71:0/1907002400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c19edf0 msgr2=0x7fca1c19f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c19edf0 0x7fca1c19f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 -- 172.21.15.71:0/1907002400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c109080 msgr2=0x7fca1c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c109080 0x7fca1c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.156+0000 7fca1b400640 1 -- 172.21.15.71:0/1907002400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1c19f8e0 con 0x7fca1c103f30 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.157+0000 7fca1b400640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 0x7fca1c19a530 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e9f0 tx=0x7fca0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:16.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.157+0000 7fca18c00640 1 -- 172.21.15.71:0/1907002400 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c00cd80 con 0x7fca1c103f30 2024-09-05T12:39:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.157+0000 7fca18c00640 1 -- 172.21.15.71:0/1907002400 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca0c005500 con 0x7fca1c103f30 2024-09-05T12:39:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.157+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca1c19fbd0 con 0x7fca1c103f30 2024-09-05T12:39:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.157+0000 7fca18c00640 1 -- 172.21.15.71:0/1907002400 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c010690 con 0x7fca1c103f30 2024-09-05T12:39:16.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.157+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca1c1a7370 con 0x7fca1c103f30 2024-09-05T12:39:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.161+0000 7fca255ee740 1 Processor -- start 2024-09-05T12:39:16.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.161+0000 7fca255ee740 1 -- start start 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca18c00640 1 -- 172.21.15.71:0/1907002400 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fca0c0046c0 con 0x7fca1c103f30 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca255ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x5560358385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca255ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035842e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca255ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035845790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca255ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55603579e100 con 0x556035838b00 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca255ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560357461b0 con 0x5560358433a0 2024-09-05T12:39:16.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca255ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55603579a0c0 con 0x5560358381e0 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca1b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035842e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035845790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca1be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x5560358385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca12a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035845790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49646/0 (socket says 172.21.15.71:49646) 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.162+0000 7fca12a00640 1 -- 172.21.15.71:0/366012197 learned_addr learned my addr 172.21.15.71:0/366012197 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca1be00640 1 -- 172.21.15.71:0/366012197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 msgr2=0x556035845790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca1be00640 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035845790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca1be00640 1 -- 172.21.15.71:0/366012197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 msgr2=0x556035842e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca1be00640 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035842e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca1be00640 1 -- 172.21.15.71:0/366012197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560354b1210 con 0x5560358381e0 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca12a00640 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035845790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca1be00640 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x5560358385c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca14006010 tx=0x7fca14005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93f888c373d5f656 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca10c00640 1 -- 172.21.15.71:0/366012197 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14031db0 con 0x5560358381e0 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca10c00640 1 -- 172.21.15.71:0/366012197 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14036040 con 0x5560358381e0 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca18c00640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fca000a2da0 0x7fca000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.163+0000 7fca10c00640 1 -- 172.21.15.71:0/366012197 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca140042a0 con 0x5560358381e0 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca18c00640 1 -- 172.21.15.71:0/1907002400 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fca0c0cfbc0 con 0x7fca1c103f30 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca1b400640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fca000a2da0 0x7fca000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 -- 172.21.15.71:0/366012197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 msgr2=0x5560358385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x5560358385c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca14006010 tx=0x7fca14005f10 comp rx=0 tx=0).stop 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 -- 172.21.15.71:0/366012197 shutdown_connections 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035845790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035842e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 --2- 172.21.15.71:0/366012197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x5560358385c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 -- 172.21.15.71:0/366012197 >> 172.21.15.71:0/366012197 conn(0x556035834750 msgr2=0x556035834b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 -- 172.21.15.71:0/366012197 shutdown_connections 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca255ee740 1 -- 172.21.15.71:0/366012197 wait complete. 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.164+0000 7fca1b400640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fca000a2da0 0x7fca000a5260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0cd030 tx=0x7fca0c0d3710 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 -- start start 2024-09-05T12:39:16.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 Processor -- start 2024-09-05T12:39:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x556035840340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035840dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556035845df0 con 0x556035838b00 2024-09-05T12:39:16.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556035727090 con 0x5560358433a0 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556035845f70 con 0x5560358381e0 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.165+0000 7fca255ee740 10 client.0 _ll_register_callbacks cb 0x55603577bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38090/0 (socket says 172.21.15.71:38090) 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 -- 172.21.15.71:0/1663614810 learned_addr learned my addr 172.21.15.71:0/1663614810 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca1be00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x556035840340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca1be00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035840dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 -- 172.21.15.71:0/1663614810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 msgr2=0x556035840340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x556035840340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 -- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 msgr2=0x556035840dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035840dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 -- 172.21.15.71:0/1663614810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55603557d620 con 0x556035838b00 2024-09-05T12:39:16.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca1be00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035840dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca12a00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fca040126e0 tx=0x7fca04012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414a0323bf02a4a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.166+0000 7fca09600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca09600640 1 -- 172.21.15.71:0/1663614810 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca040121d0 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca14036040 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca09600640 1 -- 172.21.15.71:0/1663614810 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fca04004030 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca09600640 1 -- 172.21.15.71:0/1663614810 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fca04025050 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca255ee740 10 client.16686 fetch_fsmap learned FSMap version 49 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556035851a70 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.167+0000 7fca09600640 1 -- 172.21.15.71:0/1663614810 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14036040 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca09600640 1 -- 172.21.15.71:0/1663614810 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fca04003820 con 0x556035838b00 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca255ee740 10 client.16686 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca255ee740 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca255ee740 20 client.16686 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:16.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca255ee740 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556035852f30 0x556035855320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca255ee740 4 client.16686 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560357ab830 con 0x556035852f30 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.168+0000 7fca1be00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556035852f30 0x556035855320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.169+0000 7fca1be00640 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556035852f30 0x556035855320 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceaeea66c085999f server_cookie=f03dab9600a523ec in_seq=0 out_seq=0 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.169+0000 7fca09600640 10 client.16686 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.169+0000 7fca09600640 1 -- 172.21.15.71:0/1663614810 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560357ab830 con 0x556035852f30 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.169+0000 7fca09600640 10 client.16686 handle_command_reply: tid=0 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.169+0000 7fca255ee740 1 client.16686 shutdown 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 msgr2=0x556035840880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840880 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fca040126e0 tx=0x7fca04012ba0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 shutdown_connections 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556035852f30 0x556035855320 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035840dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840880 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca09600640 0 client.16686 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 --2- 172.21.15.71:0/1663614810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x556035840340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 >> 172.21.15.71:0/1663614810 conn(0x556035834750 msgr2=0x55603583a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 shutdown_connections 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 1 -- 172.21.15.71:0/1663614810 wait complete. 2024-09-05T12:39:16.178 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.170+0000 7fca255ee740 20 client.16686 trim_cache size 0 max 16384 2024-09-05T12:39:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 Processor -- start 2024-09-05T12:39:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 -- start start 2024-09-05T12:39:16.187 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 0x55603582f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358433a0 0x55603585abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b0fe0 con 0x556035838b00 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b2bb0 con 0x5560358381e0 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca255ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b1740 con 0x5560358433a0 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 0x55603582f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca1be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 0x55603582f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49664/0 (socket says 172.21.15.71:49664) 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca1be00640 1 -- 172.21.15.71:0/2543477814 learned_addr learned my addr 172.21.15.71:0/2543477814 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.185+0000 7fca12a00640 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358433a0 0x55603585abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 -- 172.21.15.71:0/2543477814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358433a0 msgr2=0x55603585abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358433a0 0x55603585abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.188 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 -- 172.21.15.71:0/2543477814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 msgr2=0x556035840bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 -- 172.21.15.71:0/2543477814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55603583f2d0 con 0x5560358381e0 2024-09-05T12:39:16.189 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca12a00640 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:16.192 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca1be00640 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 0x55603582f6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca14009ca0 tx=0x7fca14005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b0b76042eb5ceca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca0be00640 1 -- 172.21.15.71:0/2543477814 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca14031cb0 con 0x5560358381e0 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca0be00640 1 -- 172.21.15.71:0/2543477814 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14036070 con 0x5560358381e0 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.186+0000 7fca0be00640 1 -- 172.21.15.71:0/2543477814 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca140046f0 con 0x5560358381e0 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 -- 172.21.15.71:0/2543477814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 msgr2=0x55603582f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 0x55603582f6a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca14009ca0 tx=0x7fca14005f10 comp rx=0 tx=0).stop 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 -- 172.21.15.71:0/2543477814 shutdown_connections 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358433a0 0x55603585abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035840bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 --2- 172.21.15.71:0/2543477814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358381e0 0x55603582f6a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 -- 172.21.15.71:0/2543477814 >> 172.21.15.71:0/2543477814 conn(0x556035834750 msgr2=0x55603583a460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 -- 172.21.15.71:0/2543477814 shutdown_connections 2024-09-05T12:39:16.193 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 -- 172.21.15.71:0/2543477814 wait complete. 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.187+0000 7fca255ee740 1 -- start start 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 Processor -- start 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x55603584b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035846b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035847080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b24e0 con 0x556035838b00 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1b1e10 con 0x5560358433a0 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca1c1aefe0 con 0x5560358381e0 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca255ee740 10 client.0 _ll_register_callbacks cb 0x7fca04009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035846b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.194 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca1be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x55603584b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca12a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035846b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38114/0 (socket says 172.21.15.71:38114) 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca12a00640 1 -- 172.21.15.71:0/4029909471 learned_addr learned my addr 172.21.15.71:0/4029909471 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.188+0000 7fca1be00640 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035847080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca12a00640 1 -- 172.21.15.71:0/4029909471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 msgr2=0x55603584b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca12a00640 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x55603584b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca12a00640 1 -- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 msgr2=0x556035847080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca12a00640 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035847080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca12a00640 1 -- 172.21.15.71:0/4029909471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556035838930 con 0x556035838b00 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca12a00640 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035846b40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x556035841980 tx=0x7fca0400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64844f0042ad8559 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca09600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:16.195 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca09600640 1 -- 172.21.15.71:0/4029909471 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca04022040 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca09600640 1 -- 172.21.15.71:0/4029909471 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fca04003800 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca14036070 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca09600640 1 -- 172.21.15.71:0/4029909471 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fca04037050 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca255ee740 10 client.16692 fetch_fsmap learned FSMap version 49 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.189+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5560358bd9f0 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca09600640 1 -- 172.21.15.71:0/4029909471 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca14036070 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca09600640 1 -- 172.21.15.71:0/4029909471 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7fca040054a0 con 0x556035838b00 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca255ee740 10 client.16692 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:39:16.196 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca255ee740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca255ee740 20 client.16692 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca255ee740 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5560358be1c0 0x5560358c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca255ee740 4 client.16692 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55603582fd80 con 0x5560358be1c0 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.190+0000 7fca1be00640 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5560358be1c0 0x5560358c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.191+0000 7fca1be00640 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5560358be1c0 0x5560358c05e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22db658bd86263e0 server_cookie=c5426e14838baa41 in_seq=0 out_seq=0 2024-09-05T12:39:16.197 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.191+0000 7fca09600640 10 client.16692 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca09600640 1 -- 172.21.15.71:0/4029909471 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55603582fd80 con 0x5560358be1c0 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca09600640 10 client.16692 handle_command_reply: tid=0 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca255ee740 1 client.16692 shutdown 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 msgr2=0x556035846b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca255ee740 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035846b40 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x556035841980 tx=0x7fca0400f820 comp rx=0 tx=0).stop 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 shutdown_connections 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.231+0000 7fca255ee740 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5560358be1c0 0x5560358c05e0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560358433a0 0x556035847080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556035838b00 0x556035846b40 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 1 --2- 172.21.15.71:0/4029909471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560358381e0 0x55603584b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca09600640 0 client.16692 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 >> 172.21.15.71:0/4029909471 conn(0x556035834750 msgr2=0x556035833010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 shutdown_connections 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 1 -- 172.21.15.71:0/4029909471 wait complete. 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.232+0000 7fca255ee740 20 client.16692 trim_cache size 0 max 16384 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:39:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:39:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fca000a2da0 msgr2=0x7fca000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fca000a2da0 0x7fca000a5260 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0cd030 tx=0x7fca0c0d3710 comp rx=0 tx=0).stop 2024-09-05T12:39:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 msgr2=0x7fca1c19a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 0x7fca1c19a530 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fca0c00e9f0 tx=0x7fca0c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 shutdown_connections 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca1c19edf0 0x7fca1c19f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca1c109080 0x7fca1c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fca000a2da0 0x7fca000a5260 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 --2- 172.21.15.71:0/1907002400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca1c103f30 0x7fca1c19a530 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 >> 172.21.15.71:0/1907002400 conn(0x7fca1c0ff980 msgr2=0x7fca1c100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 shutdown_connections 2024-09-05T12:39:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:16.233+0000 7fca22800640 1 -- 172.21.15.71:0/1907002400 wait complete. 2024-09-05T12:39:16.277 INFO:tasks.workunit.client.0.smithi071.stdout: 1 388487 3.20 MB/sec execute 492 sec latency 2910.640 ms 2024-09-05T12:39:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:16 smithi174 ceph-mon[34313]: pgmap v360: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:16 smithi174 ceph-mon[34313]: osdmap e108: 12 total, 12 up, 12 in 2024-09-05T12:39:16.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:16 smithi071 ceph-mon[29845]: pgmap v360: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:16.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:16 smithi071 ceph-mon[29845]: osdmap e108: 12 total, 12 up, 12 in 2024-09-05T12:39:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:16 smithi123 ceph-mon[34321]: pgmap v360: 129 pgs: 129 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-05T12:39:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:16 smithi123 ceph-mon[34321]: osdmap e108: 12 total, 12 up, 12 in 2024-09-05T12:39:17.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 390434 3.22 MB/sec execute 493 sec latency 22.106 ms 2024-09-05T12:39:18.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 391472 3.22 MB/sec execute 494 sec latency 20.755 ms 2024-09-05T12:39:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:18 smithi174 ceph-mon[34313]: pgmap v362: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-05T12:39:18.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:18 smithi071 ceph-mon[29845]: pgmap v362: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-05T12:39:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:18 smithi123 ceph-mon[34321]: pgmap v362: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-05T12:39:19.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 392360 3.22 MB/sec execute 495 sec latency 588.998 ms 2024-09-05T12:39:20.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 393555 3.23 MB/sec execute 496 sec latency 366.575 ms 2024-09-05T12:39:20.479 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:20 smithi071 ceph-mon[29845]: pgmap v363: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-09-05T12:39:20.481 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:20 smithi123 ceph-mon[34321]: pgmap v363: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-09-05T12:39:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:20 smithi174 ceph-mon[34313]: pgmap v363: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-09-05T12:39:21.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 394632 3.24 MB/sec execute 497 sec latency 17.754 ms 2024-09-05T12:39:22.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 396071 3.24 MB/sec execute 498 sec latency 126.232 ms 2024-09-05T12:39:22.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:22 smithi071 ceph-mon[29845]: pgmap v364: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:22 smithi174 ceph-mon[34313]: pgmap v364: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:22 smithi123 ceph-mon[34321]: pgmap v364: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:23.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 397115 3.25 MB/sec execute 499 sec latency 288.328 ms 2024-09-05T12:39:24.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 398222 3.26 MB/sec execute 500 sec latency 13.111 ms 2024-09-05T12:39:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:24 smithi174 ceph-mon[34313]: pgmap v365: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:24 smithi071 ceph-mon[29845]: pgmap v365: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:24 smithi123 ceph-mon[34321]: pgmap v365: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:25.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 399001 3.26 MB/sec execute 501 sec latency 518.843 ms 2024-09-05T12:39:26.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 399001 3.25 MB/sec execute 502 sec latency 1518.982 ms 2024-09-05T12:39:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:26 smithi174 ceph-mon[34313]: pgmap v366: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:26.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:26 smithi071 ceph-mon[29845]: pgmap v366: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:26 smithi123 ceph-mon[34321]: pgmap v366: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.0 MiB/s wr, 211 op/s 2024-09-05T12:39:27.278 INFO:tasks.workunit.client.0.smithi071.stdout: 1 400519 3.26 MB/sec execute 503 sec latency 1627.737 ms 2024-09-05T12:39:28.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 401771 3.27 MB/sec execute 504 sec latency 19.708 ms 2024-09-05T12:39:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:28 smithi174 ceph-mon[34313]: pgmap v367: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 297 op/s 2024-09-05T12:39:28.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:28 smithi071 ceph-mon[29845]: pgmap v367: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 297 op/s 2024-09-05T12:39:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:28 smithi123 ceph-mon[34321]: pgmap v367: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 297 op/s 2024-09-05T12:39:29.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 402551 3.27 MB/sec execute 505 sec latency 346.857 ms 2024-09-05T12:39:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 -- 172.21.15.71:0/2177090883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 msgr2=0x7f7180104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 --2- 172.21.15.71:0/2177090883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f7180104330 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7170009ca0 tx=0x7f717002a8c0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 -- 172.21.15.71:0/2177090883 shutdown_connections 2024-09-05T12:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 --2- 172.21.15.71:0/2177090883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 0x7f7180110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 --2- 172.21.15.71:0/2177090883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7180104900 0x7f7180108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 --2- 172.21.15.71:0/2177090883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f7180104330 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 -- 172.21.15.71:0/2177090883 >> 172.21.15.71:0/2177090883 conn(0x7f71800ff9e0 msgr2=0x7f7180101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 -- 172.21.15.71:0/2177090883 shutdown_connections 2024-09-05T12:39:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.348+0000 7f7185000640 1 -- 172.21.15.71:0/2177090883 wait complete. 2024-09-05T12:39:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.349+0000 7f7185000640 1 Processor -- start 2024-09-05T12:39:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.349+0000 7f7185000640 1 -- start start 2024-09-05T12:39:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.349+0000 7f7185000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f718019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.349+0000 7f7185000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 0x7f718019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f7185000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f718019ee80 0x7f718019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f7185000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7180114110 con 0x7f7180103f50 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f7185000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7180113f90 con 0x7f71801090a0 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f7185000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7180114290 con 0x7f718019ee80 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f718019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f718019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51310/0 (socket says 172.21.15.71:51310) 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 -- 172.21.15.71:0/1258154287 learned_addr learned my addr 172.21.15.71:0/1258154287 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717d400640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 0x7f718019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717e800640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f718019ee80 0x7f718019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 -- 172.21.15.71:0/1258154287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f718019ee80 msgr2=0x7f718019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f718019ee80 0x7f718019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 -- 172.21.15.71:0/1258154287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 msgr2=0x7f718019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 0x7f718019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717de00640 1 -- 172.21.15.71:0/1258154287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718019f970 con 0x7f7180103f50 2024-09-05T12:39:29.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.350+0000 7f717d400640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 0x7f718019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.351+0000 7f717de00640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f718019a5c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7170005e30 tx=0x7f7170003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.351+0000 7f7176a00640 1 -- 172.21.15.71:0/1258154287 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170031650 con 0x7f7180103f50 2024-09-05T12:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.351+0000 7f7176a00640 1 -- 172.21.15.71:0/1258154287 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71700317f0 con 0x7f7180103f50 2024-09-05T12:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.351+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718019fc00 con 0x7f7180103f50 2024-09-05T12:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.351+0000 7f7176a00640 1 -- 172.21.15.71:0/1258154287 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717002fa30 con 0x7f7180103f50 2024-09-05T12:39:29.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.351+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71801a7420 con 0x7f7180103f50 2024-09-05T12:39:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7176a00640 1 -- 172.21.15.71:0/1258154287 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f717002fc30 con 0x7f7180103f50 2024-09-05T12:39:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 Processor -- start 2024-09-05T12:39:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 -- start start 2024-09-05T12:39:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b37805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b378d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5614b36e6100 con 0x5614b3780b00 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5614b368e1b0 con 0x5614b37801e0 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.354+0000 7f7187fff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5614b36e20c0 con 0x5614b378b3a0 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b37805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51326/0 (socket says 172.21.15.71:51326) 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717de00640 1 -- 172.21.15.71:0/3883264135 learned_addr learned my addr 172.21.15.71:0/3883264135 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717d400640 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b378d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 -- 172.21.15.71:0/3883264135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 msgr2=0x5614b378d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b378d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 -- 172.21.15.71:0/3883264135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 msgr2=0x5614b378ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 -- 172.21.15.71:0/3883264135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b33f9210 con 0x5614b37801e0 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717de00640 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f7176a00640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71640a2da0 0x7f71640a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.355+0000 7f717e800640 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b37805c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7178009b80 tx=0x7f7178011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f14fb93d8ce2860 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f7176a00640 1 -- 172.21.15.71:0/1258154287 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f71700f0a00 con 0x7f7180103f50 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f716b400640 1 -- 172.21.15.71:0/3883264135 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178004050 con 0x5614b37801e0 2024-09-05T12:39:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f716b400640 1 -- 172.21.15.71:0/3883264135 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7178005500 con 0x5614b37801e0 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f716b400640 1 -- 172.21.15.71:0/3883264135 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178004cd0 con 0x5614b37801e0 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f717de00640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71640a2da0 0x7f71640a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f7187fff740 1 -- 172.21.15.71:0/3883264135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 msgr2=0x5614b37805c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f717de00640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71640a2da0 0x7f71640a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f71700b90a0 tx=0x7f71700f6510 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.356+0000 7f7187fff740 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b37805c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7178009b80 tx=0x7f7178011810 comp rx=0 tx=0).stop 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 -- 172.21.15.71:0/3883264135 shutdown_connections 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b378d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 --2- 172.21.15.71:0/3883264135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b37805c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 -- 172.21.15.71:0/3883264135 >> 172.21.15.71:0/3883264135 conn(0x5614b377c750 msgr2=0x5614b377cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 -- 172.21.15.71:0/3883264135 shutdown_connections 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 -- 172.21.15.71:0/3883264135 wait complete. 2024-09-05T12:39:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 -- start start 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.357+0000 7f7187fff740 1 Processor -- start 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3788390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b3780b00 0x5614b37888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b378b3a0 0x5614b3788e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5614b378dfd0 con 0x5614b37801e0 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5614b366f090 con 0x5614b378b3a0 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5614b378e150 con 0x5614b3780b00 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f7187fff740 10 client.0 _ll_register_callbacks cb 0x5614b36c3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3788390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3788390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51338/0 (socket says 172.21.15.71:51338) 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717e800640 1 -- 172.21.15.71:0/1764009973 learned_addr learned my addr 172.21.15.71:0/1764009973 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717d400640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b3780b00 0x5614b37888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717e800640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b378b3a0 0x5614b3788e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717d400640 1 -- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b378b3a0 msgr2=0x5614b3788e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717d400640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b378b3a0 0x5614b3788e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717d400640 1 -- 172.21.15.71:0/1764009973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 msgr2=0x5614b3788390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717d400640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3788390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717d400640 1 -- 172.21.15.71:0/1764009973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b34c5620 con 0x5614b3780b00 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.358+0000 7f717e800640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3788390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f717d400640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b3780b00 0x5614b37888d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f718006b450 tx=0x7f716c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81355330a2fe2d99 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f715fe00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f715fe00640 1 -- 172.21.15.71:0/1764009973 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c012110 con 0x5614b3780b00 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7178005500 con 0x5614b3780b00 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f715fe00640 1 -- 172.21.15.71:0/1764009973 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f716c004050 con 0x5614b3780b00 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f715fe00640 1 -- 172.21.15.71:0/1764009973 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f716c025050 con 0x5614b3780b00 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f7187fff740 10 client.26758 fetch_fsmap learned FSMap version 49 2024-09-05T12:39:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5614b3799ca0 con 0x5614b3780b00 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.359+0000 7f715fe00640 1 -- 172.21.15.71:0/1764009973 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7178005500 con 0x5614b3780b00 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f715fe00640 1 -- 172.21.15.71:0/1764009973 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f716c010430 con 0x5614b3780b00 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f7187fff740 10 client.26758 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f7187fff740 10 client.26758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f7187fff740 20 client.26758 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f7187fff740 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b379b160 0x5614b379d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f7187fff740 4 client.26758 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614b36f3830 con 0x5614b379b160 2024-09-05T12:39:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f717e800640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b379b160 0x5614b379d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f717e800640 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b379b160 0x5614b379d550 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2de933141a8cb22c server_cookie=abcd6614449e7b32 in_seq=0 out_seq=0 2024-09-05T12:39:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.360+0000 7f715fe00640 10 client.26758 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.361+0000 7f715fe00640 1 -- 172.21.15.71:0/1764009973 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614b36f3830 con 0x5614b379b160 2024-09-05T12:39:29.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.361+0000 7f715fe00640 10 client.26758 handle_command_reply: tid=0 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.361+0000 7f7187fff740 1 client.26758 shutdown 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b3780b00 msgr2=0x5614b37888d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b3780b00 0x5614b37888d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f718006b450 tx=0x7f716c00f790 comp rx=0 tx=0).stop 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 shutdown_connections 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b379b160 0x5614b379d550 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b378b3a0 0x5614b3788e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b3780b00 0x5614b37888d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 --2- 172.21.15.71:0/1764009973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3788390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f715fe00640 0 client.26758 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 >> 172.21.15.71:0/1764009973 conn(0x5614b377c750 msgr2=0x5614b37824b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 shutdown_connections 2024-09-05T12:39:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 1 -- 172.21.15.71:0/1764009973 wait complete. 2024-09-05T12:39:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.362+0000 7f7187fff740 20 client.26758 trim_cache size 0 max 16384 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 Processor -- start 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 -- start start 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3782de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b3780b00 0x5614b378a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b37a2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b1110 con 0x5614b37801e0 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b2ce0 con 0x5614b3780b00 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f7187fff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b1870 con 0x5614b378b3a0 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f717e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3782de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f717e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3782de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51350/0 (socket says 172.21.15.71:51350) 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f717e800640 1 -- 172.21.15.71:0/3585947725 learned_addr learned my addr 172.21.15.71:0/3585947725 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:29.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.377+0000 7f717d400640 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b3780b00 0x5614b378a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717e800640 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b37a2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717d400640 1 -- 172.21.15.71:0/3585947725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 msgr2=0x5614b37a2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717d400640 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b37a2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717d400640 1 -- 172.21.15.71:0/3585947725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 msgr2=0x5614b3782de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717d400640 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3782de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717d400640 1 -- 172.21.15.71:0/3585947725 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614b37868c0 con 0x5614b3780b00 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717e800640 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3782de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f717d400640 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b3780b00 0x5614b378a1f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f716c0107d0 tx=0x7f716c012320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46bfaa4487b1540 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f716aa00640 1 -- 172.21.15.71:0/3585947725 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c02e070 con 0x5614b3780b00 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f716aa00640 1 -- 172.21.15.71:0/3585947725 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f716c005590 con 0x5614b3780b00 2024-09-05T12:39:29.380 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.378+0000 7f716aa00640 1 -- 172.21.15.71:0/3585947725 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c0035f0 con 0x5614b3780b00 2024-09-05T12:39:29.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 -- 172.21.15.71:0/3585947725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b3780b00 msgr2=0x5614b378a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b3780b00 0x5614b378a1f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f716c0107d0 tx=0x7f716c012320 comp rx=0 tx=0).stop 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 -- 172.21.15.71:0/3585947725 shutdown_connections 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b37a2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b3780b00 0x5614b378a1f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 --2- 172.21.15.71:0/3585947725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b37801e0 0x5614b3782de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 -- 172.21.15.71:0/3585947725 >> 172.21.15.71:0/3585947725 conn(0x5614b377c750 msgr2=0x5614b3789ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 -- 172.21.15.71:0/3585947725 shutdown_connections 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 -- 172.21.15.71:0/3585947725 wait complete. 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 -- start start 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.379+0000 7f7187fff740 1 Processor -- start 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b378f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b3804e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b2610 con 0x5614b3780b00 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f71801b1f40 con 0x5614b37801e0 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71801af110 con 0x5614b378b3a0 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f7187fff740 10 client.0 _ll_register_callbacks cb 0x7f716c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b378f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b378f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36568/0 (socket says 172.21.15.71:36568) 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:51356/0 (socket says 172.21.15.71:51356) 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 -- 172.21.15.71:0/2315693543 learned_addr learned my addr 172.21.15.71:0/2315693543 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b3804e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 -- 172.21.15.71:0/2315693543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 msgr2=0x5614b3804e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b3804e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 -- 172.21.15.71:0/2315693543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 msgr2=0x5614b378f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.380+0000 7f717e800640 1 -- 172.21.15.71:0/2315693543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614b3805370 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f717e800640 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b378f3d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7178014040 tx=0x7f7178011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90464bc9ad55524 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f715fe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f715fe00640 1 -- 172.21.15.71:0/2315693543 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780043b0 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f716c005590 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f715fe00640 1 -- 172.21.15.71:0/2315693543 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f7178003800 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f715fe00640 1 -- 172.21.15.71:0/2315693543 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7178003a00 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f7187fff740 10 client.26750 fetch_fsmap learned FSMap version 49 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5614b3806510 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f715fe00640 1 -- 172.21.15.71:0/2315693543 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f716c005590 con 0x5614b37801e0 2024-09-05T12:39:29.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.381+0000 7f715fe00640 1 -- 172.21.15.71:0/2315693543 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 49) ==== 8382+0+0 (secure 0 0 0) 0x7f7178019430 con 0x5614b37801e0 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f7187fff740 10 client.26750 fetch_fsmap finished waiting for FSMap version 49 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f7187fff740 10 client.26750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f7187fff740 20 client.26750 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f7187fff740 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b3806950 0x5614b3808d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f7187fff740 4 client.26750 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5614b3777d80 con 0x5614b3806950 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f717d400640 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b3806950 0x5614b3808d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f717d400640 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b3806950 0x5614b3808d70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bbf50294b5481bd server_cookie=3a7c8ac72304f376 in_seq=0 out_seq=0 2024-09-05T12:39:29.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.382+0000 7f715fe00640 10 client.26750 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.424+0000 7f715fe00640 1 -- 172.21.15.71:0/2315693543 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5614b3777d80 con 0x5614b3806950 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.424+0000 7f715fe00640 10 client.26750 handle_command_reply: tid=0 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.424+0000 7f7187fff740 1 client.26750 shutdown 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.424+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 msgr2=0x5614b378f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.424+0000 7f7187fff740 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b378f3d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7178014040 tx=0x7f7178011810 comp rx=0 tx=0).stop 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 shutdown_connections 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5614b3806950 0x5614b3808d70 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614b378b3a0 0x5614b3804e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614b3780b00 0x5614b378f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 --2- 172.21.15.71:0/2315693543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5614b37801e0 0x5614b378f3d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f715fe00640 0 client.26750 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 >> 172.21.15.71:0/2315693543 conn(0x5614b377c750 msgr2=0x5614b3788b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 shutdown_connections 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 1 -- 172.21.15.71:0/2315693543 wait complete. 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.425+0000 7f7187fff740 20 client.26750 trim_cache size 0 max 16384 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:39:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:39:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:39:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71640a2da0 msgr2=0x7f71640a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71640a2da0 0x7f71640a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f71700b90a0 tx=0x7f71700f6510 comp rx=0 tx=0).stop 2024-09-05T12:39:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 msgr2=0x7f718019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f718019a5c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7170005e30 tx=0x7f7170003f90 comp rx=0 tx=0).stop 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 shutdown_connections 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f718019ee80 0x7f718019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71801090a0 0x7f718019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71640a2da0 0x7f71640a5260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 --2- 172.21.15.71:0/1258154287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7180103f50 0x7f718019a5c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 >> 172.21.15.71:0/1258154287 conn(0x7f71800ff9e0 msgr2=0x7f7180100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 shutdown_connections 2024-09-05T12:39:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:29.426+0000 7f7185000640 1 -- 172.21.15.71:0/1258154287 wait complete. 2024-09-05T12:39:30.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404195 3.28 MB/sec execute 506 sec latency 481.117 ms 2024-09-05T12:39:30.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:30 smithi174 ceph-mon[34313]: pgmap v368: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-09-05T12:39:30.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:30 smithi071 ceph-mon[29845]: pgmap v368: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-09-05T12:39:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:30 smithi123 ceph-mon[34321]: pgmap v368: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-09-05T12:39:31.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404775 3.28 MB/sec execute 507 sec latency 501.212 ms 2024-09-05T12:39:32.280 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404775 3.28 MB/sec execute 508 sec latency 1501.362 ms 2024-09-05T12:39:32.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:32 smithi174 ceph-mon[34313]: pgmap v369: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.5 MiB/s wr, 320 op/s 2024-09-05T12:39:32.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:32 smithi071 ceph-mon[29845]: pgmap v369: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.5 MiB/s wr, 320 op/s 2024-09-05T12:39:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:32 smithi123 ceph-mon[34321]: pgmap v369: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 1.5 MiB/s wr, 320 op/s 2024-09-05T12:39:33.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404775 3.27 MB/sec execute 509 sec latency 2501.488 ms 2024-09-05T12:39:33.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:33 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:39:33.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:33 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:39:33.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:33 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:39:34.187 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:39:34.187 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:39:34.188 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:39:34.279 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404775 3.26 MB/sec execute 510 sec latency 3501.549 ms 2024-09-05T12:39:34.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:34 smithi174 ceph-mon[34313]: pgmap v370: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 209 op/s 2024-09-05T12:39:34.574 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:39:34.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:34 smithi071 ceph-mon[29845]: pgmap v370: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 209 op/s 2024-09-05T12:39:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:34 smithi123 ceph-mon[34321]: pgmap v370: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 209 op/s 2024-09-05T12:39:35.280 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404775 3.26 MB/sec execute 511 sec latency 4501.746 ms 2024-09-05T12:39:35.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 -- 172.21.15.71:0/1091892184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408077040 msgr2=0x7f7408112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:35.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 --2- 172.21.15.71:0/1091892184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408077040 0x7f7408112330 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009900 tx=0x7f73f002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 -- 172.21.15.71:0/1091892184 shutdown_connections 2024-09-05T12:39:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 --2- 172.21.15.71:0/1091892184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408112870 0x7f7408114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 --2- 172.21.15.71:0/1091892184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408077040 0x7f7408112330 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 --2- 172.21.15.71:0/1091892184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 0x7f7408076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 -- 172.21.15.71:0/1091892184 >> 172.21.15.71:0/1091892184 conn(0x7f74080ff8d0 msgr2=0x7f7408101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 -- 172.21.15.71:0/1091892184 shutdown_connections 2024-09-05T12:39:35.413 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.411+0000 7f7407e00640 1 -- 172.21.15.71:0/1091892184 wait complete. 2024-09-05T12:39:35.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 Processor -- start 2024-09-05T12:39:35.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 -- start start 2024-09-05T12:39:35.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 0x7f74081a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:35.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408077040 0x7f74081a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 0x7f740819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7408118440 con 0x7f7408112870 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74081182c0 con 0x7f7408076720 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.412+0000 7f7407e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74081185c0 con 0x7f7408077040 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 0x7f740819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7406a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 0x7f74081a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 0x7f740819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46388/0 (socket says 172.21.15.71:46388) 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 -- 172.21.15.71:0/4257059619 learned_addr learned my addr 172.21.15.71:0/4257059619 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7406a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 0x7f74081a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:36146/0 (socket says 172.21.15.71:36146) 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f73fe000640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408077040 0x7f74081a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 -- 172.21.15.71:0/4257059619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408077040 msgr2=0x7f74081a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408077040 0x7f74081a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 -- 172.21.15.71:0/4257059619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 msgr2=0x7f74081a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 0x7f74081a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 -- 172.21.15.71:0/4257059619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740819efa0 con 0x7f7408112870 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f73fe000640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408077040 0x7f74081a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.413+0000 7f7407400640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 0x7f740819e730 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f740000eb10 tx=0x7f740000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.414+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400004b20 con 0x7f7408112870 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.414+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400004cc0 con 0x7f7408112870 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.414+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740819f290 con 0x7f7408112870 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.414+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400010790 con 0x7f7408112870 2024-09-05T12:39:35.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.414+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081ab5e0 con 0x7f7408112870 2024-09-05T12:39:35.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.415+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7408077e60 con 0x7f7408112870 2024-09-05T12:39:35.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.417+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7400005020 con 0x7f7408112870 2024-09-05T12:39:35.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.418+0000 7f740ca00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2cf0 0x7f73ec0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:35.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.418+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f74000cffa0 con 0x7f7408112870 2024-09-05T12:39:35.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.418+0000 7f7406a00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2cf0 0x7f73ec0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:35.420 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.419+0000 7f7406a00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2cf0 0x7f73ec0a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009bd0 tx=0x7f73f8005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:35.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.422+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f740009c7d0 con 0x7f7408112870 2024-09-05T12:39:35.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.695+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f740810da00 con 0x7f7408112870 2024-09-05T12:39:35.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.699+0000 7f740ca00640 1 -- 172.21.15.71:0/4257059619 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7f740009c0d0 con 0x7f7408112870 2024-09-05T12:39:35.705 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:39:35.705 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":50,"btime":"2024-09-05T12:39:32:235089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:39:31.831476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 50 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.704+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2cf0 msgr2=0x7f73ec0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.704+0000 7f7407e00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2cf0 0x7f73ec0a51b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f73f8009bd0 tx=0x7f73f8005f90 comp rx=0 tx=0).stop 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.704+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 msgr2=0x7f740819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.704+0000 7f7407e00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 0x7f740819e730 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f740000eb10 tx=0x7f740000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 shutdown_connections 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f73ec0a2cf0 0x7f73ec0a51b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7408112870 0x7f740819e730 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7408077040 0x7f74081a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 --2- 172.21.15.71:0/4257059619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7408076720 0x7f74081a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:35.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 >> 172.21.15.71:0/4257059619 conn(0x7f74080ff8d0 msgr2=0x7f7408113470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:35.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 shutdown_connections 2024-09-05T12:39:35.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:35.705+0000 7f7407e00640 1 -- 172.21.15.71:0/4257059619 wait complete. 2024-09-05T12:39:36.291 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.25 MB/sec execute 512 sec latency 5098.288 ms 2024-09-05T12:39:36.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:36 smithi071 ceph-mon[29845]: pgmap v371: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 209 op/s 2024-09-05T12:39:36.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:36 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4257059619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:39:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:36 smithi123 ceph-mon[34321]: pgmap v371: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 209 op/s 2024-09-05T12:39:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:36 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4257059619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:39:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:36 smithi174 ceph-mon[34313]: pgmap v371: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.0 MiB/s wr, 209 op/s 2024-09-05T12:39:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:36 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4257059619' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:39:36.889 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:39:36.890 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:39:37.285 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:39:37.304 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.25 MB/sec execute 513 sec latency 1415.349 ms 2024-09-05T12:39:38.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 -- 172.21.15.71:0/3686004927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee410e280 msgr2=0x7efee4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:38.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 --2- 172.21.15.71:0/3686004927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee410e280 0x7efee4110670 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7efedc009be0 tx=0x7efedc02a770 comp rx=0 tx=0).stop 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 -- 172.21.15.71:0/3686004927 shutdown_connections 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 --2- 172.21.15.71:0/3686004927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee410e280 0x7efee4110670 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 --2- 172.21.15.71:0/3686004927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 --2- 172.21.15.71:0/3686004927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee4074510 0x7efee40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 -- 172.21.15.71:0/3686004927 >> 172.21.15.71:0/3686004927 conn(0x7efee40fd740 msgr2=0x7efee40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 -- 172.21.15.71:0/3686004927 shutdown_connections 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.032+0000 7efee3e00640 1 -- 172.21.15.71:0/3686004927 wait complete. 2024-09-05T12:39:38.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.033+0000 7efee3e00640 1 Processor -- start 2024-09-05T12:39:38.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 -- start start 2024-09-05T12:39:38.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 0x7efee419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee410e280 0x7efee419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efee4114030 con 0x7efee4074510 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee4113eb0 con 0x7efee410e280 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efee41141b0 con 0x7efee4074e30 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 0x7efee419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 0x7efee419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46408/0 (socket says 172.21.15.71:46408) 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33550/0 (socket says 172.21.15.71:33550) 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.034+0000 7efee2a00640 1 -- 172.21.15.71:0/3030209096 learned_addr learned my addr 172.21.15.71:0/3030209096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2000640 1 -- 172.21.15.71:0/3030209096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee410e280 msgr2=0x7efee419ee80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2000640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee410e280 0x7efee419ee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2000640 1 -- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 msgr2=0x7efee419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:38.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2000640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 0x7efee419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2000640 1 -- 172.21.15.71:0/3030209096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee419f560 con 0x7efee4074e30 2024-09-05T12:39:38.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2a00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 0x7efee419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:38.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efee2000640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee419aaf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efed000d750 tx=0x7efed000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:38.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.035+0000 7efedb400640 1 -- 172.21.15.71:0/3030209096 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0017070 con 0x7efee4074e30 2024-09-05T12:39:38.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.036+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee419f850 con 0x7efee4074e30 2024-09-05T12:39:38.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.036+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41a7130 con 0x7efee4074e30 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.036+0000 7efedb400640 1 -- 172.21.15.71:0/3030209096 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed0005020 con 0x7efee4074e30 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.036+0000 7efedb400640 1 -- 172.21.15.71:0/3030209096 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0004cd0 con 0x7efee4074e30 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 Processor -- start 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 -- start start 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 0x5590e1ce2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 0x5590e1ced200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ced740 0x5590e1cefb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5590e1c48530 con 0x5590e1ce2560 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5590e1bef860 con 0x5590e1ce2e80 2024-09-05T12:39:38.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efeec4bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590e1c444f0 con 0x5590e1ced740 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efee3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 0x5590e1ce2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efee3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 0x5590e1ce2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46416/0 (socket says 172.21.15.71:46416) 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.039+0000 7efee3400640 1 -- 172.21.15.71:0/3913728777 learned_addr learned my addr 172.21.15.71:0/3913728777 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efedb400640 1 -- 172.21.15.71:0/3030209096 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7efed0010460 con 0x7efee4074e30 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee2000640 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ced740 0x5590e1cefb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee3400640 1 -- 172.21.15.71:0/3913728777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ced740 msgr2=0x5590e1cefb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:38.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee3400640 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ced740 0x5590e1cefb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee3400640 1 -- 172.21.15.71:0/3913728777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 msgr2=0x5590e1ced200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee3400640 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 0x5590e1ced200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee3400640 1 -- 172.21.15.71:0/3913728777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e19b9b90 con 0x5590e1ce2560 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.040+0000 7efee2000640 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ced740 0x5590e1cefb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efee3400640 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 0x5590e1ce2940 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efedc02ad60 tx=0x7efedc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ace9a5e58b84e26 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efecfe00640 1 -- 172.21.15.71:0/3913728777 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc040020 con 0x5590e1ce2560 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efecfe00640 1 -- 172.21.15.71:0/3913728777 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efedc030d70 con 0x5590e1ce2560 2024-09-05T12:39:38.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efecfe00640 1 -- 172.21.15.71:0/3913728777 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc036040 con 0x5590e1ce2560 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efedb400640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efec80a2d70 0x7efec80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efeec4bc740 1 -- 172.21.15.71:0/3913728777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 msgr2=0x5590e1ce2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.041+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 0x5590e1ce2940 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7efedc02ad60 tx=0x7efedc005e80 comp rx=0 tx=0).stop 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efedb400640 1 -- 172.21.15.71:0/3030209096 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7efed00cf070 con 0x7efee4074e30 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 -- 172.21.15.71:0/3913728777 shutdown_connections 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ced740 0x5590e1cefb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 0x5590e1ced200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3913728777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2560 0x5590e1ce2940 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 -- 172.21.15.71:0/3913728777 >> 172.21.15.71:0/3913728777 conn(0x5590e1cdeaf0 msgr2=0x5590e1cdeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efee2a00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efec80a2d70 0x7efec80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 -- 172.21.15.71:0/3913728777 shutdown_connections 2024-09-05T12:39:38.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.042+0000 7efeec4bc740 1 -- 172.21.15.71:0/3913728777 wait complete. 2024-09-05T12:39:38.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 -- start start 2024-09-05T12:39:38.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efee2a00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efec80a2d70 0x7efec80a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efed4009d60 tx=0x7efed4009190 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:38.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 Processor -- start 2024-09-05T12:39:38.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1ceaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1ceb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5590e1cf0300 con 0x5590e1ce2e80 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5590e1c02e60 con 0x5590e1ced740 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5590e1cf0480 con 0x5590e1ce2560 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.043+0000 7efeec4bc740 10 client.0 _ll_register_callbacks cb 0x5590e1c07cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1ceaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1ceaae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46430/0 (socket says 172.21.15.71:46430) 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 -- 172.21.15.71:0/563161231 learned_addr learned my addr 172.21.15.71:0/563161231 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee3400640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1ceb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2a00640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 -- 172.21.15.71:0/563161231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 msgr2=0x5590e1cea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 -- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 msgr2=0x5590e1ceb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1ceb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee2000640 1 -- 172.21.15.71:0/563161231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e1c0e990 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.044+0000 7efee3400640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1ceb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efee2000640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1ceaae0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efed0009610 tx=0x7efed009ae10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c777aaa5e584f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeccc00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeccc00640 1 -- 172.21.15.71:0/563161231 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0020070 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeccc00640 1 -- 172.21.15.71:0/563161231 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7efed00cc650 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efedc030d70 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeccc00640 1 -- 172.21.15.71:0/563161231 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7efed00cc850 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeec4bc740 10 client.16728 fetch_fsmap learned FSMap version 50 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5590e1cfbfb0 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.045+0000 7efeccc00640 1 -- 172.21.15.71:0/563161231 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efedc030d70 con 0x5590e1ce2e80 2024-09-05T12:39:38.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeccc00640 1 -- 172.21.15.71:0/563161231 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7efed009f160 con 0x5590e1ce2e80 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeec4bc740 10 client.16728 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeec4bc740 10 client.16728 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeec4bc740 20 client.16728 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeec4bc740 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1cfd510 0x5590e1cff950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeec4bc740 4 client.16728 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e1beb310 con 0x5590e1cfd510 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.046+0000 7efee2a00640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1cfd510 0x5590e1cff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.047+0000 7efeccc00640 10 client.16728 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.047+0000 7efee2a00640 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1cfd510 0x5590e1cff950 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6099847c7b2d6b2c server_cookie=92552f0f63b6eaa0 in_seq=0 out_seq=0 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.047+0000 7efeccc00640 1 -- 172.21.15.71:0/563161231 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590e1beb310 con 0x5590e1cfd510 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.047+0000 7efeccc00640 10 client.16728 handle_command_reply: tid=0 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.047+0000 7efeec4bc740 1 client.16728 shutdown 2024-09-05T12:39:38.055 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 msgr2=0x5590e1ceaae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1ceaae0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7efed0009610 tx=0x7efed009ae10 comp rx=0 tx=0).stop 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 shutdown_connections 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1cfd510 0x5590e1cff950 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1ceb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1ceaae0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 --2- 172.21.15.71:0/563161231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 >> 172.21.15.71:0/563161231 conn(0x5590e1cdeaf0 msgr2=0x5590e1ce4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeccc00640 0 client.16728 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 shutdown_connections 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 1 -- 172.21.15.71:0/563161231 wait complete. 2024-09-05T12:39:38.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.048+0000 7efeec4bc740 20 client.16728 trim_cache size 0 max 16384 2024-09-05T12:39:38.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.065+0000 7efeec4bc740 1 Processor -- start 2024-09-05T12:39:38.066 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.065+0000 7efeec4bc740 1 -- start start 2024-09-05T12:39:38.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1ce5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1d05220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1d05760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efeec4bc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efec400fe90 con 0x5590e1ce2e80 2024-09-05T12:39:38.067 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efeec4bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efec4011a60 con 0x5590e1ced740 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efeec4bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec40105f0 con 0x5590e1ce2560 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1d05220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1d05220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46436/0 (socket says 172.21.15.71:46436) 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 -- 172.21.15.71:0/4002640230 learned_addr learned my addr 172.21.15.71:0/4002640230 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2a00640 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1ce5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 -- 172.21.15.71:0/4002640230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 msgr2=0x5590e1ce5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1ce5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 -- 172.21.15.71:0/4002640230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 msgr2=0x5590e1d05760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1d05760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.066+0000 7efee2000640 1 -- 172.21.15.71:0/4002640230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e1ce2cb0 con 0x5590e1ce2e80 2024-09-05T12:39:38.068 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.067+0000 7efee2000640 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1d05220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efed0097600 tx=0x7efed0009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e867e7205be7cf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:38.069 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.067+0000 7efecf400640 1 -- 172.21.15.71:0/4002640230 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed009b1a0 con 0x5590e1ce2e80 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.067+0000 7efecf400640 1 -- 172.21.15.71:0/4002640230 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00b9020 con 0x5590e1ce2e80 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.067+0000 7efecf400640 1 -- 172.21.15.71:0/4002640230 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00bb0b0 con 0x5590e1ce2e80 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 -- 172.21.15.71:0/4002640230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 msgr2=0x5590e1d05220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1d05220 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7efed0097600 tx=0x7efed0009740 comp rx=0 tx=0).stop 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 -- 172.21.15.71:0/4002640230 shutdown_connections 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ced740 0x5590e1d05760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ce2e80 0x5590e1d05220 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 --2- 172.21.15.71:0/4002640230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1ce5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 -- 172.21.15.71:0/4002640230 >> 172.21.15.71:0/4002640230 conn(0x5590e1cdeaf0 msgr2=0x5590e1ce9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 -- 172.21.15.71:0/4002640230 shutdown_connections 2024-09-05T12:39:38.070 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 -- 172.21.15.71:0/4002640230 wait complete. 2024-09-05T12:39:38.071 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.068+0000 7efeec4bc740 1 -- start start 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 Processor -- start 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cf1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 0x5590e1cf21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ced740 0x5590e1d671f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efec4011390 con 0x5590e1ced740 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efec4010cc0 con 0x5590e1ce2e80 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efec400de90 con 0x5590e1ce2560 2024-09-05T12:39:38.072 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efeec4bc740 10 client.0 _ll_register_callbacks cb 0x7efed009afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efee2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cf1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efee2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cf1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33584/0 (socket says 172.21.15.71:33584) 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.069+0000 7efee2a00640 1 -- 172.21.15.71:0/3430670881 learned_addr learned my addr 172.21.15.71:0/3430670881 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.070+0000 7efee3400640 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ced740 0x5590e1d671f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.070+0000 7efee2a00640 1 -- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 msgr2=0x5590e1cf21e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.070+0000 7efee2a00640 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 0x5590e1cf21e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.070+0000 7efee2a00640 1 -- 172.21.15.71:0/3430670881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ced740 msgr2=0x5590e1d671f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.070+0000 7efee2a00640 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ced740 0x5590e1d671f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:38.073 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.070+0000 7efee2a00640 1 -- 172.21.15.71:0/3430670881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e1d677e0 con 0x5590e1ce2560 2024-09-05T12:39:38.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.077+0000 7efee2a00640 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cf1ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efed400eeb0 tx=0x7efed4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f044452863237fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:38.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.077+0000 7efeccc00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:39:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeccc00640 1 -- 172.21.15.71:0/3430670881 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed401b040 con 0x5590e1ce2560 2024-09-05T12:39:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeccc00640 1 -- 172.21.15.71:0/3430670881 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7efed4021630 con 0x5590e1ce2560 2024-09-05T12:39:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00b9020 con 0x5590e1ce2560 2024-09-05T12:39:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeccc00640 1 -- 172.21.15.71:0/3430670881 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7efed4005970 con 0x5590e1ce2560 2024-09-05T12:39:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 10 client.26788 fetch_fsmap learned FSMap version 50 2024-09-05T12:39:38.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5590e1d68b00 con 0x5590e1ce2560 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeccc00640 1 -- 172.21.15.71:0/3430670881 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efed00b9020 con 0x5590e1ce2560 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeccc00640 1 -- 172.21.15.71:0/3430670881 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7efed4006160 con 0x5590e1ce2560 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 10 client.26788 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 10 client.26788 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 20 client.26788 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1d69320 0x5590e1d6b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 4 client.26788 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:39:38.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.078+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5590e1c55c60 con 0x5590e1d69320 2024-09-05T12:39:38.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.079+0000 7efee2000640 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1d69320 0x5590e1d6b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:38.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.079+0000 7efee2000640 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1d69320 0x5590e1d6b740 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be8490d96873fbfc server_cookie=fafc89c647843a77 in_seq=0 out_seq=0 2024-09-05T12:39:38.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:38.079+0000 7efeccc00640 10 client.26788 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:38.291 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.24 MB/sec execute 514 sec latency 2415.388 ms 2024-09-05T12:39:38.469 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:38 smithi071 ceph-mon[29845]: pgmap v372: 129 pgs: 129 active+clean; 138 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-09-05T12:39:38.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:38 smithi174 ceph-mon[34313]: pgmap v372: 129 pgs: 129 active+clean; 138 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-09-05T12:39:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:38 smithi123 ceph-mon[34321]: pgmap v372: 129 pgs: 129 active+clean; 138 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 227 op/s 2024-09-05T12:39:39.291 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.23 MB/sec execute 515 sec latency 3415.460 ms 2024-09-05T12:39:40.291 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.23 MB/sec execute 516 sec latency 4415.500 ms 2024-09-05T12:39:40.512 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:40 smithi071 ceph-mon[29845]: pgmap v373: 129 pgs: 129 active+clean; 138 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 889 KiB/s wr, 117 op/s 2024-09-05T12:39:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:40 smithi123 ceph-mon[34321]: pgmap v373: 129 pgs: 129 active+clean; 138 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 889 KiB/s wr, 117 op/s 2024-09-05T12:39:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:40 smithi174 ceph-mon[34313]: pgmap v373: 129 pgs: 129 active+clean; 138 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 889 KiB/s wr, 117 op/s 2024-09-05T12:39:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeccc00640 1 -- 172.21.15.71:0/3430670881 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5590e1c55c60 con 0x5590e1d69320 2024-09-05T12:39:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeccc00640 10 client.26788 handle_command_reply: tid=0 2024-09-05T12:39:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 client.26788 shutdown 2024-09-05T12:39:41.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 msgr2=0x5590e1cf1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cf1ca0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efed400eeb0 tx=0x7efed4011000 comp rx=0 tx=0).stop 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 shutdown_connections 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5590e1d69320 0x5590e1d6b740 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590e1ced740 0x5590e1d671f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5590e1ce2e80 0x5590e1cf21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeccc00640 0 client.26788 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 --2- 172.21.15.71:0/3430670881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5590e1ce2560 0x5590e1cf1ca0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.052+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 >> 172.21.15.71:0/3430670881 conn(0x5590e1cdeaf0 msgr2=0x5590e1ce8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 shutdown_connections 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efeec4bc740 1 -- 172.21.15.71:0/3430670881 wait complete. 2024-09-05T12:39:41.054 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efeec4bc740 20 client.26788 trim_cache size 0 max 16384 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 50, 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 38, 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "e6b00742": { 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 38, 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:39:41.055 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:39:41.056 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efec80a2d70 msgr2=0x7efec80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efec80a2d70 0x7efec80a5230 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efed4009d60 tx=0x7efed4009190 comp rx=0 tx=0).stop 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 msgr2=0x7efee419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee419aaf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7efed000d750 tx=0x7efed000dc10 comp rx=0 tx=0).stop 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 shutdown_connections 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee410e280 0x7efee419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee4074e30 0x7efee419aaf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7efec80a2d70 0x7efec80a5230 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 --2- 172.21.15.71:0/3030209096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efee4074510 0x7efee419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:41.057 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 >> 172.21.15.71:0/3030209096 conn(0x7efee40fd740 msgr2=0x7efee40fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:41.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 shutdown_connections 2024-09-05T12:39:41.058 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:39:41.053+0000 7efee3e00640 1 -- 172.21.15.71:0/3030209096 wait complete. 2024-09-05T12:39:41.291 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.22 MB/sec execute 517 sec latency 5415.540 ms 2024-09-05T12:39:42.186 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e6b00742', quiesce took 8.0 seconds. Will release after: 22.0 2024-09-05T12:39:42.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.21 MB/sec execute 518 sec latency 6415.604 ms 2024-09-05T12:39:42.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:42 smithi071 ceph-mon[29845]: pgmap v374: 129 pgs: 129 active+clean; 137 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-05T12:39:42.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 -- 172.21.15.71:0/2469003698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164076720 msgr2=0x7f5164076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 --2- 172.21.15.71:0/2469003698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164076720 0x7f5164076b00 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5154009860 tx=0x7f515402a340 comp rx=0 tx=0).stop 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 -- 172.21.15.71:0/2469003698 shutdown_connections 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 --2- 172.21.15.71:0/2469003698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5164112960 0x7f5164114d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 --2- 172.21.15.71:0/2469003698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5164077040 0x7f5164112420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 --2- 172.21.15.71:0/2469003698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164076720 0x7f5164076b00 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 -- 172.21.15.71:0/2469003698 >> 172.21.15.71:0/2469003698 conn(0x7f51640ff980 msgr2=0x7f5164101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 -- 172.21.15.71:0/2469003698 shutdown_connections 2024-09-05T12:39:42.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.540+0000 7f516ae00640 1 -- 172.21.15.71:0/2469003698 wait complete. 2024-09-05T12:39:42.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.541+0000 7f516ae00640 1 Processor -- start 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.541+0000 7f516ae00640 1 -- start start 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f516ae00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5164076720 0x7f51641a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f516ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5164077040 0x7f51641a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f516ae00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 0x7f516419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f516ae00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51641184c0 con 0x7f5164112960 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f516ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5164118340 con 0x7f5164077040 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f516ae00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5164118640 con 0x7f5164076720 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 0x7f516419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 0x7f516419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46464/0 (socket says 172.21.15.71:46464) 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 -- 172.21.15.71:0/2613075007 learned_addr learned my addr 172.21.15.71:0/2613075007 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5163e00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5164076720 0x7f51641a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5163400640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5164077040 0x7f51641a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 -- 172.21.15.71:0/2613075007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5164076720 msgr2=0x7f51641a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5164076720 0x7f51641a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 -- 172.21.15.71:0/2613075007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5164077040 msgr2=0x7f51641a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5164077040 0x7f51641a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.542+0000 7f5168e00640 1 -- 172.21.15.71:0/2613075007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516419efa0 con 0x7f5164112960 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.543+0000 7f5168e00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 0x7f516419e7d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f515c00eb40 tx=0x7f515c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.543+0000 7f5160c00640 1 -- 172.21.15.71:0/2613075007 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c004b20 con 0x7f5164112960 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.543+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516419f290 con 0x7f5164112960 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.543+0000 7f5160c00640 1 -- 172.21.15.71:0/2613075007 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f515c004cc0 con 0x7f5164112960 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.543+0000 7f5160c00640 1 -- 172.21.15.71:0/2613075007 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c0107f0 con 0x7f5164112960 2024-09-05T12:39:42.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.543+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641ab4b0 con 0x7f5164112960 2024-09-05T12:39:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.546+0000 7f516dc45740 1 Processor -- start 2024-09-05T12:39:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.546+0000 7f5160c00640 1 -- 172.21.15.71:0/2613075007 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f515c005000 con 0x7f5164112960 2024-09-05T12:39:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.546+0000 7f516dc45740 1 -- start start 2024-09-05T12:39:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f516dc45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a94f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f516dc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a959e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f516dc45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f516dc45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55763a8b5100 con 0x55763a94f1e0 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f516dc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55763a85d1b0 con 0x55763a94fb00 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f516dc45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55763a8b10c0 con 0x55763a95a3a0 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a94f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a94f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46472/0 (socket says 172.21.15.71:46472) 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 -- 172.21.15.71:0/3339031306 learned_addr learned my addr 172.21.15.71:0/3339031306 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163e00640 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a959e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 -- 172.21.15.71:0/3339031306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 msgr2=0x55763a95c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 -- 172.21.15.71:0/3339031306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 msgr2=0x55763a959e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a959e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.547+0000 7f5163400640 1 -- 172.21.15.71:0/3339031306 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55763a5c8210 con 0x55763a94f1e0 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5160c00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f51480a2da0 0x7f51480a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5163400640 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a94f5c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5150005a90 tx=0x7f5150005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f785c0f0fe6dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5160c00640 1 -- 172.21.15.71:0/2613075007 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f515c0cff30 con 0x7f5164112960 2024-09-05T12:39:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5163e00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f51480a2da0 0x7f51480a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5159600640 1 -- 172.21.15.71:0/3339031306 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150004020 con 0x55763a94f1e0 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5159600640 1 -- 172.21.15.71:0/3339031306 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51500041c0 con 0x55763a94f1e0 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.548+0000 7f5159600640 1 -- 172.21.15.71:0/3339031306 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515000f870 con 0x55763a94f1e0 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 -- 172.21.15.71:0/3339031306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 msgr2=0x55763a94f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a94f5c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5150005a90 tx=0x7f5150005a00 comp rx=0 tx=0).stop 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f5163e00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f51480a2da0 0x7f51480a5260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f515402a930 tx=0x7f51540058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 -- 172.21.15.71:0/3339031306 shutdown_connections 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a959e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 --2- 172.21.15.71:0/3339031306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a94f5c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 -- 172.21.15.71:0/3339031306 >> 172.21.15.71:0/3339031306 conn(0x55763a94b750 msgr2=0x55763a94bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 -- 172.21.15.71:0/3339031306 shutdown_connections 2024-09-05T12:39:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 -- 172.21.15.71:0/3339031306 wait complete. 2024-09-05T12:39:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 -- start start 2024-09-05T12:39:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.549+0000 7f516dc45740 1 Processor -- start 2024-09-05T12:39:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a9572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a94fb00 0x55763a957830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a95a3a0 0x55763a957d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55763a95cf70 con 0x55763a94f1e0 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55763a83e090 con 0x55763a95a3a0 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55763a95d0f0 con 0x55763a94fb00 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f516dc45740 10 client.0 _ll_register_callbacks cb 0x55763a892f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f5163e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a9572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f5163e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a9572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46480/0 (socket says 172.21.15.71:46480) 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f5163e00640 1 -- 172.21.15.71:0/3221644938 learned_addr learned my addr 172.21.15.71:0/3221644938 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f5168e00640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a95a3a0 0x55763a957d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.550+0000 7f5163400640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a94fb00 0x55763a957830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5163e00640 1 -- 172.21.15.71:0/3221644938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a94fb00 msgr2=0x55763a957830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5163e00640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a94fb00 0x55763a957830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5163e00640 1 -- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a95a3a0 msgr2=0x55763a957d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5163e00640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a95a3a0 0x55763a957d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5163e00640 1 -- 172.21.15.71:0/3221644938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763a694620 con 0x55763a94f1e0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5168e00640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a95a3a0 0x55763a957d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f5163e00640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a9572f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5154009860 tx=0x7f515402a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e40f3cddbc19e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f514e000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f514e000640 1 -- 172.21.15.71:0/3221644938 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515404c020 con 0x55763a94f1e0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f514e000640 1 -- 172.21.15.71:0/3221644938 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f5154003a00 con 0x55763a94f1e0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51500041c0 con 0x55763a94f1e0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f514e000640 1 -- 172.21.15.71:0/3221644938 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5154057050 con 0x55763a94f1e0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f516dc45740 10 client.16746 fetch_fsmap learned FSMap version 50 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.551+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55763a968ae0 con 0x55763a94f1e0 2024-09-05T12:39:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f514e000640 1 -- 172.21.15.71:0/3221644938 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51500041c0 con 0x55763a94f1e0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f514e000640 1 -- 172.21.15.71:0/3221644938 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f515403dc30 con 0x55763a94f1e0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f516dc45740 10 client.16746 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f516dc45740 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f516dc45740 20 client.16746 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f516dc45740 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a969ff0 0x55763a96c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f516dc45740 4 client.16746 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55763a8c2830 con 0x55763a969ff0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.552+0000 7f5163400640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a969ff0 0x55763a96c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.553+0000 7f5163400640 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a969ff0 0x55763a96c420 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8030b4b5fa8ee8c server_cookie=6b1f6d3d0bfd80c2 in_seq=0 out_seq=0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.553+0000 7f514e000640 10 client.16746 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f514e000640 1 -- 172.21.15.71:0/3221644938 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55763a8c2830 con 0x55763a969ff0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f514e000640 10 client.16746 handle_command_reply: tid=0 2024-09-05T12:39:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 client.16746 shutdown 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 msgr2=0x55763a9572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a9572f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5154009860 tx=0x7f515402a340 comp rx=0 tx=0).stop 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 shutdown_connections 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a969ff0 0x55763a96c420 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a95a3a0 0x55763a957d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a94fb00 0x55763a957830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f514e000640 0 client.16746 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 --2- 172.21.15.71:0/3221644938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a9572f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 >> 172.21.15.71:0/3221644938 conn(0x55763a94b750 msgr2=0x55763a9514b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 shutdown_connections 2024-09-05T12:39:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 1 -- 172.21.15.71:0/3221644938 wait complete. 2024-09-05T12:39:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.554+0000 7f516dc45740 20 client.16746 trim_cache size 0 max 16384 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 Processor -- start 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 -- start start 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a959a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a951350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a971d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b54a0 con 0x55763a94f1e0 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b7070 con 0x55763a94fb00 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f516dc45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b5c00 con 0x55763a95a3a0 2024-09-05T12:39:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f5163e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a959a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f5163e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a959a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46492/0 (socket says 172.21.15.71:46492) 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.569+0000 7f5163e00640 1 -- 172.21.15.71:0/2905688314 learned_addr learned my addr 172.21.15.71:0/2905688314 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5168e00640 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a971d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163400640 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a951350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163e00640 1 -- 172.21.15.71:0/2905688314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 msgr2=0x55763a971d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163e00640 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a971d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163e00640 1 -- 172.21.15.71:0/2905688314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 msgr2=0x55763a951350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163e00640 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a951350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163e00640 1 -- 172.21.15.71:0/2905688314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55763a94b580 con 0x55763a94f1e0 2024-09-05T12:39:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5168e00640 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a971d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5163e00640 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a959a50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5154030b20 tx=0x7f5154030920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd444cad4932646 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5158c00640 1 -- 172.21.15.71:0/2905688314 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5154060070 con 0x55763a94f1e0 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5158c00640 1 -- 172.21.15.71:0/2905688314 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5154043070 con 0x55763a94f1e0 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.570+0000 7f5158c00640 1 -- 172.21.15.71:0/2905688314 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515404c040 con 0x55763a94f1e0 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 -- 172.21.15.71:0/2905688314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 msgr2=0x55763a959a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a959a50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f5154030b20 tx=0x7f5154030920 comp rx=0 tx=0).stop 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 -- 172.21.15.71:0/2905688314 shutdown_connections 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a971d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94fb00 0x55763a951350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 --2- 172.21.15.71:0/2905688314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94f1e0 0x55763a959a50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 -- 172.21.15.71:0/2905688314 >> 172.21.15.71:0/2905688314 conn(0x55763a94b750 msgr2=0x55763a950c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:42.574 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 -- 172.21.15.71:0/2905688314 shutdown_connections 2024-09-05T12:39:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 -- 172.21.15.71:0/2905688314 wait complete. 2024-09-05T12:39:42.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 -- start start 2024-09-05T12:39:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.571+0000 7f516dc45740 1 Processor -- start 2024-09-05T12:39:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94f1e0 0x55763a962980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 0x55763a95dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b69a0 con 0x55763a94fb00 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b62d0 con 0x55763a94f1e0 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f51641b34a0 con 0x55763a95a3a0 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f516dc45740 10 client.0 _ll_register_callbacks cb 0x7f5150005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5163400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 0x55763a95dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5163e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94f1e0 0x55763a962980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5163400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 0x55763a95dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46494/0 (socket says 172.21.15.71:46494) 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5168e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5163400640 1 -- 172.21.15.71:0/830333871 learned_addr learned my addr 172.21.15.71:0/830333871 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5163400640 1 -- 172.21.15.71:0/830333871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 msgr2=0x55763a95e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.572+0000 7f5163400640 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f5163400640 1 -- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94f1e0 msgr2=0x55763a962980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f5163400640 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94f1e0 0x55763a962980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f5163400640 1 -- 172.21.15.71:0/830333871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55763a946d60 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f5163400640 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 0x55763a95dca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f51500093d0 tx=0x7f5150003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb64be0b586232f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f514e000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f514e000640 1 -- 172.21.15.71:0/830333871 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150005d10 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f514e000640 1 -- 172.21.15.71:0/830333871 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f5150004080 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5154043070 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f514e000640 1 -- 172.21.15.71:0/830333871 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f5150038050 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f516dc45740 10 client.16752 fetch_fsmap learned FSMap version 50 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55763a9d4b40 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.573+0000 7f514e000640 1 -- 172.21.15.71:0/830333871 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5154043070 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f514e000640 1 -- 172.21.15.71:0/830333871 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f515001d020 con 0x55763a94fb00 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f516dc45740 10 client.16752 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f516dc45740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f516dc45740 20 client.16752 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f516dc45740 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a9d5120 0x55763a9d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f516dc45740 4 client.16752 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:39:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55763a8bcb80 con 0x55763a9d5120 2024-09-05T12:39:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f5163e00640 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a9d5120 0x55763a9d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f514e000640 10 client.16752 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.574+0000 7f5163e00640 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a9d5120 0x55763a9d7540 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92284995dd4f68d3 server_cookie=64600e56ab1c17b5 in_seq=0 out_seq=0 2024-09-05T12:39:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f514e000640 1 -- 172.21.15.71:0/830333871 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55763a8bcb80 con 0x55763a9d5120 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f514e000640 10 client.16752 handle_command_reply: tid=0 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 client.16752 shutdown 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 msgr2=0x55763a95dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 0x55763a95dca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f51500093d0 tx=0x7f5150003f70 comp rx=0 tx=0).stop 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 shutdown_connections 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55763a9d5120 0x55763a9d7540 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55763a95a3a0 0x55763a95e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f514e000640 0 client.16752 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55763a94fb00 0x55763a95dca0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 --2- 172.21.15.71:0/830333871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55763a94f1e0 0x55763a962980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 >> 172.21.15.71:0/830333871 conn(0x55763a94b750 msgr2=0x55763a957ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 shutdown_connections 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 1 -- 172.21.15.71:0/830333871 wait complete. 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.599+0000 7f516dc45740 20 client.16752 trim_cache size 0 max 16384 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:39:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f51480a2da0 msgr2=0x7f51480a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f51480a2da0 0x7f51480a5260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f515402a930 tx=0x7f51540058d0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 msgr2=0x7f516419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 0x7f516419e7d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f515c00eb40 tx=0x7f515c00c2d0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 shutdown_connections 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f51480a2da0 0x7f51480a5260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5164112960 0x7f516419e7d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5164077040 0x7f51641a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 --2- 172.21.15.71:0/2613075007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5164076720 0x7f51641a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 >> 172.21.15.71:0/2613075007 conn(0x7f51640ff980 msgr2=0x7f5164101d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.600+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 shutdown_connections 2024-09-05T12:39:42.604 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:42.601+0000 7f516ae00640 1 -- 172.21.15.71:0/2613075007 wait complete. 2024-09-05T12:39:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:42 smithi123 ceph-mon[34321]: pgmap v374: 129 pgs: 129 active+clean; 137 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-05T12:39:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:42 smithi174 ceph-mon[34313]: pgmap v374: 129 pgs: 129 active+clean; 137 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 177 op/s 2024-09-05T12:39:43.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.21 MB/sec execute 519 sec latency 7415.712 ms 2024-09-05T12:39:44.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.20 MB/sec execute 520 sec latency 8415.782 ms 2024-09-05T12:39:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:44 smithi071 ceph-mon[29845]: pgmap v375: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-05T12:39:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:44 smithi123 ceph-mon[34321]: pgmap v375: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-05T12:39:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:44 smithi174 ceph-mon[34313]: pgmap v375: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-05T12:39:45.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.20 MB/sec execute 521 sec latency 9415.900 ms 2024-09-05T12:39:46.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.19 MB/sec execute 522 sec latency 10416.027 ms 2024-09-05T12:39:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:46 smithi071 ceph-mon[29845]: pgmap v376: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-05T12:39:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:46 smithi123 ceph-mon[34321]: pgmap v376: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-05T12:39:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:46 smithi174 ceph-mon[34313]: pgmap v376: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 2.1 MiB/s wr, 94 op/s 2024-09-05T12:39:47.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.18 MB/sec execute 523 sec latency 11416.074 ms 2024-09-05T12:39:48.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.18 MB/sec execute 524 sec latency 12416.237 ms 2024-09-05T12:39:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:48 smithi071 ceph-mon[29845]: pgmap v377: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-05T12:39:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:48 smithi123 ceph-mon[34321]: pgmap v377: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-05T12:39:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:48 smithi174 ceph-mon[34313]: pgmap v377: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 2.1 MiB/s wr, 108 op/s 2024-09-05T12:39:49.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.17 MB/sec execute 525 sec latency 13416.370 ms 2024-09-05T12:39:50.292 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.17 MB/sec execute 526 sec latency 14416.452 ms 2024-09-05T12:39:50.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:50 smithi174 ceph-mon[34313]: pgmap v378: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-05T12:39:50.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:50 smithi071 ceph-mon[29845]: pgmap v378: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-05T12:39:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:50 smithi123 ceph-mon[34321]: pgmap v378: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 1.7 MiB/s wr, 90 op/s 2024-09-05T12:39:51.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.16 MB/sec execute 527 sec latency 15416.535 ms 2024-09-05T12:39:52.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.15 MB/sec execute 528 sec latency 16416.857 ms 2024-09-05T12:39:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:52 smithi071 ceph-mon[29845]: pgmap v379: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-05T12:39:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:52 smithi123 ceph-mon[34321]: pgmap v379: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-05T12:39:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:52 smithi174 ceph-mon[34313]: pgmap v379: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-09-05T12:39:53.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.15 MB/sec execute 529 sec latency 17417.017 ms 2024-09-05T12:39:54.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.14 MB/sec execute 530 sec latency 18417.095 ms 2024-09-05T12:39:54.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:54 smithi071 ceph-mon[29845]: pgmap v380: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 303 KiB/s wr, 36 op/s 2024-09-05T12:39:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:54 smithi123 ceph-mon[34321]: pgmap v380: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 303 KiB/s wr, 36 op/s 2024-09-05T12:39:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:54 smithi174 ceph-mon[34313]: pgmap v380: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 303 KiB/s wr, 36 op/s 2024-09-05T12:39:55.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.14 MB/sec execute 531 sec latency 19417.162 ms 2024-09-05T12:39:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 -- 172.21.15.71:0/3682051494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 msgr2=0x7f2bd8114b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/3682051494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 0x7f2bd8114b70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009be0 tx=0x7f2bd002a6c0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 -- 172.21.15.71:0/3682051494 shutdown_connections 2024-09-05T12:39:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/3682051494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 0x7f2bd8114b70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/3682051494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd8077040 0x7f2bd8112240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.725 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/3682051494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 -- 172.21.15.71:0/3682051494 >> 172.21.15.71:0/3682051494 conn(0x7f2bd80ff7c0 msgr2=0x7f2bd8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 -- 172.21.15.71:0/3682051494 shutdown_connections 2024-09-05T12:39:55.726 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.723+0000 7f2bd7e00640 1 -- 172.21.15.71:0/3682051494 wait complete. 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 Processor -- start 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 -- start start 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd81a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd8077040 0x7f2bd81a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 0x7f2bd819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8118290 con 0x7f2bd8076720 2024-09-05T12:39:55.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8118110 con 0x7f2bd8112780 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.724+0000 7f2bd7e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8118410 con 0x7f2bd8077040 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd81a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd81a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36916/0 (socket says 172.21.15.71:36916) 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 0x7f2bd819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 -- 172.21.15.71:0/275937958 learned_addr learned my addr 172.21.15.71:0/275937958 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd8077040 0x7f2bd81a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 -- 172.21.15.71:0/275937958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd8077040 msgr2=0x7f2bd81a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd8077040 0x7f2bd81a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 -- 172.21.15.71:0/275937958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 msgr2=0x7f2bd819e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 0x7f2bd819e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.725+0000 7f2bd6a00640 1 -- 172.21.15.71:0/275937958 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd819ef40 con 0x7f2bd8076720 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.726+0000 7f2bd6a00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd81a45c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e9c0 tx=0x7f2bc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.726+0000 7f2bdca00640 1 -- 172.21.15.71:0/275937958 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ccc0 con 0x7f2bd8076720 2024-09-05T12:39:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.726+0000 7f2bdca00640 1 -- 172.21.15.71:0/275937958 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc8005500 con 0x7f2bd8076720 2024-09-05T12:39:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.726+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd819f230 con 0x7f2bd8076720 2024-09-05T12:39:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.726+0000 7f2bdca00640 1 -- 172.21.15.71:0/275937958 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8010430 con 0x7f2bd8076720 2024-09-05T12:39:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.726+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81ab540 con 0x7f2bd8076720 2024-09-05T12:39:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 Processor -- start 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2bdca00640 1 -- 172.21.15.71:0/275937958 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2bc80046c0 con 0x7f2bd8076720 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 -- start start 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e93790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88ea0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55be88df9100 con 0x55be88e933b0 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55be88da11b0 con 0x55be88e9e570 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.729+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55be88df50c0 con 0x55be88e93cd0 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e93790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88ea0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e93790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36918/0 (socket says 172.21.15.71:36918) 2024-09-05T12:39:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 -- 172.21.15.71:0/2904968634 learned_addr learned my addr 172.21.15.71:0/2904968634 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 -- 172.21.15.71:0/2904968634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 msgr2=0x55be88e9e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 -- 172.21.15.71:0/2904968634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 msgr2=0x55be88ea0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88ea0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd7400640 1 -- 172.21.15.71:0/2904968634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be88b0c210 con 0x55be88e933b0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bd6a00640 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.730+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88ea0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e93790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8075b10 tx=0x7f2bd0005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e6b78da09e981fd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bdca00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bbc0a2cf0 0x7f2bbc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bc7e00640 1 -- 172.21.15.71:0/2904968634 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0042030 con 0x55be88e933b0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bc7e00640 1 -- 172.21.15.71:0/2904968634 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0037040 con 0x55be88e933b0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bc7e00640 1 -- 172.21.15.71:0/2904968634 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0036070 con 0x55be88e933b0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bdca00640 1 -- 172.21.15.71:0/275937958 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f2bc80cfb30 con 0x7f2bd8076720 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.731+0000 7f2bd6a00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bbc0a2cf0 0x7f2bbc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2904968634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 msgr2=0x55be88e93790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2bd6a00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bbc0a2cf0 0x7f2bbc0a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8098630 tx=0x7f2bc80d35e0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:55.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e93790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8075b10 tx=0x7f2bd0005c60 comp rx=0 tx=0).stop 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2904968634 shutdown_connections 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88ea0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2904968634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e93790 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2904968634 >> 172.21.15.71:0/2904968634 conn(0x55be88e8f920 msgr2=0x55be88e8fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2904968634 shutdown_connections 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2904968634 wait complete. 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.732+0000 7f2be0ca0740 1 -- start start 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 Processor -- start 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88e9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 0x55be88e9c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55be88ea0ff0 con 0x55be88e9e570 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55be88d82090 con 0x55be88e933b0 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55be88ea1170 con 0x55be88e93cd0 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2be0ca0740 10 client.0 _ll_register_callbacks cb 0x55be88dd6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 0x55be88e9c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 0x55be88e9c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36930/0 (socket says 172.21.15.71:36930) 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.733+0000 7f2bd7400640 1 -- 172.21.15.71:0/2943766947 learned_addr learned my addr 172.21.15.71:0/2943766947 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88e9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 -- 172.21.15.71:0/2943766947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 msgr2=0x55be88e9bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 -- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 msgr2=0x55be88e9b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88e9b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 -- 172.21.15.71:0/2943766947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be88bd8620 con 0x55be88e9e570 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bd7400640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 0x55be88e9c040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0003ab0 tx=0x7f2bd0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddf3f75e0042797 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bc4c00640 1 -- 172.21.15.71:0/2943766947 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0036050 con 0x55be88e9e570 2024-09-05T12:39:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2bc4c00640 1 -- 172.21.15.71:0/2943766947 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f2bd0030e60 con 0x55be88e9e570 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.734+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd0037040 con 0x55be88e9e570 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2bc4c00640 1 -- 172.21.15.71:0/2943766947 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2bd0057050 con 0x55be88e9e570 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 10 client.16764 fetch_fsmap learned FSMap version 50 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55be88eacdb0 con 0x55be88e9e570 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2bc4c00640 1 -- 172.21.15.71:0/2943766947 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0037040 con 0x55be88e9e570 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2bc4c00640 1 -- 172.21.15.71:0/2943766947 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f2bd003c040 con 0x55be88e9e570 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 10 client.16764 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 20 client.16764 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88eae100 0x55be88eb04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 4 client.16764 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.735+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be88e06830 con 0x55be88eae100 2024-09-05T12:39:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.736+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88eae100 0x55be88eb04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.736+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88eae100 0x55be88eb04f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11cac9d469e668d4 server_cookie=8c87090fb8ade8ab in_seq=0 out_seq=0 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.736+0000 7f2bc4c00640 10 client.16764 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2bc4c00640 1 -- 172.21.15.71:0/2943766947 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be88e06830 con 0x55be88eae100 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2bc4c00640 10 client.16764 handle_command_reply: tid=0 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 client.16764 shutdown 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 msgr2=0x55be88e9c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 0x55be88e9c040 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0003ab0 tx=0x7f2bd0003ce0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 shutdown_connections 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88eae100 0x55be88eb04f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e9e570 0x55be88e9c040 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/2943766947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88e9b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 >> 172.21.15.71:0/2943766947 conn(0x55be88e8f920 msgr2=0x55be88e95630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2bc4c00640 0 client.16764 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 shutdown_connections 2024-09-05T12:39:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.737+0000 7f2be0ca0740 1 -- 172.21.15.71:0/2943766947 wait complete. 2024-09-05T12:39:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.738+0000 7f2be0ca0740 20 client.16764 trim_cache size 0 max 16384 2024-09-05T12:39:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.752+0000 7f2be0ca0740 1 Processor -- start 2024-09-05T12:39:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 -- start start 2024-09-05T12:39:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e8a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88eb5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b5460 con 0x55be88e933b0 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b7030 con 0x55be88e9e570 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b5bc0 con 0x55be88e93cd0 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e8a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e8a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36942/0 (socket says 172.21.15.71:36942) 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 -- 172.21.15.71:0/3177640667 learned_addr learned my addr 172.21.15.71:0/3177640667 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 -- 172.21.15.71:0/3177640667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 msgr2=0x55be88e9bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 -- 172.21.15.71:0/3177640667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 msgr2=0x55be88eb5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88eb5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.753+0000 7f2bd7400640 1 -- 172.21.15.71:0/3177640667 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be88e9a570 con 0x55be88e933b0 2024-09-05T12:39:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.754+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e8a8d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0003c20 tx=0x7f2bd0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be1581bdf3a2a58 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.754+0000 7f2bc7400640 1 -- 172.21.15.71:0/3177640667 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0036070 con 0x55be88e933b0 2024-09-05T12:39:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.754+0000 7f2bc7400640 1 -- 172.21.15.71:0/3177640667 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd004b030 con 0x55be88e933b0 2024-09-05T12:39:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.754+0000 7f2bc7400640 1 -- 172.21.15.71:0/3177640667 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0042040 con 0x55be88e933b0 2024-09-05T12:39:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3177640667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 msgr2=0x55be88e8a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e8a8d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0003c20 tx=0x7f2bd0003b20 comp rx=0 tx=0).stop 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3177640667 shutdown_connections 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e9e570 0x55be88eb5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e93cd0 0x55be88e9bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3177640667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e933b0 0x55be88e8a8d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3177640667 >> 172.21.15.71:0/3177640667 conn(0x55be88e8f920 msgr2=0x55be88e94550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3177640667 shutdown_connections 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3177640667 wait complete. 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 -- start start 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.755+0000 7f2be0ca0740 1 Processor -- start 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88ea6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 0x55be88ea6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e9e570 0x55be88ea3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b6960 con 0x55be88e93cd0 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b6290 con 0x55be88e933b0 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b3460 con 0x55be88e9e570 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2be0ca0740 10 client.0 _ll_register_callbacks cb 0x7f2bc0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 0x55be88ea6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 0x55be88ea6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36946/0 (socket says 172.21.15.71:36946) 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 -- 172.21.15.71:0/3215566092 learned_addr learned my addr 172.21.15.71:0/3215566092 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88ea6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e9e570 0x55be88ea3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 -- 172.21.15.71:0/3215566092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e9e570 msgr2=0x55be88ea3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e9e570 0x55be88ea3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 -- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 msgr2=0x55be88ea6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88ea6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bcfe00640 1 -- 172.21.15.71:0/3215566092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be88ea3ae0 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.756+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88ea6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2bcfe00640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 0x55be88ea6b80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0004700 tx=0x7f2bc001f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5f13c75a9d3a1c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2bc4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2bc4c00640 1 -- 172.21.15.71:0/3215566092 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0004230 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2bc4c00640 1 -- 172.21.15.71:0/3215566092 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7f2bc00043d0 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd004b030 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2bc4c00640 1 -- 172.21.15.71:0/3215566092 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f2bc0010880 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2be0ca0740 10 client.16770 fetch_fsmap learned FSMap version 50 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55be88f19570 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.757+0000 7f2bc4c00640 1 -- 172.21.15.71:0/3215566092 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd004b030 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2bc4c00640 1 -- 172.21.15.71:0/3215566092 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f2bc0004830 con 0x55be88e93cd0 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2be0ca0740 10 client.16770 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2be0ca0740 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:39:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2be0ca0740 20 client.16770 populate_metadata read hostname 'smithi071' 2024-09-05T12:39:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88f19d20 0x55be88f1c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:39:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2be0ca0740 4 client.16770 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:39:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55be88e00b80 con 0x55be88f19d20 2024-09-05T12:39:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.758+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88f19d20 0x55be88f1c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:39:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.759+0000 7f2bd7400640 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88f19d20 0x55be88f1c140 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa75a0fb776438f server_cookie=b790829c829ea038 in_seq=0 out_seq=0 2024-09-05T12:39:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.759+0000 7f2bc4c00640 10 client.16770 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:55.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2bc4c00640 1 -- 172.21.15.71:0/3215566092 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55be88e00b80 con 0x55be88f19d20 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2bc4c00640 10 client.16770 handle_command_reply: tid=0 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 client.16770 shutdown 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 msgr2=0x55be88ea6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 0x55be88ea6b80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0004700 tx=0x7f2bc001f990 comp rx=0 tx=0).stop 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 shutdown_connections 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55be88f19d20 0x55be88f1c140 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be88e9e570 0x55be88ea3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be88e93cd0 0x55be88ea6b80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2bc4c00640 0 client.16770 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 --2- 172.21.15.71:0/3215566092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55be88e933b0 0x55be88ea6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 >> 172.21.15.71:0/3215566092 conn(0x55be88e8f920 msgr2=0x55be88e95630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 shutdown_connections 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 1 -- 172.21.15.71:0/3215566092 wait complete. 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.779+0000 7f2be0ca0740 20 client.16770 trim_cache size 0 max 16384 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:39:55.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bbc0a2cf0 msgr2=0x7f2bbc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bbc0a2cf0 0x7f2bbc0a51b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8098630 tx=0x7f2bc80d35e0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 msgr2=0x7f2bd81a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd81a45c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2bc800e9c0 tx=0x7f2bc800ee80 comp rx=0 tx=0).stop 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 shutdown_connections 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bd8112780 0x7f2bd819e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd8077040 0x7f2bd81a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2bbc0a2cf0 0x7f2bbc0a51b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 --2- 172.21.15.71:0/275937958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bd8076720 0x7f2bd81a45c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:39:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 >> 172.21.15.71:0/275937958 conn(0x7f2bd80ff7c0 msgr2=0x7f2bd8114540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:39:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 shutdown_connections 2024-09-05T12:39:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:39:55.780+0000 7f2bd7e00640 1 -- 172.21.15.71:0/275937958 wait complete. 2024-09-05T12:39:56.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.13 MB/sec execute 532 sec latency 20417.209 ms 2024-09-05T12:39:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:56 smithi071 ceph-mon[29845]: pgmap v381: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-09-05T12:39:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:56 smithi123 ceph-mon[34321]: pgmap v381: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-09-05T12:39:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:56 smithi174 ceph-mon[34313]: pgmap v381: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-09-05T12:39:57.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.12 MB/sec execute 533 sec latency 21417.307 ms 2024-09-05T12:39:58.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.12 MB/sec execute 534 sec latency 22417.419 ms 2024-09-05T12:39:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:39:58 smithi071 ceph-mon[29845]: pgmap v382: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 75 KiB/s wr, 27 op/s 2024-09-05T12:39:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:39:58 smithi123 ceph-mon[34321]: pgmap v382: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 75 KiB/s wr, 27 op/s 2024-09-05T12:39:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:39:58 smithi174 ceph-mon[34313]: pgmap v382: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 75 KiB/s wr, 27 op/s 2024-09-05T12:39:59.293 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.11 MB/sec execute 535 sec latency 23417.504 ms 2024-09-05T12:40:00.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.11 MB/sec execute 536 sec latency 24417.590 ms 2024-09-05T12:40:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:00 smithi071 ceph-mon[29845]: pgmap v383: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T12:40:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:00 smithi071 ceph-mon[29845]: overall HEALTH_OK 2024-09-05T12:40:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:00 smithi123 ceph-mon[34321]: pgmap v383: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T12:40:00.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:00 smithi123 ceph-mon[34321]: overall HEALTH_OK 2024-09-05T12:40:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:00 smithi174 ceph-mon[34313]: pgmap v383: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 26 KiB/s wr, 12 op/s 2024-09-05T12:40:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:00 smithi174 ceph-mon[34313]: overall HEALTH_OK 2024-09-05T12:40:01.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.10 MB/sec execute 537 sec latency 25417.680 ms 2024-09-05T12:40:02.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.09 MB/sec execute 538 sec latency 26417.761 ms 2024-09-05T12:40:02.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:02 smithi071 ceph-mon[29845]: pgmap v384: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-05T12:40:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:02 smithi123 ceph-mon[34321]: pgmap v384: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-05T12:40:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:02 smithi174 ceph-mon[34313]: pgmap v384: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 52 KiB/s wr, 20 op/s 2024-09-05T12:40:03.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.09 MB/sec execute 539 sec latency 27417.880 ms 2024-09-05T12:40:04.187 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:40:04.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.08 MB/sec execute 540 sec latency 28417.973 ms 2024-09-05T12:40:04.568 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:40:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:04 smithi071 ceph-mon[29845]: pgmap v385: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 72 KiB/s wr, 32 op/s 2024-09-05T12:40:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:04 smithi123 ceph-mon[34321]: pgmap v385: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 72 KiB/s wr, 32 op/s 2024-09-05T12:40:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:04 smithi174 ceph-mon[34313]: pgmap v385: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 72 KiB/s wr, 32 op/s 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.250+0000 7fd98de00640 1 -- 172.21.15.71:0/3345879228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 msgr2=0x7fd98810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.250+0000 7fd98de00640 1 --2- 172.21.15.71:0/3345879228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 0x7fd98810b9a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd974009900 tx=0x7fd97402a480 comp rx=0 tx=0).stop 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 -- 172.21.15.71:0/3345879228 shutdown_connections 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 --2- 172.21.15.71:0/3345879228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd98810e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 --2- 172.21.15.71:0/3345879228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 0x7fd98810b9a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 --2- 172.21.15.71:0/3345879228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd988102b90 0x7fd988102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 -- 172.21.15.71:0/3345879228 >> 172.21.15.71:0/3345879228 conn(0x7fd9880fc610 msgr2=0x7fd9880fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:05.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 -- 172.21.15.71:0/3345879228 shutdown_connections 2024-09-05T12:40:05.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.251+0000 7fd98de00640 1 -- 172.21.15.71:0/3345879228 wait complete. 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 Processor -- start 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 -- start start 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd988102b90 0x7fd9881007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 0x7fd988100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd988101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd988111e10 con 0x7fd9881034b0 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd988111c90 con 0x7fd98810bfc0 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98de00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd988111f90 con 0x7fd988102b90 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd987e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 0x7fd988100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:05.255 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.252+0000 7fd98d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd988101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd988101270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51958/0 (socket says 172.21.15.71:51958) 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 -- 172.21.15.71:0/4189283177 learned_addr learned my addr 172.21.15.71:0/4189283177 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98ca00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd988102b90 0x7fd9881007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 -- 172.21.15.71:0/4189283177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd988102b90 msgr2=0x7fd9881007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd988102b90 0x7fd9881007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 -- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 msgr2=0x7fd988100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 0x7fd988100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 -- 172.21.15.71:0/4189283177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9881a67b0 con 0x7fd98810bfc0 2024-09-05T12:40:05.256 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd98d400640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd988101270 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd98000eb10 tx=0x7fd98000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.253+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980004b20 con 0x7fd98810bfc0 2024-09-05T12:40:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.254+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881a6aa0 con 0x7fd98810bfc0 2024-09-05T12:40:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.254+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd980004cc0 con 0x7fd98810bfc0 2024-09-05T12:40:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.254+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980010760 con 0x7fd98810bfc0 2024-09-05T12:40:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.254+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881a6f60 con 0x7fd98810bfc0 2024-09-05T12:40:05.257 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.255+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd988102f70 con 0x7fd98810bfc0 2024-09-05T12:40:05.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.257+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd980005000 con 0x7fd98810bfc0 2024-09-05T12:40:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.258+0000 7fd985600640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd9700a2cf0 0x7fd9700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.258+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fd9800cff50 con 0x7fd98810bfc0 2024-09-05T12:40:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.258+0000 7fd98ca00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd9700a2cf0 0x7fd9700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:05.260 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.259+0000 7fd98ca00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd9700a2cf0 0x7fd9700a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009bd0 tx=0x7fd97c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:05.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.260+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd9800541b0 con 0x7fd98810bfc0 2024-09-05T12:40:05.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.08 MB/sec execute 541 sec latency 29418.057 ms 2024-09-05T12:40:05.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.472+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd988101d20 con 0x7fd98810bfc0 2024-09-05T12:40:05.475 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.473+0000 7fd985600640 1 -- 172.21.15.71:0/4189283177 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7fd988101d20 con 0x7fd98810bfc0 2024-09-05T12:40:05.475 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:40:05.475 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":50,"btime":"2024-09-05T12:39:32:235089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:39:31.831476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:40:05.475 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 50 2024-09-05T12:40:05.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd9700a2cf0 msgr2=0x7fd9700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:05.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd9700a2cf0 0x7fd9700a51b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009bd0 tx=0x7fd97c005f90 comp rx=0 tx=0).stop 2024-09-05T12:40:05.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 msgr2=0x7fd988101270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:05.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd988101270 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd98000eb10 tx=0x7fd98000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 shutdown_connections 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd98810bfc0 0x7fd988101270 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd9700a2cf0 0x7fd9700a51b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9881034b0 0x7fd988100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 --2- 172.21.15.71:0/4189283177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd988102b90 0x7fd9881007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 >> 172.21.15.71:0/4189283177 conn(0x7fd9880fc610 msgr2=0x7fd9880fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 shutdown_connections 2024-09-05T12:40:05.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:05.475+0000 7fd98de00640 1 -- 172.21.15.71:0/4189283177 wait complete. 2024-09-05T12:40:06.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.07 MB/sec execute 542 sec latency 30418.126 ms 2024-09-05T12:40:06.309 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id e6b00742 --release --await' 2024-09-05T12:40:06.309 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id e6b00742 --release --await 2024-09-05T12:40:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:06 smithi071 ceph-mon[29845]: pgmap v386: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 67 KiB/s wr, 30 op/s 2024-09-05T12:40:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:06 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4189283177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:06.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:06 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:40:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:06 smithi123 ceph-mon[34321]: pgmap v386: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 67 KiB/s wr, 30 op/s 2024-09-05T12:40:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:06 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4189283177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:06 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:40:06.688 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:06 smithi174 ceph-mon[34313]: pgmap v386: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 67 KiB/s wr, 30 op/s 2024-09-05T12:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:06 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4189283177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:06 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:40:07.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 404776 3.07 MB/sec execute 543 sec latency 31418.187 ms 2024-09-05T12:40:07.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 -- 172.21.15.71:0/1708830442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee0410e450 msgr2=0x7fee04110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 --2- 172.21.15.71:0/1708830442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee0410e450 0x7fee04110840 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009ca0 tx=0x7fedfc02a950 comp rx=0 tx=0).stop 2024-09-05T12:40:07.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 -- 172.21.15.71:0/1708830442 shutdown_connections 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 --2- 172.21.15.71:0/1708830442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee0410e450 0x7fee04110840 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 --2- 172.21.15.71:0/1708830442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee04074e30 0x7fee0410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 --2- 172.21.15.71:0/1708830442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee04074510 0x7fee040748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 -- 172.21.15.71:0/1708830442 >> 172.21.15.71:0/1708830442 conn(0x7fee040fd8f0 msgr2=0x7fee040ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 -- 172.21.15.71:0/1708830442 shutdown_connections 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.330+0000 7fee0aa00640 1 -- 172.21.15.71:0/1708830442 wait complete. 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.331+0000 7fee0aa00640 1 Processor -- start 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.331+0000 7fee0aa00640 1 -- start start 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.331+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee04074510 0x7fee0419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.331+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 0x7fee0419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee0aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0410e450 0x7fee0419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee0aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fee041141b0 con 0x7fee04074e30 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee0aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee04114030 con 0x7fee04074510 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee0aa00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee04114330 con 0x7fee0410e450 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 0x7fee0419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 0x7fee0419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52730/0 (socket says 172.21.15.71:52730) 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 -- 172.21.15.71:0/269759447 learned_addr learned my addr 172.21.15.71:0/269759447 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03e00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee04074510 0x7fee0419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee08a00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0410e450 0x7fee0419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 -- 172.21.15.71:0/269759447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0410e450 msgr2=0x7fee0419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0410e450 0x7fee0419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 -- 172.21.15.71:0/269759447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee04074510 msgr2=0x7fee0419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee04074510 0x7fee0419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.332+0000 7fee03400640 1 -- 172.21.15.71:0/269759447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0419f640 con 0x7fee04074e30 2024-09-05T12:40:07.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.333+0000 7fee03400640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 0x7fee0419abd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e9f0 tx=0x7fedf000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.333+0000 7fee00c00640 1 -- 172.21.15.71:0/269759447 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf000cd80 con 0x7fee04074e30 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.333+0000 7fee00c00640 1 -- 172.21.15.71:0/269759447 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf0010910 con 0x7fee04074e30 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.333+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0419f930 con 0x7fee04074e30 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.333+0000 7fee00c00640 1 -- 172.21.15.71:0/269759447 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0018990 con 0x7fee04074e30 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.333+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee041a7210 con 0x7fee04074e30 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.336+0000 7fee0d5bd740 1 Processor -- start 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 -- start start 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d6fb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fbc30 0x558c6d705fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d7064f0 0x558c6d7088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558c6d661650 con 0x558c6d7064f0 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c6d608f10 con 0x558c6d6fb310 2024-09-05T12:40:07.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c6d65d610 con 0x558c6d6fbc30 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee08a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d6fb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee08a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d6fb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51986/0 (socket says 172.21.15.71:51986) 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee08a00640 1 -- 172.21.15.71:0/2806936068 learned_addr learned my addr 172.21.15.71:0/2806936068 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.337+0000 7fee03e00640 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fbc30 0x558c6d705fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee08a00640 1 -- 172.21.15.71:0/2806936068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fbc30 msgr2=0x558c6d705fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee08a00640 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fbc30 0x558c6d705fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee08a00640 1 -- 172.21.15.71:0/2806936068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d7064f0 msgr2=0x558c6d7088e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee08a00640 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d7064f0 0x558c6d7088e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee08a00640 1 -- 172.21.15.71:0/2806936068 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6d370580 con 0x558c6d6fb310 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee08a00640 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d6fb6f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fee040754f0 tx=0x7fedfc009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1b24986d92cbd9c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fee00c00640 1 -- 172.21.15.71:0/269759447 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fedf0010ac0 con 0x7fee04074e30 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fedf9200640 1 -- 172.21.15.71:0/2806936068 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc034040 con 0x558c6d6fb310 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fedf9200640 1 -- 172.21.15.71:0/2806936068 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedfc003e50 con 0x558c6d6fb310 2024-09-05T12:40:07.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.338+0000 7fedf9200640 1 -- 172.21.15.71:0/2806936068 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0042a0 con 0x558c6d6fb310 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2806936068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 msgr2=0x558c6d6fb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d6fb6f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fee040754f0 tx=0x7fedfc009a20 comp rx=0 tx=0).stop 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2806936068 shutdown_connections 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d7064f0 0x558c6d7088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fbc30 0x558c6d705fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2806936068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d6fb6f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2806936068 >> 172.21.15.71:0/2806936068 conn(0x558c6d6f78a0 msgr2=0x558c6d6f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2806936068 shutdown_connections 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2806936068 wait complete. 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.339+0000 7fee00c00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fede80a2c00 0x7fede80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.340+0000 7fee00c00640 1 -- 172.21.15.71:0/269759447 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fedf00cfb80 con 0x7fee04074e30 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.340+0000 7fee03e00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fede80a2c00 0x7fede80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.340+0000 7fee0d5bd740 1 -- start start 2024-09-05T12:40:07.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.340+0000 7fee0d5bd740 1 Processor -- start 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d703430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03e00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fede80a2c00 0x7fede80a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009cc0 tx=0x7fedf4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d703970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d703eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558c6d709060 con 0x558c6d6fbc30 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c6d61c350 con 0x558c6d6fb310 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c6d7091e0 con 0x558c6d7064f0 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee0d5bd740 10 client.0 _ll_register_callbacks cb 0x558c6d63f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d703970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d703970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52746/0 (socket says 172.21.15.71:52746) 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 -- 172.21.15.71:0/2189527959 learned_addr learned my addr 172.21.15.71:0/2189527959 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:07.357 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03e00640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d703430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee08a00640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d703eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 -- 172.21.15.71:0/2189527959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 msgr2=0x558c6d703eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d703eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 -- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 msgr2=0x558c6d703430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d703430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03400640 1 -- 172.21.15.71:0/2189527959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6d43a960 con 0x558c6d6fbc30 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.341+0000 7fee03e00640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d703430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7fee03400640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d703970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e9c0 tx=0x7fedf009b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eea03a9e7853d439 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7feded600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7feded600640 1 -- 172.21.15.71:0/2189527959 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf00a0060 con 0x558c6d6fbc30 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7feded600640 1 -- 172.21.15.71:0/2189527959 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fedf0018b30 con 0x558c6d6fbc30 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedfc003e50 con 0x558c6d6fbc30 2024-09-05T12:40:07.358 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7feded600640 1 -- 172.21.15.71:0/2189527959 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fedf00a4920 con 0x558c6d6fbc30 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7fee0d5bd740 10 client.16785 fetch_fsmap learned FSMap version 50 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.342+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558c6d714cc0 con 0x558c6d6fbc30 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7feded600640 1 -- 172.21.15.71:0/2189527959 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedfc003e50 con 0x558c6d6fbc30 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7feded600640 1 -- 172.21.15.71:0/2189527959 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7fedf009b100 con 0x558c6d6fbc30 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee0d5bd740 10 client.16785 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee0d5bd740 10 client.16785 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee0d5bd740 20 client.16785 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d716180 0x558c6d718570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee0d5bd740 4 client.16785 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c6d66ebc0 con 0x558c6d716180 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.343+0000 7fee03e00640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d716180 0x558c6d718570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.344+0000 7fee03e00640 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d716180 0x558c6d718570 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7bb8e789621a1fc server_cookie=8cb1da7dae39bca3 in_seq=0 out_seq=0 2024-09-05T12:40:07.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.344+0000 7feded600640 10 client.16785 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.344+0000 7feded600640 1 -- 172.21.15.71:0/2189527959 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c6d66ebc0 con 0x558c6d716180 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.344+0000 7feded600640 10 client.16785 handle_command_reply: tid=0 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 client.16785 shutdown 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 msgr2=0x558c6d703970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d703970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e9c0 tx=0x7fedf009b700 comp rx=0 tx=0).stop 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 shutdown_connections 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d716180 0x558c6d718570 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d703eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d703970 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2189527959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fb310 0x558c6d703430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 >> 172.21.15.71:0/2189527959 conn(0x558c6d6f78a0 msgr2=0x558c6d6f6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7feded600640 0 client.16785 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 shutdown_connections 2024-09-05T12:40:07.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2189527959 wait complete. 2024-09-05T12:40:07.361 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.345+0000 7fee0d5bd740 20 client.16785 trim_cache size 0 max 16384 2024-09-05T12:40:07.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 Processor -- start 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 -- start start 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fb310 0x558c6d71dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fbc30 0x558c6d71e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d6f6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b1220 con 0x558c6d6fb310 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b2df0 con 0x558c6d6fbc30 2024-09-05T12:40:07.365 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.363+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b1980 con 0x558c6d7064f0 2024-09-05T12:40:07.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fb310 0x558c6d71dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.366 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fb310 0x558c6d71dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52762/0 (socket says 172.21.15.71:52762) 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03e00640 1 -- 172.21.15.71:0/2909483009 learned_addr learned my addr 172.21.15.71:0/2909483009 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fbc30 0x558c6d71e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee08a00640 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d6f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 -- 172.21.15.71:0/2909483009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 msgr2=0x558c6d6f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d6f6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 -- 172.21.15.71:0/2909483009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fb310 msgr2=0x558c6d71dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fb310 0x558c6d71dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 -- 172.21.15.71:0/2909483009 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c6d7019a0 con 0x558c6d6fbc30 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee08a00640 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d6f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.364+0000 7fee03400640 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fbc30 0x558c6d71e2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedf009b9b0 tx=0x7fedf009be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bc976835fe939b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fedefe00640 1 -- 172.21.15.71:0/2909483009 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf009b100 con 0x558c6d6fbc30 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fedefe00640 1 -- 172.21.15.71:0/2909483009 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf0097ad0 con 0x558c6d6fbc30 2024-09-05T12:40:07.367 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fedefe00640 1 -- 172.21.15.71:0/2909483009 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf0018d90 con 0x558c6d6fbc30 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2909483009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fbc30 msgr2=0x558c6d71e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fbc30 0x558c6d71e2b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fedf009b9b0 tx=0x7fedf009be30 comp rx=0 tx=0).stop 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2909483009 shutdown_connections 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d7064f0 0x558c6d6f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d6fbc30 0x558c6d71e2b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/2909483009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fb310 0x558c6d71dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.365+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2909483009 >> 172.21.15.71:0/2909483009 conn(0x558c6d6f78a0 msgr2=0x558c6d6fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:07.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.366+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2909483009 shutdown_connections 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.366+0000 7fee0d5bd740 1 -- 172.21.15.71:0/2909483009 wait complete. 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.366+0000 7fee0d5bd740 1 -- start start 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.366+0000 7fee0d5bd740 1 Processor -- start 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fb310 0x558c6d70ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d700250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d7064f0 0x558c6d7007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b2720 con 0x558c6d6fbc30 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee041b2050 con 0x558c6d7064f0 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee041af220 con 0x558c6d6fb310 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee0d5bd740 10 client.0 _ll_register_callbacks cb 0x7fedf0005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d700250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee03e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fb310 0x558c6d70ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee03400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d700250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52770/0 (socket says 172.21.15.71:52770) 2024-09-05T12:40:07.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee03e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fb310 0x558c6d70ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42528/0 (socket says 172.21.15.71:42528) 2024-09-05T12:40:07.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee03400640 1 -- 172.21.15.71:0/1986871458 learned_addr learned my addr 172.21.15.71:0/1986871458 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:07.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.367+0000 7fee08a00640 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d7064f0 0x558c6d7007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee03400640 1 -- 172.21.15.71:0/1986871458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fb310 msgr2=0x558c6d70ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee03400640 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fb310 0x558c6d70ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee03400640 1 -- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d7064f0 msgr2=0x558c6d7007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee03400640 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d7064f0 0x558c6d7007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee03400640 1 -- 172.21.15.71:0/1986871458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c6d700fe0 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee03400640 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d700250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fedf00a4ae0 tx=0x7fedf0097eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cdcda21fb8a1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7feded600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7feded600640 1 -- 172.21.15.71:0/1986871458 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf00a0060 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7feded600640 1 -- 172.21.15.71:0/1986871458 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9438+0+0 (secure 0 0 0) 0x7fedf0018b30 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf0097ad0 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7feded600640 1 -- 172.21.15.71:0/1986871458 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fedf008b050 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.368+0000 7fee0d5bd740 10 client.16791 fetch_fsmap learned FSMap version 50 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x558c6d780e10 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7feded600640 1 -- 172.21.15.71:0/1986871458 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedf0097ad0 con 0x558c6d6fbc30 2024-09-05T12:40:07.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7feded600640 1 -- 172.21.15.71:0/1986871458 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7fedf00a11f0 con 0x558c6d6fbc30 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 10 client.16791 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 10 client.16791 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 20 client.16791 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d7812c0 0x558c6d7836e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 4 client.16791 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e6b00742", "release": true, "await": true}] 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e6b00742", "release": true, "await": true}) -- 0x558c6d6690d0 con 0x558c6d7812c0 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.369+0000 7fee03e00640 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d7812c0 0x558c6d7836e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.370+0000 7fee03e00640 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d7812c0 0x558c6d7836e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d53bbbb2b45445f5 server_cookie=3e172b7ccdf44c1f in_seq=0 out_seq=0 2024-09-05T12:40:07.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.370+0000 7feded600640 10 client.16791 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:07.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7feded600640 1 -- 172.21.15.71:0/1986871458 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558c6d6690d0 con 0x558c6d7812c0 2024-09-05T12:40:07.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7feded600640 10 client.16791 handle_command_reply: tid=0 2024-09-05T12:40:07.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 client.16791 shutdown 2024-09-05T12:40:07.377 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 msgr2=0x558c6d700250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d700250 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fedf00a4ae0 tx=0x7fedf0097eb0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 shutdown_connections 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558c6d7812c0 0x558c6d7836e0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c6d7064f0 0x558c6d7007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558c6d6fbc30 0x558c6d700250 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 --2- 172.21.15.71:0/1986871458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c6d6fb310 0x558c6d70ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 >> 172.21.15.71:0/1986871458 conn(0x558c6d6f78a0 msgr2=0x558c6d6f6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.375+0000 7feded600640 0 client.16791 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.376+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 shutdown_connections 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.376+0000 7fee0d5bd740 1 -- 172.21.15.71:0/1986871458 wait complete. 2024-09-05T12:40:07.378 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.376+0000 7fee0d5bd740 20 client.16791 trim_cache size 0 max 16384 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 50, 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 40, 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "e6b00742": { 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 40, 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:40:07.379 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:07.380 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:07.381 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:40:07.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fede80a2c00 msgr2=0x7fede80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fede80a2c00 0x7fede80a50c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009cc0 tx=0x7fedf4005f90 comp rx=0 tx=0).stop 2024-09-05T12:40:07.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 msgr2=0x7fee0419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:07.381 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 0x7fee0419abd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fedf000e9f0 tx=0x7fedf000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 shutdown_connections 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee0410e450 0x7fee0419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fede80a2c00 0x7fede80a50c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee04074e30 0x7fee0419abd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 --2- 172.21.15.71:0/269759447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee04074510 0x7fee0419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 >> 172.21.15.71:0/269759447 conn(0x7fee040fd8f0 msgr2=0x7fee040fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.377+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 shutdown_connections 2024-09-05T12:40:07.382 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:07.378+0000 7fee0aa00640 1 -- 172.21.15.71:0/269759447 wait complete. 2024-09-05T12:40:07.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:07 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:40:07.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:07 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:40:07.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:07 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:40:07.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:07 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:40:07.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:07 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:40:07.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:07 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:40:07.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:07 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:40:07.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:07 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:40:07.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:07 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:40:08.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 405448 3.06 MB/sec execute 544 sec latency 31503.667 ms 2024-09-05T12:40:08.326 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e6b00742', total seconds elapsed: 34.1 2024-09-05T12:40:08.326 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-05T12:40:08.327 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:40:08.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:08 smithi071 ceph-mon[29845]: pgmap v387: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 KiB/s wr, 35 op/s 2024-09-05T12:40:08.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:08 smithi071 ceph-mon[29845]: osdmap e109: 12 total, 12 up, 12 in 2024-09-05T12:40:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:08 smithi123 ceph-mon[34321]: pgmap v387: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 KiB/s wr, 35 op/s 2024-09-05T12:40:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:08 smithi123 ceph-mon[34321]: osdmap e109: 12 total, 12 up, 12 in 2024-09-05T12:40:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:40:08.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:08 smithi174 ceph-mon[34313]: pgmap v387: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 KiB/s wr, 35 op/s 2024-09-05T12:40:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:08 smithi174 ceph-mon[34313]: osdmap e109: 12 total, 12 up, 12 in 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 -- 172.21.15.71:0/3814076857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0878074ee0 msgr2=0x7f087810dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 --2- 172.21.15.71:0/3814076857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0878074ee0 0x7f087810dc10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0864009900 tx=0x7f086402a480 comp rx=0 tx=0).stop 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 -- 172.21.15.71:0/3814076857 shutdown_connections 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 --2- 172.21.15.71:0/3814076857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f087810e150 0x7f0878110560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 --2- 172.21.15.71:0/3814076857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0878074ee0 0x7f087810dc10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 --2- 172.21.15.71:0/3814076857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08780745c0 0x7f08780749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 -- 172.21.15.71:0/3814076857 >> 172.21.15.71:0/3814076857 conn(0x7f08780fd5d0 msgr2=0x7f08780ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 -- 172.21.15.71:0/3814076857 shutdown_connections 2024-09-05T12:40:08.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.910+0000 7f087ee00640 1 -- 172.21.15.71:0/3814076857 wait complete. 2024-09-05T12:40:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 Processor -- start 2024-09-05T12:40:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 -- start start 2024-09-05T12:40:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08780745c0 0x7f087819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 0x7f087819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f087810e150 0x7f087819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0878113ec0 con 0x7f0878074ee0 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0878113d40 con 0x7f08780745c0 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087ee00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0878114040 con 0x7f087810e150 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.911+0000 7f087d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 0x7f087819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087e400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f087810e150 0x7f087819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08780745c0 0x7f087819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 0x7f087819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52786/0 (socket says 172.21.15.71:52786) 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 -- 172.21.15.71:0/1232143559 learned_addr learned my addr 172.21.15.71:0/1232143559 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 -- 172.21.15.71:0/1232143559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f087810e150 msgr2=0x7f087819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f087810e150 0x7f087819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 -- 172.21.15.71:0/1232143559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08780745c0 msgr2=0x7f087819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08780745c0 0x7f087819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 -- 172.21.15.71:0/1232143559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087819f480 con 0x7f0878074ee0 2024-09-05T12:40:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f087d000640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 0x7f087819a9d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0864006010 tx=0x7f0864004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.912+0000 7f0876000640 1 -- 172.21.15.71:0/1232143559 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086403f030 con 0x7f0878074ee0 2024-09-05T12:40:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.913+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087819f710 con 0x7f0878074ee0 2024-09-05T12:40:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.913+0000 7f0876000640 1 -- 172.21.15.71:0/1232143559 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0864004440 con 0x7f0878074ee0 2024-09-05T12:40:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.913+0000 7f0876000640 1 -- 172.21.15.71:0/1232143559 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864049060 con 0x7f0878074ee0 2024-09-05T12:40:08.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.913+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08781a7000 con 0x7f0878074ee0 2024-09-05T12:40:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0876000640 1 -- 172.21.15.71:0/1232143559 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0864042030 con 0x7f0878074ee0 2024-09-05T12:40:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 Processor -- start 2024-09-05T12:40:08.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 -- start start 2024-09-05T12:40:08.919 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab12030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab14960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56226aa6d100 con 0x56226ab07cd0 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56226aa151b0 con 0x56226ab12570 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.916+0000 7f0883213740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56226aa690c0 con 0x56226ab073b0 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab14960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab07790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab14960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:52028/0 (socket says 172.21.15.71:52028) 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab07790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42564/0 (socket says 172.21.15.71:42564) 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087d000640 1 -- 172.21.15.71:0/2844812664 learned_addr learned my addr 172.21.15.71:0/2844812664 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087da00640 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab12030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 -- 172.21.15.71:0/2844812664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 msgr2=0x56226ab14960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.920 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab14960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 -- 172.21.15.71:0/2844812664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 msgr2=0x56226ab12030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab12030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 -- 172.21.15.71:0/2844812664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226a780210 con 0x56226ab073b0 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087d000640 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab14960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f0876000640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f08600a2cf0 0x7f08600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f087e400640 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab07790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0878076600 tx=0x7f0870011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e15629963d8203 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.917+0000 7f0876000640 1 -- 172.21.15.71:0/1232143559 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7f08640eff90 con 0x7f0878074ee0 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.918+0000 7f087da00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f08600a2cf0 0x7f08600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.918+0000 7f086aa00640 1 -- 172.21.15.71:0/2844812664 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870004d20 con 0x56226ab073b0 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.918+0000 7f086aa00640 1 -- 172.21.15.71:0/2844812664 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870004050 con 0x56226ab073b0 2024-09-05T12:40:08.921 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.918+0000 7f086aa00640 1 -- 172.21.15.71:0/2844812664 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870005280 con 0x56226ab073b0 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 -- 172.21.15.71:0/2844812664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 msgr2=0x56226ab07790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab07790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0878076600 tx=0x7f0870011810 comp rx=0 tx=0).stop 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 -- 172.21.15.71:0/2844812664 shutdown_connections 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab14960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab12030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 --2- 172.21.15.71:0/2844812664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab07790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.925 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 -- 172.21.15.71:0/2844812664 >> 172.21.15.71:0/2844812664 conn(0x56226ab03920 msgr2=0x56226ab03d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 -- 172.21.15.71:0/2844812664 shutdown_connections 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f087da00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f08600a2cf0 0x7f08600a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f086c009da0 tx=0x7f086c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 -- 172.21.15.71:0/2844812664 wait complete. 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.919+0000 7f0883213740 1 -- start start 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 Processor -- start 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 0x56226ab0f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab07cd0 0x56226ab0fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab10140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56226ab14ff0 con 0x56226ab12570 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56226a9f6090 con 0x56226ab073b0 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56226ab15170 con 0x56226ab07cd0 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f0883213740 10 client.0 _ll_register_callbacks cb 0x56226aa4af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:08.926 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.920+0000 7f087da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 0x56226ab0f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab07cd0 0x56226ab0fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 0x56226ab0f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:52042/0 (socket says 172.21.15.71:52042) 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab07cd0 0x56226ab0fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42574/0 (socket says 172.21.15.71:42574) 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 -- 172.21.15.71:0/201850529 learned_addr learned my addr 172.21.15.71:0/201850529 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087e400640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab10140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 -- 172.21.15.71:0/201850529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab07cd0 msgr2=0x56226ab0fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab07cd0 0x56226ab0fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 -- 172.21.15.71:0/201850529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 msgr2=0x56226ab10140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab10140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 -- 172.21.15.71:0/201850529 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226a84c620 con 0x56226ab073b0 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087e400640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab10140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f087da00640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 0x56226ab0f6c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f086c0045f0 tx=0x7f086c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930e2138bd1827dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.921+0000 7f085f400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:40:08.927 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f085f400640 1 -- 172.21.15.71:0/201850529 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c01f640 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f085f400640 1 -- 172.21.15.71:0/201850529 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7f086c01f7e0 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0870004050 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f085f400640 1 -- 172.21.15.71:0/201850529 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f086c01f9e0 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 10 client.26828 fetch_fsmap learned FSMap version 50 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56226ab20ea0 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f085f400640 1 -- 172.21.15.71:0/201850529 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870004050 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f085f400640 1 -- 172.21.15.71:0/201850529 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f086c027070 con 0x56226ab073b0 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 10 client.26828 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 20 client.26828 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.922+0000 7f0883213740 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab22360 0x56226ab247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.923+0000 7f0883213740 4 client.26828 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:08.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.923+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56226aa7a830 con 0x56226ab22360 2024-09-05T12:40:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.923+0000 7f087d000640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab22360 0x56226ab247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.923+0000 7f087d000640 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab22360 0x56226ab247b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859c14f57dd980d server_cookie=c40d94f39f29027a in_seq=0 out_seq=0 2024-09-05T12:40:08.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.923+0000 7f085f400640 10 client.26828 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f085f400640 1 -- 172.21.15.71:0/201850529 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56226aa7a830 con 0x56226ab22360 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f085f400640 10 client.26828 handle_command_reply: tid=0 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 client.26828 shutdown 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 msgr2=0x56226ab0f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 0x56226ab0f6c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f086c0045f0 tx=0x7f086c004730 comp rx=0 tx=0).stop 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 shutdown_connections 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab22360 0x56226ab247b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab10140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab07cd0 0x56226ab0fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 --2- 172.21.15.71:0/201850529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab073b0 0x56226ab0f6c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 >> 172.21.15.71:0/201850529 conn(0x56226ab03920 msgr2=0x56226ab09680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.924+0000 7f085f400640 0 client.26828 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.925+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 shutdown_connections 2024-09-05T12:40:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.925+0000 7f0883213740 1 -- 172.21.15.71:0/201850529 wait complete. 2024-09-05T12:40:08.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.925+0000 7f0883213740 20 client.26828 trim_cache size 0 max 16384 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 Processor -- start 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 -- start start 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab2a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 0x56226ab2a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab0c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f08781b1190 con 0x56226ab12570 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08781b2d60 con 0x56226ab07cd0 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.941+0000 7f0883213740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08781b18f0 con 0x56226ab073b0 2024-09-05T12:40:08.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 0x56226ab2a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 0x56226ab2a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:52046/0 (socket says 172.21.15.71:52046) 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 -- 172.21.15.71:0/2624775395 learned_addr learned my addr 172.21.15.71:0/2624775395 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087da00640 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab2a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087e400640 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab0c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 -- 172.21.15.71:0/2624775395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 msgr2=0x56226ab2a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab2a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 -- 172.21.15.71:0/2624775395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 msgr2=0x56226ab0c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab0c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 -- 172.21.15.71:0/2624775395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56226ab0dc10 con 0x56226ab07cd0 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087e400640 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab0c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f087d000640 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 0x56226ab2a650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08640bbed0 tx=0x7f08640f3a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306dcddf3017546f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:08.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f086a000640 1 -- 172.21.15.71:0/2624775395 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0864035040 con 0x56226ab07cd0 2024-09-05T12:40:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f086a000640 1 -- 172.21.15.71:0/2624775395 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08640bc1d0 con 0x56226ab07cd0 2024-09-05T12:40:08.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.942+0000 7f086a000640 1 -- 172.21.15.71:0/2624775395 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08640c01b0 con 0x56226ab07cd0 2024-09-05T12:40:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 -- 172.21.15.71:0/2624775395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 msgr2=0x56226ab2a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 0x56226ab2a650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f08640bbed0 tx=0x7f08640f3a40 comp rx=0 tx=0).stop 2024-09-05T12:40:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 -- 172.21.15.71:0/2624775395 shutdown_connections 2024-09-05T12:40:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab12570 0x56226ab0c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab07cd0 0x56226ab2a650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 --2- 172.21.15.71:0/2624775395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab2a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 -- 172.21.15.71:0/2624775395 >> 172.21.15.71:0/2624775395 conn(0x56226ab03920 msgr2=0x56226ab08550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 -- 172.21.15.71:0/2624775395 shutdown_connections 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.943+0000 7f0883213740 1 -- 172.21.15.71:0/2624775395 wait complete. 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 -- start start 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 Processor -- start 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab0c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab0cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab0d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f08781b2690 con 0x56226ab07cd0 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08781b1fc0 con 0x56226ab12570 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08781af190 con 0x56226ab073b0 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.944+0000 7f0883213740 10 client.0 _ll_register_callbacks cb 0x7f086c009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:08.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab0cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab0c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab0d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab0cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52824/0 (socket says 172.21.15.71:52824) 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab0c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42596/0 (socket says 172.21.15.71:42596) 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087d000640 1 -- 172.21.15.71:0/3210356368 learned_addr learned my addr 172.21.15.71:0/3210356368 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 -- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 msgr2=0x56226ab0d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab0d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 -- 172.21.15.71:0/3210356368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 msgr2=0x56226ab0cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab0cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 -- 172.21.15.71:0/3210356368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56226ab8c0e0 con 0x56226ab073b0 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087d000640 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f087da00640 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab0c6a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f086c02aff0 tx=0x7f086c029670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581f2fc76e784728 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:08.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.945+0000 7f085f400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f085f400640 1 -- 172.21.15.71:0/3210356368 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f086c03c070 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08640bc1d0 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f085f400640 1 -- 172.21.15.71:0/3210356368 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7f086c029860 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f085f400640 1 -- 172.21.15.71:0/3210356368 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f086c003d90 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 10 client.26866 fetch_fsmap learned FSMap version 50 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x56226ab8d3d0 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f085f400640 1 -- 172.21.15.71:0/3210356368 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08640bc1d0 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f085f400640 1 -- 172.21.15.71:0/3210356368 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f086c03f020 con 0x56226ab073b0 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 10 client.26866 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 10 client.26866 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 20 client.26866 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab8db90 0x56226ab8ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:08.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 4 client.26866 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:40:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.946+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56226aa74b80 con 0x56226ab8db90 2024-09-05T12:40:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.947+0000 7f087d000640 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab8db90 0x56226ab8ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.947+0000 7f087d000640 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab8db90 0x56226ab8ffb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f433f1c52d9a2a server_cookie=51614086ca231619 in_seq=0 out_seq=0 2024-09-05T12:40:08.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.947+0000 7f085f400640 10 client.26866 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f085f400640 1 -- 172.21.15.71:0/3210356368 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56226aa74b80 con 0x56226ab8db90 2024-09-05T12:40:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f085f400640 10 client.26866 handle_command_reply: tid=0 2024-09-05T12:40:08.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 client.26866 shutdown 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 msgr2=0x56226ab0c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab0c6a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f086c02aff0 tx=0x7f086c029670 comp rx=0 tx=0).stop 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 shutdown_connections 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56226ab8db90 0x56226ab8ffb0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56226ab12570 0x56226ab0d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56226ab07cd0 0x56226ab0cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 --2- 172.21.15.71:0/3210356368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56226ab073b0 0x56226ab0c6a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f085f400640 0 client.26866 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 >> 172.21.15.71:0/3210356368 conn(0x56226ab03920 msgr2=0x56226ab1d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 shutdown_connections 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 1 -- 172.21.15.71:0/3210356368 wait complete. 2024-09-05T12:40:08.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f0883213740 20 client.26866 trim_cache size 0 max 16384 2024-09-05T12:40:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:40:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:40:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:40:08.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f08600a2cf0 msgr2=0x7f08600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f087ee00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f08600a2cf0 0x7f08600a51b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f086c009da0 tx=0x7f086c005f90 comp rx=0 tx=0).stop 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 msgr2=0x7f087819a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.981+0000 7f087ee00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 0x7f087819a9d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f0864006010 tx=0x7f0864004040 comp rx=0 tx=0).stop 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 shutdown_connections 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f087810e150 0x7f087819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f08600a2cf0 0x7f08600a51b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0878074ee0 0x7f087819a9d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 --2- 172.21.15.71:0/1232143559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08780745c0 0x7f087819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 >> 172.21.15.71:0/1232143559 conn(0x7f08780fd5d0 msgr2=0x7f08780fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 shutdown_connections 2024-09-05T12:40:08.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:08.982+0000 7f087ee00640 1 -- 172.21.15.71:0/1232143559 wait complete. 2024-09-05T12:40:09.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406092 3.06 MB/sec execute 545 sec latency 346.443 ms 2024-09-05T12:40:09.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 -- 172.21.15.71:0/3757166125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f190010c180 msgr2=0x7f190010e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:09.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 --2- 172.21.15.71:0/3757166125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f190010c180 0x7f190010e620 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f18f8009ca0 tx=0x7f18f802a950 comp rx=0 tx=0).stop 2024-09-05T12:40:09.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 -- 172.21.15.71:0/3757166125 shutdown_connections 2024-09-05T12:40:09.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 --2- 172.21.15.71:0/3757166125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f190010c180 0x7f190010e620 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 --2- 172.21.15.71:0/3757166125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1900103670 0x7f190010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 --2- 172.21.15.71:0/3757166125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f1900103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 -- 172.21.15.71:0/3757166125 >> 172.21.15.71:0/3757166125 conn(0x7f19000fc7d0 msgr2=0x7f19000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 -- 172.21.15.71:0/3757166125 shutdown_connections 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.439+0000 7f1907e00640 1 -- 172.21.15.71:0/3757166125 wait complete. 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 Processor -- start 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 -- start start 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f190019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1900103670 0x7f190019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190010c180 0x7f190019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1900111f40 con 0x7f1900103670 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1900111dc0 con 0x7f1900102d50 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.440+0000 7f1907e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19001120c0 con 0x7f190010c180 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f190019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:09.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1904a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1900103670 0x7f190019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f190019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:52080/0 (socket says 172.21.15.71:52080) 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 -- 172.21.15.71:0/4004705394 learned_addr learned my addr 172.21.15.71:0/4004705394 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190010c180 0x7f190019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 -- 172.21.15.71:0/4004705394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190010c180 msgr2=0x7f190019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190010c180 0x7f190019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 -- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1900103670 msgr2=0x7f190019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1900103670 0x7f190019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 -- 172.21.15.71:0/4004705394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190019f5c0 con 0x7f1900102d50 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.441+0000 7f1905400640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f190019a610 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.442+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400cd80 con 0x7f1900102d50 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.442+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190019f8b0 con 0x7f1900102d50 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.442+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f4010910 con 0x7f1900102d50 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.442+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f40188f0 con 0x7f1900102d50 2024-09-05T12:40:09.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.442+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19001a7190 con 0x7f1900102d50 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.443+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1900103130 con 0x7f1900102d50 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.445+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f18f4010ac0 con 0x7f1900102d50 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.446+0000 7f18fe000640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f18e80a2d70 0x7f18e80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.446+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7f18f40cfbe0 con 0x7f1900102d50 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.446+0000 7f1904a00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f18e80a2d70 0x7f18e80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.447+0000 7f1904a00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f18e80a2d70 0x7f18e80a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0095f0 tx=0x7f18ec005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:09.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.450+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18f409bd60 con 0x7f1900102d50 2024-09-05T12:40:09.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.662+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f190019b970 con 0x7f1900102d50 2024-09-05T12:40:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.663+0000 7f18fe000640 1 -- 172.21.15.71:0/4004705394 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) ==== 94+0+11977 (secure 0 0 0) 0x7f18f40a0060 con 0x7f1900102d50 2024-09-05T12:40:09.665 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:40:09.665 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":50,"btime":"2024-09-05T12:39:32:235089+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:39:31.831476+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:40:09.665 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 50 2024-09-05T12:40:09.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f18e80a2d70 msgr2=0x7f18e80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:09.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f18e80a2d70 0x7f18e80a5230 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0095f0 tx=0x7f18ec005f90 comp rx=0 tx=0).stop 2024-09-05T12:40:09.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 msgr2=0x7f190019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f190019a610 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 shutdown_connections 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190010c180 0x7f190019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f18e80a2d70 0x7f18e80a5230 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1900103670 0x7f190019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 --2- 172.21.15.71:0/4004705394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1900102d50 0x7f190019a610 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 >> 172.21.15.71:0/4004705394 conn(0x7f19000fc7d0 msgr2=0x7f1900101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 shutdown_connections 2024-09-05T12:40:09.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:09.665+0000 7f1907e00640 1 -- 172.21.15.71:0/4004705394 wait complete. 2024-09-05T12:40:10.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406092 3.06 MB/sec execute 546 sec latency 1346.583 ms 2024-09-05T12:40:10.618 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:10 smithi071 ceph-mon[29845]: pgmap v389: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-09-05T12:40:10.618 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:10 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/4004705394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:10.656 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-05T12:40:10.657 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-05T12:40:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:10 smithi123 ceph-mon[34321]: pgmap v389: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-09-05T12:40:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:10 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/4004705394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:10 smithi174 ceph-mon[34313]: pgmap v389: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 64 KiB/s wr, 33 op/s 2024-09-05T12:40:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:10 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/4004705394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:11.029 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:40:11.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406092 3.05 MB/sec execute 547 sec latency 2346.640 ms 2024-09-05T12:40:11.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.632+0000 7fe9d2e00640 1 -- 172.21.15.71:0/121818042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 msgr2=0x7fe9cc114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.632+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/121818042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc114c60 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009ca0 tx=0x7fe9c402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 -- 172.21.15.71:0/121818042 shutdown_connections 2024-09-05T12:40:11.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/121818042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc114c60 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/121818042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9cc077040 0x7fe9cc112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/121818042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc076720 0x7fe9cc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 -- 172.21.15.71:0/121818042 >> 172.21.15.71:0/121818042 conn(0x7fe9cc0ff8d0 msgr2=0x7fe9cc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 -- 172.21.15.71:0/121818042 shutdown_connections 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.633+0000 7fe9d2e00640 1 -- 172.21.15.71:0/121818042 wait complete. 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.634+0000 7fe9d2e00640 1 Processor -- start 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.634+0000 7fe9d2e00640 1 -- start start 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.634+0000 7fe9d2e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9cc076720 0x7fe9cc1a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc077040 0x7fe9cc1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1183d0 con 0x7fe9cc112870 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc118250 con 0x7fe9cc077040 2024-09-05T12:40:11.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc118550 con 0x7fe9cc076720 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc077040 0x7fe9cc1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc19e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52864/0 (socket says 172.21.15.71:52864) 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 -- 172.21.15.71:0/255492075 learned_addr learned my addr 172.21.15.71:0/255492075 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9cc076720 0x7fe9cc1a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 -- 172.21.15.71:0/255492075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9cc076720 msgr2=0x7fe9cc1a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9cc076720 0x7fe9cc1a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 -- 172.21.15.71:0/255492075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc077040 msgr2=0x7fe9cc1a4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc077040 0x7fe9cc1a4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d2400640 1 -- 172.21.15.71:0/255492075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9cc19ef30 con 0x7fe9cc112870 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.635+0000 7fe9d1000640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc077040 0x7fe9cc1a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.636+0000 7fe9d2400640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc19e6c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe9c402ade0 tx=0x7fe9c40319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:11.676 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.636+0000 7fe9ca000640 1 -- 172.21.15.71:0/255492075 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4003b70 con 0x7fe9cc112870 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.636+0000 7fe9ca000640 1 -- 172.21.15.71:0/255492075 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4003d10 con 0x7fe9cc112870 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.636+0000 7fe9ca000640 1 -- 172.21.15.71:0/255492075 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c402fc40 con 0x7fe9cc112870 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.636+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9cc19f1c0 con 0x7fe9cc112870 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.636+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9cc1ab3a0 con 0x7fe9cc112870 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9ca000640 1 -- 172.21.15.71:0/255492075 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe9c4004050 con 0x7fe9cc112870 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 Processor -- start 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 -- start start 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 0x5638be3e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be3f0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3f0740 0x5638be3f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5638be34b530 con 0x5638be3e5560 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5638be2f2860 con 0x5638be3e5e80 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.639+0000 7fe9d7071740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638be3474f0 con 0x5638be3f0740 2024-09-05T12:40:11.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 0x5638be3e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 0x5638be3e5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52880/0 (socket says 172.21.15.71:52880) 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 -- 172.21.15.71:0/1190571908 learned_addr learned my addr 172.21.15.71:0/1190571908 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be3f0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3f0740 0x5638be3f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 -- 172.21.15.71:0/1190571908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3f0740 msgr2=0x5638be3f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3f0740 0x5638be3f2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 -- 172.21.15.71:0/1190571908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 msgr2=0x5638be3f0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be3f0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 -- 172.21.15.71:0/1190571908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638be0bcb90 con 0x5638be3e5560 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9ca000640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe9b40a2d70 0x7fe9b40a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.640+0000 7fe9d1000640 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 0x5638be3e5940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80138f0 tx=0x7fe9b8009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2e4fb38c2d9da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.641+0000 7fe9ca000640 1 -- 172.21.15.71:0/255492075 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7fe9c40f08c0 con 0x7fe9cc112870 2024-09-05T12:40:11.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.641+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe9b40a2d70 0x7fe9b40a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.641+0000 7fe9bea00640 1 -- 172.21.15.71:0/1190571908 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8004050 con 0x5638be3e5560 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.641+0000 7fe9bea00640 1 -- 172.21.15.71:0/1190571908 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b800fe20 con 0x5638be3e5560 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.641+0000 7fe9bea00640 1 -- 172.21.15.71:0/1190571908 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003be0 con 0x5638be3e5560 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 -- 172.21.15.71:0/1190571908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 msgr2=0x5638be3e5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 0x5638be3e5940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80138f0 tx=0x7fe9b8009be0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe9b40a2d70 0x7fe9b40a5230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009da0 tx=0x7fe9c0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 -- 172.21.15.71:0/1190571908 shutdown_connections 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3f0740 0x5638be3f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be3f0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 --2- 172.21.15.71:0/1190571908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5560 0x5638be3e5940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 -- 172.21.15.71:0/1190571908 >> 172.21.15.71:0/1190571908 conn(0x5638be3e1af0 msgr2=0x5638be3e1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 -- 172.21.15.71:0/1190571908 shutdown_connections 2024-09-05T12:40:11.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 -- 172.21.15.71:0/1190571908 wait complete. 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 -- start start 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.642+0000 7fe9d7071740 1 Processor -- start 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be3ed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5e80 0x5638be3edb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3f0740 0x5638be3ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5638be3f3310 con 0x5638be3e5e80 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5638be305e60 con 0x5638be3f0740 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638be3f3490 con 0x5638be3e5560 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d7071740 10 client.0 _ll_register_callbacks cb 0x5638be30acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be3ed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d1a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be3ed5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42648/0 (socket says 172.21.15.71:42648) 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3f0740 0x5638be3ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d1a00640 1 -- 172.21.15.71:0/2951043534 learned_addr learned my addr 172.21.15.71:0/2951043534 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.643+0000 7fe9d1000640 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5e80 0x5638be3edb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d2400640 1 -- 172.21.15.71:0/2951043534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 msgr2=0x5638be3ed5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d2400640 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be3ed5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d2400640 1 -- 172.21.15.71:0/2951043534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5e80 msgr2=0x5638be3edb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d2400640 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5e80 0x5638be3edb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d2400640 1 -- 172.21.15.71:0/2951043534 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638be311990 con 0x5638be3f0740 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d2400640 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3f0740 0x5638be3ee070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40edcd0 tx=0x7fe9c40bc870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e05873fe1d02f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9b3400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9b3400640 1 -- 172.21.15.71:0/2951043534 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40bd1a0 con 0x5638be3f0740 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9b800fe20 con 0x5638be3f0740 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9b3400640 1 -- 172.21.15.71:0/2951043534 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7fe9c40ede40 con 0x5638be3f0740 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9b3400640 1 -- 172.21.15.71:0/2951043534 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe9c40bcaa0 con 0x5638be3f0740 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d7071740 10 client.26852 fetch_fsmap learned FSMap version 50 2024-09-05T12:40:11.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.644+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5638be3f3310 con 0x5638be3f0740 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9b3400640 1 -- 172.21.15.71:0/2951043534 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9b800fe20 con 0x5638be3f0740 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9b3400640 1 -- 172.21.15.71:0/2951043534 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7fe9c40db2a0 con 0x5638be3f0740 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d7071740 10 client.26852 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d7071740 10 client.26852 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d7071740 20 client.26852 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d7071740 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be400370 0x5638be4027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d7071740 4 client.26852 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638be2ee310 con 0x5638be400370 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.645+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be400370 0x5638be4027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.646+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be400370 0x5638be4027b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=747b8a85e39a5c76 server_cookie=d2387770a9748024 in_seq=0 out_seq=0 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.646+0000 7fe9b3400640 10 client.26852 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.646+0000 7fe9b3400640 1 -- 172.21.15.71:0/2951043534 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5638be2ee310 con 0x5638be400370 2024-09-05T12:40:11.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.646+0000 7fe9b3400640 10 client.26852 handle_command_reply: tid=0 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.646+0000 7fe9d7071740 1 client.26852 shutdown 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3f0740 msgr2=0x5638be3ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3f0740 0x5638be3ee070 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40edcd0 tx=0x7fe9c40bc870 comp rx=0 tx=0).stop 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 shutdown_connections 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be400370 0x5638be4027b0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3f0740 0x5638be3ee070 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3e5e80 0x5638be3edb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9b3400640 0 client.26852 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 --2- 172.21.15.71:0/2951043534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be3ed5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 >> 172.21.15.71:0/2951043534 conn(0x5638be3e1af0 msgr2=0x5638be3e7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 shutdown_connections 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 1 -- 172.21.15.71:0/2951043534 wait complete. 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.647+0000 7fe9d7071740 20 client.26852 trim_cache size 0 max 16384 2024-09-05T12:40:11.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.664+0000 7fe9d7071740 1 Processor -- start 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.664+0000 7fe9d7071740 1 -- start start 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be4080c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be408600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3ea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d7071740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b55a0 con 0x5638be3f0740 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d7071740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b7170 con 0x5638be3e5e80 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d7071740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b5d00 con 0x5638be3e5560 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3ea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.684 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3ea850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52888/0 (socket says 172.21.15.71:52888) 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 -- 172.21.15.71:0/3208175565 learned_addr learned my addr 172.21.15.71:0/3208175565 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d1000640 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be408600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be4080c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 -- 172.21.15.71:0/3208175565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 msgr2=0x5638be4080c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be4080c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 -- 172.21.15.71:0/3208175565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 msgr2=0x5638be408600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be408600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d2400640 1 -- 172.21.15.71:0/3208175565 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638be3e1920 con 0x5638be3f0740 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.665+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be4080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d2400640 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3ea850 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40b8eb0 tx=0x7fe9c40bafb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2142bd3ac7b79e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9be000640 1 -- 172.21.15.71:0/3208175565 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40bc5e0 con 0x5638be3f0740 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9be000640 1 -- 172.21.15.71:0/3208175565 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4036040 con 0x5638be3f0740 2024-09-05T12:40:11.685 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9be000640 1 -- 172.21.15.71:0/3208175565 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40c5280 con 0x5638be3f0740 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 -- 172.21.15.71:0/3208175565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 msgr2=0x5638be3ea850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3ea850 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40b8eb0 tx=0x7fe9c40bafb0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 -- 172.21.15.71:0/3208175565 shutdown_connections 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3ea850 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5e80 0x5638be408600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 --2- 172.21.15.71:0/3208175565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5560 0x5638be4080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.666+0000 7fe9d7071740 1 -- 172.21.15.71:0/3208175565 >> 172.21.15.71:0/3208175565 conn(0x5638be3e1af0 msgr2=0x5638be3e7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 -- 172.21.15.71:0/3208175565 shutdown_connections 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 -- 172.21.15.71:0/3208175565 wait complete. 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 -- start start 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 Processor -- start 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5560 0x5638be3f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5e80 0x5638be3f5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.686 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b6aa0 con 0x5638be3f0740 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b63d0 con 0x5638be3e5560 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.667+0000 7fe9d7071740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9cc1b35a0 con 0x5638be3e5e80 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d7071740 10 client.0 _ll_register_callbacks cb 0x7fe9c4003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d1a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5560 0x5638be3f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3f5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52892/0 (socket says 172.21.15.71:52892) 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 -- 172.21.15.71:0/81251165 learned_addr learned my addr 172.21.15.71:0/81251165 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d1000640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5e80 0x5638be3f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 -- 172.21.15.71:0/81251165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5e80 msgr2=0x5638be3f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5e80 0x5638be3f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 -- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5560 msgr2=0x5638be3f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5560 0x5638be3f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 -- 172.21.15.71:0/81251165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638be46b0a0 con 0x5638be3f0740 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d1000640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5e80 0x5638be3f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:40:11.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9d2400640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3f5a90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40bc7a0 tx=0x7fe9c4031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3621652dbc6e841f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.668+0000 7fe9b3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9b3400640 1 -- 172.21.15.71:0/81251165 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40bd1a0 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9b3400640 1 -- 172.21.15.71:0/81251165 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9526+0+0 (secure 0 0 0) 0x7fe9c40eddb0 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9c4036040 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9b3400640 1 -- 172.21.15.71:0/81251165 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe9c40edfb0 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9d7071740 10 client.16833 fetch_fsmap learned FSMap version 50 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5638be46b0a0 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9b3400640 1 -- 172.21.15.71:0/81251165 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9c4036040 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.669+0000 7fe9b3400640 1 -- 172.21.15.71:0/81251165 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7fe9c40bbef0 con 0x5638be3f0740 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d7071740 10 client.16833 fetch_fsmap finished waiting for FSMap version 50 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d7071740 10 client.16833 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d7071740 20 client.16833 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d7071740 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be46c500 0x5638be46e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:11.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d7071740 4 client.16833 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-05T12:40:11.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5638be358c60 con 0x5638be46c500 2024-09-05T12:40:11.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be46c500 0x5638be46e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:11.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9d1a00640 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be46c500 0x5638be46e8f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5724ea0b1470a82a server_cookie=35f6b6cc58dc3744 in_seq=0 out_seq=0 2024-09-05T12:40:11.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.670+0000 7fe9b3400640 10 client.16833 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:11.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9b3400640 1 -- 172.21.15.71:0/81251165 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5638be358c60 con 0x5638be46c500 2024-09-05T12:40:11.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9b3400640 10 client.16833 handle_command_reply: tid=0 2024-09-05T12:40:11.728 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 client.16833 shutdown 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 msgr2=0x5638be3f5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3f5a90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40bc7a0 tx=0x7fe9c4031b10 comp rx=0 tx=0).stop 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 shutdown_connections 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5638be46c500 0x5638be46e8f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638be3f0740 0x5638be3f5a90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638be3e5e80 0x5638be3f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9b3400640 0 client.16833 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 --2- 172.21.15.71:0/81251165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638be3e5560 0x5638be3f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 >> 172.21.15.71:0/81251165 conn(0x5638be3e1af0 msgr2=0x5638be3ed590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 shutdown_connections 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.726+0000 7fe9d7071740 1 -- 172.21.15.71:0/81251165 wait complete. 2024-09-05T12:40:11.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d7071740 20 client.16833 trim_cache size 0 max 16384 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 50, 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 42, 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "358ac758": { 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 42, 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:40:11.730 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:11.731 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:11.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:11.732 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:11.732 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:40:11.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe9b40a2d70 msgr2=0x7fe9b40a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe9b40a2d70 0x7fe9b40a5230 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009da0 tx=0x7fe9c0005f90 comp rx=0 tx=0).stop 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 msgr2=0x7fe9cc19e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc19e6c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe9c402ade0 tx=0x7fe9c40319f0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 shutdown_connections 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe9b40a2d70 0x7fe9b40a5230 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9cc112870 0x7fe9cc19e6c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9cc077040 0x7fe9cc1a4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 --2- 172.21.15.71:0/255492075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9cc076720 0x7fe9cc1a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.727+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 >> 172.21.15.71:0/255492075 conn(0x7fe9cc0ff8d0 msgr2=0x7fe9cc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.728+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 shutdown_connections 2024-09-05T12:40:11.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:11.728+0000 7fe9d2e00640 1 -- 172.21.15.71:0/255492075 wait complete. 2024-09-05T12:40:12.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.05 MB/sec execute 548 sec latency 2767.110 ms 2024-09-05T12:40:12.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:12 smithi071 ceph-mon[29845]: pgmap v390: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 148 KiB/s wr, 61 op/s 2024-09-05T12:40:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:12 smithi123 ceph-mon[34321]: pgmap v390: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 148 KiB/s wr, 61 op/s 2024-09-05T12:40:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:12 smithi174 ceph-mon[34313]: pgmap v390: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 148 KiB/s wr, 61 op/s 2024-09-05T12:40:12.795 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '358ac758', quiesce took 4.5 seconds. Will release after: 25.5 2024-09-05T12:40:13.294 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.04 MB/sec execute 549 sec latency 1575.577 ms 2024-09-05T12:40:14.295 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.04 MB/sec execute 550 sec latency 2575.688 ms 2024-09-05T12:40:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:14 smithi071 ceph-mon[29845]: pgmap v391: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-05T12:40:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:14 smithi123 ceph-mon[34321]: pgmap v391: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-05T12:40:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:14 smithi174 ceph-mon[34313]: pgmap v391: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-05T12:40:15.295 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.03 MB/sec execute 551 sec latency 3575.847 ms 2024-09-05T12:40:16.295 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.03 MB/sec execute 552 sec latency 4576.008 ms 2024-09-05T12:40:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:16 smithi174 ceph-mon[34313]: pgmap v392: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-05T12:40:16.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:16 smithi174 ceph-mon[34313]: osdmap e110: 12 total, 12 up, 12 in 2024-09-05T12:40:16.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:16 smithi071 ceph-mon[29845]: pgmap v392: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-05T12:40:16.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:16 smithi071 ceph-mon[29845]: osdmap e110: 12 total, 12 up, 12 in 2024-09-05T12:40:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:16 smithi123 ceph-mon[34321]: pgmap v392: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 224 KiB/s wr, 56 op/s 2024-09-05T12:40:16.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:16 smithi123 ceph-mon[34321]: osdmap e110: 12 total, 12 up, 12 in 2024-09-05T12:40:17.295 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.02 MB/sec execute 553 sec latency 5576.149 ms 2024-09-05T12:40:18.295 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.01 MB/sec execute 554 sec latency 6576.282 ms 2024-09-05T12:40:18.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:18 smithi071 ceph-mon[29845]: pgmap v394: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 290 KiB/s wr, 70 op/s 2024-09-05T12:40:18.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:18 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:40:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:18 smithi174 ceph-mon[34313]: pgmap v394: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 290 KiB/s wr, 70 op/s 2024-09-05T12:40:18.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:18 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:40:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:18 smithi123 ceph-mon[34321]: pgmap v394: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 290 KiB/s wr, 70 op/s 2024-09-05T12:40:18.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:18 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:40:19.295 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.01 MB/sec execute 555 sec latency 7576.389 ms 2024-09-05T12:40:20.296 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.00 MB/sec execute 556 sec latency 8576.549 ms 2024-09-05T12:40:20.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:20 smithi174 ceph-mon[34313]: pgmap v395: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 260 KiB/s wr, 62 op/s 2024-09-05T12:40:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:20 smithi071 ceph-mon[29845]: pgmap v395: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 260 KiB/s wr, 62 op/s 2024-09-05T12:40:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:20 smithi123 ceph-mon[34321]: pgmap v395: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 260 KiB/s wr, 62 op/s 2024-09-05T12:40:21.296 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 3.00 MB/sec execute 557 sec latency 9576.642 ms 2024-09-05T12:40:22.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 -- 172.21.15.71:0/2266400743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe81044a0 msgr2=0x7f5fe81109d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 --2- 172.21.15.71:0/2266400743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe81044a0 0x7f5fe81109d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009cd0 tx=0x7f5fe002a8f0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 -- 172.21.15.71:0/2266400743 shutdown_connections 2024-09-05T12:40:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 --2- 172.21.15.71:0/2266400743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe81044a0 0x7f5fe81109d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 --2- 172.21.15.71:0/2266400743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8103b00 0x7f5fe8103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 --2- 172.21.15.71:0/2266400743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fe8109ba0 0x7f5fe8109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 -- 172.21.15.71:0/2266400743 >> 172.21.15.71:0/2266400743 conn(0x7f5fe80ff7c0 msgr2=0x7f5fe8101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 -- 172.21.15.71:0/2266400743 shutdown_connections 2024-09-05T12:40:22.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.103+0000 7f5fe7200640 1 -- 172.21.15.71:0/2266400743 wait complete. 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.104+0000 7f5fe7200640 1 Processor -- start 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.104+0000 7f5fe7200640 1 -- start start 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 0x7f5fe819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fe81044a0 0x7f5fe819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe7200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8109ba0 0x7f5fe819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113f60 con 0x7f5fe8103b00 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe8113de0 con 0x7f5fe81044a0 2024-09-05T12:40:22.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe7200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81140e0 con 0x7f5fe8109ba0 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 0x7f5fe819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 0x7f5fe819a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48220/0 (socket says 172.21.15.71:48220) 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 -- 172.21.15.71:0/3986928189 learned_addr learned my addr 172.21.15.71:0/3986928189 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5400640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fe81044a0 0x7f5fe819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe6800640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8109ba0 0x7f5fe819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 -- 172.21.15.71:0/3986928189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8109ba0 msgr2=0x7f5fe819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8109ba0 0x7f5fe819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 -- 172.21.15.71:0/3986928189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fe81044a0 msgr2=0x7f5fe819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fe81044a0 0x7f5fe819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe5e00640 1 -- 172.21.15.71:0/3986928189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fe819f430 con 0x7f5fe8103b00 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.105+0000 7f5fe6800640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8109ba0 0x7f5fe819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:40:22.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.106+0000 7f5fe5e00640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 0x7f5fe819a440 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800e9f0 tx=0x7f5fd800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.106+0000 7f5fdea00640 1 -- 172.21.15.71:0/3986928189 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd800cd80 con 0x7f5fe8103b00 2024-09-05T12:40:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.106+0000 7f5fdea00640 1 -- 172.21.15.71:0/3986928189 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd8010910 con 0x7f5fe8103b00 2024-09-05T12:40:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.106+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fe819f720 con 0x7f5fe8103b00 2024-09-05T12:40:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.106+0000 7f5fdea00640 1 -- 172.21.15.71:0/3986928189 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8018990 con 0x7f5fe8103b00 2024-09-05T12:40:22.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.106+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fe81a6fb0 con 0x7f5fe8103b00 2024-09-05T12:40:22.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.109+0000 7f5fdea00640 1 -- 172.21.15.71:0/3986928189 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5fd8010ac0 con 0x7f5fe8103b00 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.109+0000 7f5fef9ed740 1 Processor -- start 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.109+0000 7f5fef9ed740 1 -- start start 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d21790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d2c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d2e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556fa1c87100 con 0x556fa1d21cd0 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556fa1c2f1b0 con 0x556fa1d213b0 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556fa1c830c0 con 0x556fa1d2c570 2024-09-05T12:40:22.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fe5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d2c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fe5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d2e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fe5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d2e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58106/0 (socket says 172.21.15.71:58106) 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fe5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d2c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48230/0 (socket says 172.21.15.71:48230) 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.110+0000 7f5fe5400640 1 -- 172.21.15.71:0/3582679744 learned_addr learned my addr 172.21.15.71:0/3582679744 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe6800640 1 -- 172.21.15.71:0/3582679744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 msgr2=0x556fa1d2e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe6800640 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d2e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe6800640 1 -- 172.21.15.71:0/3582679744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 msgr2=0x556fa1d2c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe6800640 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d2c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe6800640 1 -- 172.21.15.71:0/3582679744 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa199a210 con 0x556fa1d213b0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fdea00640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5fcc0a2d70 0x7f5fcc0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe6800640 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d21790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fe002ae10 tx=0x7f5fe0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0b232d42387df4e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fe5e00640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5fcc0a2d70 0x7f5fcc0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fdea00640 1 -- 172.21.15.71:0/3986928189 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f5fd80cfac0 con 0x7f5fe8103b00 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.111+0000 7f5fd3000640 1 -- 172.21.15.71:0/3582679744 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0030720 con 0x556fa1d213b0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fd3000640 1 -- 172.21.15.71:0/3582679744 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe0036040 con 0x556fa1d213b0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fd3000640 1 -- 172.21.15.71:0/3582679744 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0004330 con 0x556fa1d213b0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 -- 172.21.15.71:0/3582679744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 msgr2=0x556fa1d21790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fe5e00640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5fcc0a2d70 0x7f5fcc0a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8098050 tx=0x7f5fd80d3570 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d21790 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5fe002ae10 tx=0x7f5fe0009a20 comp rx=0 tx=0).stop 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 -- 172.21.15.71:0/3582679744 shutdown_connections 2024-09-05T12:40:22.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d2e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d2c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/3582679744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d21790 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.112+0000 7f5fef9ed740 1 -- 172.21.15.71:0/3582679744 >> 172.21.15.71:0/3582679744 conn(0x556fa1d1d920 msgr2=0x556fa1d1dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.113+0000 7f5fef9ed740 1 -- 172.21.15.71:0/3582679744 shutdown_connections 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.113+0000 7f5fef9ed740 1 -- 172.21.15.71:0/3582679744 wait complete. 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.113+0000 7f5fef9ed740 1 -- start start 2024-09-05T12:40:22.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.113+0000 7f5fef9ed740 1 Processor -- start 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d295f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d21cd0 0x556fa1d29b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d2c570 0x556fa1d2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556fa1d2eff0 con 0x556fa1d2c570 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556fa1c10090 con 0x556fa1d213b0 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556fa1d2f170 con 0x556fa1d21cd0 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fef9ed740 10 client.0 _ll_register_callbacks cb 0x556fa1c64f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d2c570 0x556fa1d2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d2c570 0x556fa1d2a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48232/0 (socket says 172.21.15.71:48232) 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe6800640 1 -- 172.21.15.71:0/2618803258 learned_addr learned my addr 172.21.15.71:0/2618803258 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:22.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d21cd0 0x556fa1d29b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe5400640 1 -- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 msgr2=0x556fa1d295f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d295f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe5400640 1 -- 172.21.15.71:0/2618803258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d2c570 msgr2=0x556fa1d2a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d2c570 0x556fa1d2a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.114+0000 7f5fe5400640 1 -- 172.21.15.71:0/2618803258 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa1a66620 con 0x556fa1d21cd0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d21cd0 0x556fa1d29b30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400b900 tx=0x7f5fd400bdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51deceb8ae5b522 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fc7400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fc7400640 1 -- 172.21.15.71:0/2618803258 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4011440 con 0x556fa1d21cd0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe0036040 con 0x556fa1d21cd0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fc7400640 1 -- 172.21.15.71:0/2618803258 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f5fd40115e0 con 0x556fa1d21cd0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fc7400640 1 -- 172.21.15.71:0/2618803258 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5fd40117e0 con 0x556fa1d21cd0 2024-09-05T12:40:22.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fef9ed740 10 client.26902 fetch_fsmap learned FSMap version 51 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556fa1d3add0 con 0x556fa1d21cd0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.115+0000 7f5fc7400640 1 -- 172.21.15.71:0/2618803258 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe0036040 con 0x556fa1d21cd0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fc7400640 1 -- 172.21.15.71:0/2618803258 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f5fd401b920 con 0x556fa1d21cd0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fef9ed740 10 client.26902 fetch_fsmap finished waiting for FSMap version 51 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fef9ed740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fef9ed740 20 client.26902 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1d3c2e0 0x556fa1d3e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fef9ed740 4 client.26902 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fa1c94830 con 0x556fa1d3c2e0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.116+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1d3c2e0 0x556fa1d3e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.117+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1d3c2e0 0x556fa1d3e6d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2f05e2d59ea105 server_cookie=a69ffedf09ae8191 in_seq=0 out_seq=0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.117+0000 7f5fc7400640 10 client.26902 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.117+0000 7f5fc7400640 1 -- 172.21.15.71:0/2618803258 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fa1c94830 con 0x556fa1d3c2e0 2024-09-05T12:40:22.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.117+0000 7f5fc7400640 10 client.26902 handle_command_reply: tid=0 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.117+0000 7f5fef9ed740 1 client.26902 shutdown 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d21cd0 msgr2=0x556fa1d29b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d21cd0 0x556fa1d29b30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400b900 tx=0x7f5fd400bdc0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 shutdown_connections 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1d3c2e0 0x556fa1d3e6d0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d2c570 0x556fa1d2a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d21cd0 0x556fa1d29b30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fc7400640 0 client.26902 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2618803258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d213b0 0x556fa1d295f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 >> 172.21.15.71:0/2618803258 conn(0x556fa1d1d920 msgr2=0x556fa1d23680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 shutdown_connections 2024-09-05T12:40:22.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2618803258 wait complete. 2024-09-05T12:40:22.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.118+0000 7f5fef9ed740 20 client.26902 trim_cache size 0 max 16384 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 Processor -- start 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 -- start start 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 0x556fa1d2b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d43fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d2c570 0x556fa1d444e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81b1150 con 0x556fa1d21cd0 2024-09-05T12:40:22.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81b2cc0 con 0x556fa1d2c570 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.133+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81b1850 con 0x556fa1d213b0 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d43fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 0x556fa1d2b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 0x556fa1d2b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58120/0 (socket says 172.21.15.71:58120) 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d43fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48236/0 (socket says 172.21.15.71:48236) 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 -- 172.21.15.71:0/2132152328 learned_addr learned my addr 172.21.15.71:0/2132152328 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d2c570 0x556fa1d444e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 -- 172.21.15.71:0/2132152328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d2c570 msgr2=0x556fa1d444e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d2c570 0x556fa1d444e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 -- 172.21.15.71:0/2132152328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 msgr2=0x556fa1d43fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.136 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d43fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.137 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.134+0000 7f5fe6800640 1 -- 172.21.15.71:0/2132152328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fa1d3ea40 con 0x556fa1d213b0 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.135+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 0x556fa1d2b450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0030f30 tx=0x7f5fe0030f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9860bccd8c95a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.135+0000 7f5fd2600640 1 -- 172.21.15.71:0/2132152328 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0003b70 con 0x556fa1d213b0 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.135+0000 7f5fd2600640 1 -- 172.21.15.71:0/2132152328 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe0036030 con 0x556fa1d213b0 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.135+0000 7f5fd2600640 1 -- 172.21.15.71:0/2132152328 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0031440 con 0x556fa1d213b0 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2132152328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 msgr2=0x556fa1d2b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 0x556fa1d2b450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0030f30 tx=0x7f5fe0030f60 comp rx=0 tx=0).stop 2024-09-05T12:40:22.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2132152328 shutdown_connections 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d2c570 0x556fa1d444e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d21cd0 0x556fa1d43fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2132152328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d213b0 0x556fa1d2b450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2132152328 >> 172.21.15.71:0/2132152328 conn(0x556fa1d1d920 msgr2=0x556fa1d2ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2132152328 shutdown_connections 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2132152328 wait complete. 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.136+0000 7f5fef9ed740 1 -- start start 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 Processor -- start 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 0x556fa1d303d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d21cd0 0x556fa1d30910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d30e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81b25f0 con 0x556fa1d213b0 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81b1f20 con 0x556fa1d21cd0 2024-09-05T12:40:22.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fe81af150 con 0x556fa1d2c570 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fef9ed740 10 client.0 _ll_register_callbacks cb 0x7f5fd4004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 0x556fa1d303d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fe5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d21cd0 0x556fa1d30910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fe6800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 0x556fa1d303d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48238/0 (socket says 172.21.15.71:48238) 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.137+0000 7f5fe6800640 1 -- 172.21.15.71:0/2601197440 learned_addr learned my addr 172.21.15.71:0/2601197440 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d30e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe5400640 1 -- 172.21.15.71:0/2601197440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 msgr2=0x556fa1d30e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d30e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe5400640 1 -- 172.21.15.71:0/2601197440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 msgr2=0x556fa1d303d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 0x556fa1d303d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe5400640 1 -- 172.21.15.71:0/2601197440 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fa1da6170 con 0x556fa1d21cd0 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 0x556fa1d303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:22.143 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fe5400640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d21cd0 0x556fa1d30910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5fd401bd60 tx=0x7f5fd40041d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81da7ce05711097d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fc7400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.138+0000 7f5fc7400640 1 -- 172.21.15.71:0/2601197440 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd402e030 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe0036030 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fc7400640 1 -- 172.21.15.71:0/2601197440 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f5fd4017040 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fc7400640 1 -- 172.21.15.71:0/2601197440 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5fd4011c10 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 10 client.26876 fetch_fsmap learned FSMap version 51 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x556fa1da7420 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fc7400640 1 -- 172.21.15.71:0/2601197440 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe0036030 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fc7400640 1 -- 172.21.15.71:0/2601197440 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 51) ==== 8378+0+0 (secure 0 0 0) 0x7f5fd401b7a0 con 0x556fa1d21cd0 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 10 client.26876 fetch_fsmap finished waiting for FSMap version 51 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:22.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 20 client.26876 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1da7b10 0x556fa1da9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 4 client.26876 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:40:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.139+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fa1c8eb80 con 0x556fa1da7b10 2024-09-05T12:40:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.140+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1da7b10 0x556fa1da9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.140+0000 7f5fe6800640 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1da7b10 0x556fa1da9f30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ba99f482b8c3d9d server_cookie=2aac6bcf9103078d in_seq=0 out_seq=0 2024-09-05T12:40:22.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.140+0000 7f5fc7400640 10 client.26876 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fc7400640 1 -- 172.21.15.71:0/2601197440 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fa1c8eb80 con 0x556fa1da7b10 2024-09-05T12:40:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fc7400640 10 client.26876 handle_command_reply: tid=0 2024-09-05T12:40:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 client.26876 shutdown 2024-09-05T12:40:22.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d21cd0 msgr2=0x556fa1d30910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d21cd0 0x556fa1d30910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5fd401bd60 tx=0x7f5fd40041d0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 shutdown_connections 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556fa1da7b10 0x556fa1da9f30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fa1d2c570 0x556fa1d30e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556fa1d21cd0 0x556fa1d30910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 --2- 172.21.15.71:0/2601197440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa1d213b0 0x556fa1d303d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fc7400640 0 client.26876 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 >> 172.21.15.71:0/2601197440 conn(0x556fa1d1d920 msgr2=0x556fa1d23630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 shutdown_connections 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 1 -- 172.21.15.71:0/2601197440 wait complete. 2024-09-05T12:40:22.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.166+0000 7f5fef9ed740 20 client.26876 trim_cache size 0 max 16384 2024-09-05T12:40:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:40:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:40:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:40:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:40:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5fcc0a2d70 msgr2=0x7f5fcc0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5fcc0a2d70 0x7f5fcc0a5230 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8098050 tx=0x7f5fd80d3570 comp rx=0 tx=0).stop 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 msgr2=0x7f5fe819a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 0x7f5fe819a440 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800e9f0 tx=0x7f5fd800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 shutdown_connections 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fe8109ba0 0x7f5fe819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fe81044a0 0x7f5fe819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5fcc0a2d70 0x7f5fcc0a5230 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 --2- 172.21.15.71:0/3986928189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5fe8103b00 0x7f5fe819a440 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 >> 172.21.15.71:0/3986928189 conn(0x7f5fe80ff7c0 msgr2=0x7f5fe80fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 shutdown_connections 2024-09-05T12:40:22.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:22.167+0000 7f5fe7200640 1 -- 172.21.15.71:0/3986928189 wait complete. 2024-09-05T12:40:22.296 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.99 MB/sec execute 558 sec latency 10577.031 ms 2024-09-05T12:40:22.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:22 smithi071 ceph-mon[29845]: pgmap v396: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 249 KiB/s wr, 34 op/s 2024-09-05T12:40:22.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:22 smithi174 ceph-mon[34313]: pgmap v396: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 249 KiB/s wr, 34 op/s 2024-09-05T12:40:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:22 smithi123 ceph-mon[34321]: pgmap v396: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 249 KiB/s wr, 34 op/s 2024-09-05T12:40:23.296 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.99 MB/sec execute 559 sec latency 11577.178 ms 2024-09-05T12:40:24.296 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.98 MB/sec execute 560 sec latency 12577.321 ms 2024-09-05T12:40:24.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:24 smithi174 ceph-mon[34313]: pgmap v397: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 23 op/s 2024-09-05T12:40:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:24 smithi071 ceph-mon[29845]: pgmap v397: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 23 op/s 2024-09-05T12:40:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:24 smithi123 ceph-mon[34321]: pgmap v397: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 23 op/s 2024-09-05T12:40:25.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.98 MB/sec execute 561 sec latency 13577.482 ms 2024-09-05T12:40:26.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.97 MB/sec execute 562 sec latency 14577.625 ms 2024-09-05T12:40:26.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:26 smithi174 ceph-mon[34313]: pgmap v398: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 23 op/s 2024-09-05T12:40:26.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:26 smithi071 ceph-mon[29845]: pgmap v398: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 23 op/s 2024-09-05T12:40:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:26 smithi123 ceph-mon[34321]: pgmap v398: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 163 KiB/s wr, 23 op/s 2024-09-05T12:40:27.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.97 MB/sec execute 563 sec latency 15577.738 ms 2024-09-05T12:40:28.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.96 MB/sec execute 564 sec latency 16577.842 ms 2024-09-05T12:40:28.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:28 smithi174 ceph-mon[34313]: pgmap v399: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-09-05T12:40:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:28 smithi071 ceph-mon[29845]: pgmap v399: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-09-05T12:40:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:28 smithi123 ceph-mon[34321]: pgmap v399: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 190 KiB/s wr, 34 op/s 2024-09-05T12:40:29.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.96 MB/sec execute 565 sec latency 17577.953 ms 2024-09-05T12:40:30.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.95 MB/sec execute 566 sec latency 18578.076 ms 2024-09-05T12:40:30.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:30 smithi071 ceph-mon[29845]: pgmap v400: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 147 KiB/s wr, 23 op/s 2024-09-05T12:40:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:30 smithi123 ceph-mon[34321]: pgmap v400: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 147 KiB/s wr, 23 op/s 2024-09-05T12:40:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:30 smithi174 ceph-mon[34313]: pgmap v400: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 147 KiB/s wr, 23 op/s 2024-09-05T12:40:31.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.95 MB/sec execute 567 sec latency 19578.215 ms 2024-09-05T12:40:32.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.94 MB/sec execute 568 sec latency 20578.313 ms 2024-09-05T12:40:32.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:32 smithi071 ceph-mon[29845]: pgmap v401: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 150 KiB/s wr, 27 op/s 2024-09-05T12:40:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:32 smithi123 ceph-mon[34321]: pgmap v401: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 150 KiB/s wr, 27 op/s 2024-09-05T12:40:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:32 smithi174 ceph-mon[34313]: pgmap v401: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 150 KiB/s wr, 27 op/s 2024-09-05T12:40:33.297 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.94 MB/sec execute 569 sec latency 21578.420 ms 2024-09-05T12:40:33.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:33 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:40:33.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:33 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:40:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:33 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:40:34.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.93 MB/sec execute 570 sec latency 22578.545 ms 2024-09-05T12:40:34.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:34 smithi071 ceph-mon[29845]: pgmap v402: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 70 KiB/s wr, 22 op/s 2024-09-05T12:40:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:34 smithi123 ceph-mon[34321]: pgmap v402: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 70 KiB/s wr, 22 op/s 2024-09-05T12:40:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:34 smithi174 ceph-mon[34313]: pgmap v402: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 70 KiB/s wr, 22 op/s 2024-09-05T12:40:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 -- 172.21.15.71:0/1480195560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff11410e170 msgr2=0x7ff114110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 --2- 172.21.15.71:0/1480195560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff11410e170 0x7ff114110580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff10c009be0 tx=0x7ff10c02a750 comp rx=0 tx=0).stop 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 -- 172.21.15.71:0/1480195560 shutdown_connections 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 --2- 172.21.15.71:0/1480195560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff11410e170 0x7ff114110580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 --2- 172.21.15.71:0/1480195560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff114074ee0 0x7ff11410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 --2- 172.21.15.71:0/1480195560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1140745c0 0x7ff1140749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 -- 172.21.15.71:0/1480195560 >> 172.21.15.71:0/1480195560 conn(0x7ff1140fd5f0 msgr2=0x7ff1140ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 -- 172.21.15.71:0/1480195560 shutdown_connections 2024-09-05T12:40:35.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.291+0000 7ff113200640 1 -- 172.21.15.71:0/1480195560 wait complete. 2024-09-05T12:40:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 Processor -- start 2024-09-05T12:40:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 -- start start 2024-09-05T12:40:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1140745c0 0x7ff11419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff114074ee0 0x7ff11419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 0x7ff11419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff114113ee0 con 0x7ff11410e170 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff114113d60 con 0x7ff1140745c0 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.292+0000 7ff113200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff114114060 con 0x7ff114074ee0 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 0x7ff11419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1140745c0 0x7ff11419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff109400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff114074ee0 0x7ff11419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 0x7ff11419edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40752/0 (socket says 172.21.15.71:40752) 2024-09-05T12:40:35.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff112800640 1 -- 172.21.15.71:0/2582539030 learned_addr learned my addr 172.21.15.71:0/2582539030 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 -- 172.21.15.71:0/2582539030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff114074ee0 msgr2=0x7ff11419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff114074ee0 0x7ff11419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 -- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 msgr2=0x7ff11419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 0x7ff11419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 -- 172.21.15.71:0/2582539030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11419f4f0 con 0x7ff1140745c0 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff112800640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 0x7ff11419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.293+0000 7ff111e00640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1140745c0 0x7ff11419a500 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e9f0 tx=0x7ff10400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.294+0000 7ff10b400640 1 -- 172.21.15.71:0/2582539030 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10400ccc0 con 0x7ff1140745c0 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.294+0000 7ff10b400640 1 -- 172.21.15.71:0/2582539030 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff104010910 con 0x7ff1140745c0 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.294+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11419f7e0 con 0x7ff1140745c0 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.294+0000 7ff10b400640 1 -- 172.21.15.71:0/2582539030 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104018960 con 0x7ff1140745c0 2024-09-05T12:40:35.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.294+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1141a7070 con 0x7ff1140745c0 2024-09-05T12:40:35.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.93 MB/sec execute 571 sec latency 23578.614 ms 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.297+0000 7ff10b400640 1 -- 172.21.15.71:0/2582539030 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff104010ac0 con 0x7ff1140745c0 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.297+0000 7ff11c07d740 1 Processor -- start 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.297+0000 7ff11c07d740 1 -- start start 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.297+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c08f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08fcd0 0x56168c09a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c09c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff11c07d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56168bff5100 con 0x56168c08f3b0 2024-09-05T12:40:35.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff11c07d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56168bf9d1b0 con 0x56168c08fcd0 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff11c07d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56168bff10c0 con 0x56168c09a570 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c08f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff111e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08fcd0 0x56168c09a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c08f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40764/0 (socket says 172.21.15.71:40764) 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 -- 172.21.15.71:0/921059688 learned_addr learned my addr 172.21.15.71:0/921059688 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff109400640 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c09c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 -- 172.21.15.71:0/921059688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 msgr2=0x56168c09c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c09c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 -- 172.21.15.71:0/921059688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08fcd0 msgr2=0x56168c09a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08fcd0 0x56168c09a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff112800640 1 -- 172.21.15.71:0/921059688 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168bd08210 con 0x56168c08f3b0 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.298+0000 7ff10b400640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff0f80a2d70 0x7ff0f80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.299+0000 7ff112800640 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c08f790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff10c030e50 tx=0x7ff10c030e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5cb8207108345d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.299+0000 7ff10b400640 1 -- 172.21.15.71:0/2582539030 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7ff1040cfb50 con 0x7ff1140745c0 2024-09-05T12:40:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.299+0000 7ff111e00640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff0f80a2d70 0x7ff0f80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.299+0000 7ff103400640 1 -- 172.21.15.71:0/921059688 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c041030 con 0x56168c08f3b0 2024-09-05T12:40:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.299+0000 7ff103400640 1 -- 172.21.15.71:0/921059688 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10c003970 con 0x56168c08f3b0 2024-09-05T12:40:35.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.299+0000 7ff103400640 1 -- 172.21.15.71:0/921059688 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c036040 con 0x56168c08f3b0 2024-09-05T12:40:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 -- 172.21.15.71:0/921059688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 msgr2=0x56168c08f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff111e00640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff0f80a2d70 0x7ff0f80a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff104098700 tx=0x7ff1040d3600 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c08f790 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ff10c030e50 tx=0x7ff10c030e80 comp rx=0 tx=0).stop 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 -- 172.21.15.71:0/921059688 shutdown_connections 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c09c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08fcd0 0x56168c09a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 --2- 172.21.15.71:0/921059688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c08f790 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 -- 172.21.15.71:0/921059688 >> 172.21.15.71:0/921059688 conn(0x56168c08b920 msgr2=0x56168c08bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 -- 172.21.15.71:0/921059688 shutdown_connections 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 -- 172.21.15.71:0/921059688 wait complete. 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.300+0000 7ff11c07d740 1 -- start start 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 Processor -- start 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c097690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c097bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c09a570 0x56168c098110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56168c09cff0 con 0x56168c09a570 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56168bf7e090 con 0x56168c08f3b0 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56168c09d170 con 0x56168c08fcd0 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff11c07d740 10 client.0 _ll_register_callbacks cb 0x56168bfd2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c09a570 0x56168c098110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff109400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c097bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c09a570 0x56168c098110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40774/0 (socket says 172.21.15.71:40774) 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff112800640 1 -- 172.21.15.71:0/49090463 learned_addr learned my addr 172.21.15.71:0/49090463 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.301+0000 7ff112800640 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c097690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff109400640 1 -- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 msgr2=0x56168c097690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff109400640 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c097690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff109400640 1 -- 172.21.15.71:0/49090463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c09a570 msgr2=0x56168c098110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff109400640 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c09a570 0x56168c098110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff109400640 1 -- 172.21.15.71:0/49090463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168bdd4620 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff109400640 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c097bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1140ffc50 tx=0x7ff0fc009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=405f0245b9a18137 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff0f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff0f7e00640 1 -- 172.21.15.71:0/49090463 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004020 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10c003970 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff0f7e00640 1 -- 172.21.15.71:0/49090463 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7ff0fc0041c0 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.302+0000 7ff0f7e00640 1 -- 172.21.15.71:0/49090463 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc024050 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 10 client.26920 fetch_fsmap learned FSMap version 52 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56168c0a8ed0 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff0f7e00640 1 -- 172.21.15.71:0/49090463 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10c003970 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff0f7e00640 1 -- 172.21.15.71:0/49090463 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7ff0fc003850 con 0x56168c08fcd0 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 10 client.26920 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:40:35.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 20 client.26920 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c0aa3e0 0x56168c0ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 4 client.26920 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.303+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56168c002830 con 0x56168c0aa3e0 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.304+0000 7ff112800640 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c0aa3e0 0x56168c0ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.304+0000 7ff112800640 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c0aa3e0 0x56168c0ac7d0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=208aa0bf5854d8e3 server_cookie=e6e186b42522cafe in_seq=0 out_seq=0 2024-09-05T12:40:35.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.304+0000 7ff0f7e00640 10 client.26920 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.305+0000 7ff0f7e00640 1 -- 172.21.15.71:0/49090463 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56168c002830 con 0x56168c0aa3e0 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.305+0000 7ff0f7e00640 10 client.26920 handle_command_reply: tid=0 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.305+0000 7ff11c07d740 1 client.26920 shutdown 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.305+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 msgr2=0x56168c097bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c097bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1140ffc50 tx=0x7ff0fc009f90 comp rx=0 tx=0).stop 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 shutdown_connections 2024-09-05T12:40:35.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c0aa3e0 0x56168c0ac7d0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c09a570 0x56168c098110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c097bd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff0f7e00640 0 client.26920 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 --2- 172.21.15.71:0/49090463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c097690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 >> 172.21.15.71:0/49090463 conn(0x56168c08b920 msgr2=0x56168c091680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 shutdown_connections 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 1 -- 172.21.15.71:0/49090463 wait complete. 2024-09-05T12:40:35.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.306+0000 7ff11c07d740 20 client.26920 trim_cache size 0 max 16384 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 Processor -- start 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 -- start start 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c097f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c091520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c09a570 0x56168c0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141b0f70 con 0x56168c08f3b0 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141b2ab0 con 0x56168c09a570 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff11c07d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141b1640 con 0x56168c08fcd0 2024-09-05T12:40:35.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c097f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff112800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c097f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40786/0 (socket says 172.21.15.71:40786) 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.321+0000 7ff112800640 1 -- 172.21.15.71:0/668662979 learned_addr learned my addr 172.21.15.71:0/668662979 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff109400640 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c091520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff112800640 1 -- 172.21.15.71:0/668662979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 msgr2=0x56168c091520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff112800640 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c091520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff112800640 1 -- 172.21.15.71:0/668662979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c09a570 msgr2=0x56168c0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff112800640 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c09a570 0x56168c0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff112800640 1 -- 172.21.15.71:0/668662979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56168c08f130 con 0x56168c08f3b0 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff112800640 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c097f40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0040b0 tx=0x7ff10c030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9fede446f8ccb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff102a00640 1 -- 172.21.15.71:0/668662979 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c036040 con 0x56168c08f3b0 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff102a00640 1 -- 172.21.15.71:0/668662979 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10c058020 con 0x56168c08f3b0 2024-09-05T12:40:35.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.322+0000 7ff102a00640 1 -- 172.21.15.71:0/668662979 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10c041040 con 0x56168c08f3b0 2024-09-05T12:40:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 -- 172.21.15.71:0/668662979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 msgr2=0x56168c097f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c097f40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff10c0040b0 tx=0x7ff10c030de0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 -- 172.21.15.71:0/668662979 shutdown_connections 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c09a570 0x56168c0b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c08fcd0 0x56168c091520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 --2- 172.21.15.71:0/668662979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08f3b0 0x56168c097f40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 -- 172.21.15.71:0/668662979 >> 172.21.15.71:0/668662979 conn(0x56168c08b920 msgr2=0x56168c090e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 -- 172.21.15.71:0/668662979 shutdown_connections 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 -- 172.21.15.71:0/668662979 wait complete. 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 -- start start 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.323+0000 7ff11c07d740 1 Processor -- start 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c0945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08fcd0 0x56168c094b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c095070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141b23e0 con 0x56168c08fcd0 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141b1d10 con 0x56168c08f3b0 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1141aef70 con 0x56168c09a570 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff11c07d740 10 client.0 _ll_register_callbacks cb 0x7ff0fc009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff112800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c0945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff109400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08fcd0 0x56168c094b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff112800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c0945f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42038/0 (socket says 172.21.15.71:42038) 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff112800640 1 -- 172.21.15.71:0/334249903 learned_addr learned my addr 172.21.15.71:0/334249903 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.324+0000 7ff112800640 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c095070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff109400640 1 -- 172.21.15.71:0/334249903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 msgr2=0x56168c095070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff109400640 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c095070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff109400640 1 -- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 msgr2=0x56168c0945f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:35.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff109400640 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c0945f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff109400640 1 -- 172.21.15.71:0/334249903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56168c114060 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff109400640 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08fcd0 0x56168c094b30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc003cf0 tx=0x7ff0fc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e15ada7aa5cf92 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff0f7e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff0f7e00640 1 -- 172.21.15.71:0/334249903 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc005df0 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff0f7e00640 1 -- 172.21.15.71:0/334249903 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7ff0fc003910 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff10c058020 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff0f7e00640 1 -- 172.21.15.71:0/334249903 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc035050 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff11c07d740 10 client.16869 fetch_fsmap learned FSMap version 52 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.325+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x56168c1152b0 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff0f7e00640 1 -- 172.21.15.71:0/334249903 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff10c058020 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff0f7e00640 1 -- 172.21.15.71:0/334249903 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7ff0fc030030 con 0x56168c08fcd0 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff11c07d740 10 client.16869 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff11c07d740 10 client.16869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:35.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff11c07d740 20 client.16869 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff11c07d740 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c1159c0 0x56168c117de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff11c07d740 4 client.16869 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:40:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56168bffcb80 con 0x56168c1159c0 2024-09-05T12:40:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.326+0000 7ff112800640 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c1159c0 0x56168c117de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.327+0000 7ff112800640 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c1159c0 0x56168c117de0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ebeed5dd6c9d13 server_cookie=8bec87ea7befab0 in_seq=0 out_seq=0 2024-09-05T12:40:35.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.327+0000 7ff0f7e00640 10 client.16869 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff0f7e00640 1 -- 172.21.15.71:0/334249903 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56168bffcb80 con 0x56168c1159c0 2024-09-05T12:40:35.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff0f7e00640 10 client.16869 handle_command_reply: tid=0 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 client.16869 shutdown 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08fcd0 msgr2=0x56168c094b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08fcd0 0x56168c094b30 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc003cf0 tx=0x7ff0fc0038a0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 shutdown_connections 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56168c1159c0 0x56168c117de0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56168c09a570 0x56168c095070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff0f7e00640 0 client.16869 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56168c08fcd0 0x56168c094b30 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 --2- 172.21.15.71:0/334249903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56168c08f3b0 0x56168c0945f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 >> 172.21.15.71:0/334249903 conn(0x56168c08b920 msgr2=0x56168c08a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 shutdown_connections 2024-09-05T12:40:35.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.347+0000 7ff11c07d740 1 -- 172.21.15.71:0/334249903 wait complete. 2024-09-05T12:40:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff11c07d740 20 client.16869 trim_cache size 0 max 16384 2024-09-05T12:40:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:40:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:40:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:40:35.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff0f80a2d70 msgr2=0x7ff0f80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff0f80a2d70 0x7ff0f80a5230 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff104098700 tx=0x7ff1040d3600 comp rx=0 tx=0).stop 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1140745c0 msgr2=0x7ff11419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1140745c0 0x7ff11419a500 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff10400e9f0 tx=0x7ff10400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 shutdown_connections 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff0f80a2d70 0x7ff0f80a5230 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff11410e170 0x7ff11419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff114074ee0 0x7ff11419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 --2- 172.21.15.71:0/2582539030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1140745c0 0x7ff11419a500 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 >> 172.21.15.71:0/2582539030 conn(0x7ff1140fd5f0 msgr2=0x7ff1140fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.348+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 shutdown_connections 2024-09-05T12:40:35.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:35.349+0000 7ff113200640 1 -- 172.21.15.71:0/2582539030 wait complete. 2024-09-05T12:40:36.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.92 MB/sec execute 572 sec latency 24578.724 ms 2024-09-05T12:40:36.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:36 smithi071 ceph-mon[29845]: pgmap v403: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-05T12:40:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:36 smithi123 ceph-mon[34321]: pgmap v403: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-05T12:40:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:36 smithi174 ceph-mon[34313]: pgmap v403: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 57 KiB/s wr, 19 op/s 2024-09-05T12:40:37.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.92 MB/sec execute 573 sec latency 25578.844 ms 2024-09-05T12:40:38.297 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:40:38.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.91 MB/sec execute 574 sec latency 26579.004 ms 2024-09-05T12:40:38.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:38 smithi071 ceph-mon[29845]: pgmap v404: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 78 KiB/s wr, 28 op/s 2024-09-05T12:40:38.678 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:40:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:38 smithi123 ceph-mon[34321]: pgmap v404: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 78 KiB/s wr, 28 op/s 2024-09-05T12:40:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:38 smithi174 ceph-mon[34313]: pgmap v404: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 78 KiB/s wr, 28 op/s 2024-09-05T12:40:39.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.90 MB/sec execute 575 sec latency 27579.080 ms 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 -- 172.21.15.71:0/1570243014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c104010 msgr2=0x7f017c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 --2- 172.21.15.71:0/1570243014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c104010 0x7f017c1043f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f016c009860 tx=0x7f016c02a340 comp rx=0 tx=0).stop 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 -- 172.21.15.71:0/1570243014 shutdown_connections 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 --2- 172.21.15.71:0/1570243014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f017c109140 0x7f017c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 --2- 172.21.15.71:0/1570243014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c1049c0 0x7f017c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 --2- 172.21.15.71:0/1570243014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c104010 0x7f017c1043f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 -- 172.21.15.71:0/1570243014 >> 172.21.15.71:0/1570243014 conn(0x7f017c0ffa50 msgr2=0x7f017c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 -- 172.21.15.71:0/1570243014 shutdown_connections 2024-09-05T12:40:39.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.325+0000 7f0180c00640 1 -- 172.21.15.71:0/1570243014 wait complete. 2024-09-05T12:40:39.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.326+0000 7f0180c00640 1 Processor -- start 2024-09-05T12:40:39.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.326+0000 7f0180c00640 1 -- start start 2024-09-05T12:40:39.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0180c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 0x7f017c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0180c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f017c1049c0 0x7f017c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0180c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c109140 0x7f017c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0180c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f017c114170 con 0x7f017c109140 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0180c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f017c113ff0 con 0x7f017c104010 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0180c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f017c1142f0 con 0x7f017c1049c0 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 0x7f017c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 0x7f017c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42058/0 (socket says 172.21.15.71:42058) 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 -- 172.21.15.71:0/2065700708 learned_addr learned my addr 172.21.15.71:0/2065700708 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f017a800640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c109140 0x7f017c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179400640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f017c1049c0 0x7f017c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 -- 172.21.15.71:0/2065700708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f017c1049c0 msgr2=0x7f017c19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f017c1049c0 0x7f017c19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 -- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c109140 msgr2=0x7f017c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.327+0000 7f0179e00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c109140 0x7f017c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0179e00640 1 -- 172.21.15.71:0/2065700708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c19f5e0 con 0x7f017c104010 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0179e00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 0x7f017c19a630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f016c035040 tx=0x7f016c033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c03f020 con 0x7f017c104010 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f017c19f870 con 0x7f017c104010 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f016c032770 con 0x7f017c104010 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016c037040 con 0x7f017c104010 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.328+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f017c1a71b0 con 0x7f017c104010 2024-09-05T12:40:39.331 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.329+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017c1043f0 con 0x7f017c104010 2024-09-05T12:40:39.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.331+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f016c042020 con 0x7f017c104010 2024-09-05T12:40:39.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.333+0000 7f0172a00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f01600a2cf0 0x7f01600a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:39.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.333+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f016c0ef910 con 0x7f017c104010 2024-09-05T12:40:39.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.333+0000 7f0179400640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f01600a2cf0 0x7f01600a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:39.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.333+0000 7f0179400640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f01600a2cf0 0x7f01600a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f017c0ffe70 tx=0x7f0168005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:39.335 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.335+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f016c049030 con 0x7f017c104010 2024-09-05T12:40:39.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.546+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f017c19b990 con 0x7f017c104010 2024-09-05T12:40:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.547+0000 7f0172a00640 1 -- 172.21.15.71:0/2065700708 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11972 (secure 0 0 0) 0x7f016c0bc250 con 0x7f017c104010 2024-09-05T12:40:39.548 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:40:39.552 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":52,"btime":"2024-09-05T12:40:32:263490+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:40:31.377721+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 52 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f01600a2cf0 msgr2=0x7f01600a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f01600a2cf0 0x7f01600a51b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f017c0ffe70 tx=0x7f0168005f90 comp rx=0 tx=0).stop 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 msgr2=0x7f017c19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 0x7f017c19a630 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f016c035040 tx=0x7f016c033d90 comp rx=0 tx=0).stop 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 shutdown_connections 2024-09-05T12:40:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f01600a2cf0 0x7f01600a51b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f017c109140 0x7f017c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f017c1049c0 0x7f017c19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 --2- 172.21.15.71:0/2065700708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f017c104010 0x7f017c19a630 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 >> 172.21.15.71:0/2065700708 conn(0x7f017c0ffa50 msgr2=0x7f017c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.549+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 shutdown_connections 2024-09-05T12:40:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:39.550+0000 7f0180c00640 1 -- 172.21.15.71:0/2065700708 wait complete. 2024-09-05T12:40:40.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.90 MB/sec execute 576 sec latency 28579.148 ms 2024-09-05T12:40:40.344 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 358ac758 --release --await' 2024-09-05T12:40:40.344 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 358ac758 --release --await 2024-09-05T12:40:40.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:40 smithi071 ceph-mon[29845]: pgmap v405: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-05T12:40:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2065700708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:40 smithi123 ceph-mon[34321]: pgmap v405: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-05T12:40:40.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2065700708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:40.717 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:40:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:40 smithi174 ceph-mon[34313]: pgmap v405: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-09-05T12:40:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2065700708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:40:41.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 406095 2.89 MB/sec execute 577 sec latency 29579.239 ms 2024-09-05T12:40:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 -- 172.21.15.71:0/3338066136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 msgr2=0x7fc4c81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.310 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 --2- 172.21.15.71:0/3338066136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c81043f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8009900 tx=0x7fc4b802a410 comp rx=0 tx=0).stop 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 -- 172.21.15.71:0/3338066136 shutdown_connections 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 --2- 172.21.15.71:0/3338066136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8109140 0x7fc4c8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 --2- 172.21.15.71:0/3338066136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c81049c0 0x7fc4c8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 --2- 172.21.15.71:0/3338066136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c81043f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.309+0000 7fc4cce00640 1 -- 172.21.15.71:0/3338066136 >> 172.21.15.71:0/3338066136 conn(0x7fc4c80ffa50 msgr2=0x7fc4c8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.310+0000 7fc4cce00640 1 -- 172.21.15.71:0/3338066136 shutdown_connections 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.310+0000 7fc4cce00640 1 -- 172.21.15.71:0/3338066136 wait complete. 2024-09-05T12:40:41.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.311+0000 7fc4cce00640 1 Processor -- start 2024-09-05T12:40:41.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.311+0000 7fc4cce00640 1 -- start start 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4cce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4cce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 0x7fc4c819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4cce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8109140 0x7fc4c819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4cce00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8114170 con 0x7fc4c8104010 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4cce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8113ff0 con 0x7fc4c8109140 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4cce00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81142f0 con 0x7fc4c81049c0 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 0x7fc4c819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 0x7fc4c819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:46462/0 (socket says 172.21.15.71:46462) 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4c5400640 1 -- 172.21.15.71:0/1103761057 learned_addr learned my addr 172.21.15.71:0/1103761057 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:41.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.312+0000 7fc4c6800640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8109140 0x7fc4c819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c6800640 1 -- 172.21.15.71:0/1103761057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 msgr2=0x7fc4c819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c6800640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 0x7fc4c819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c6800640 1 -- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 msgr2=0x7fc4c819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c6800640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c6800640 1 -- 172.21.15.71:0/1103761057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c819f5e0 con 0x7fc4c8109140 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c5400640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 0x7fc4c819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4c6800640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8109140 0x7fc4c819ef00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009fd0 tx=0x7fc4c000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4bea00640 1 -- 172.21.15.71:0/1103761057 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c000ce20 con 0x7fc4c8109140 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4bea00640 1 -- 172.21.15.71:0/1103761057 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c0005020 con 0x7fc4c8109140 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.313+0000 7fc4bea00640 1 -- 172.21.15.71:0/1103761057 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0010720 con 0x7fc4c8109140 2024-09-05T12:40:41.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.314+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c819f8d0 con 0x7fc4c8109140 2024-09-05T12:40:41.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.314+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c81a71b0 con 0x7fc4c8109140 2024-09-05T12:40:41.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.317+0000 7fc4cf874740 1 Processor -- start 2024-09-05T12:40:41.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.317+0000 7fc4bea00640 1 -- 172.21.15.71:0/1103761057 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc4c0005540 con 0x7fc4c8109140 2024-09-05T12:40:41.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 -- start start 2024-09-05T12:40:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cc08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556b66c19650 con 0x556b66cb3310 2024-09-05T12:40:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b66bc0f10 con 0x556b66cb3c30 2024-09-05T12:40:41.319 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4cf874740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b66c15610 con 0x556b66cbe4f0 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40822/0 (socket says 172.21.15.71:40822) 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42084/0 (socket says 172.21.15.71:42084) 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.318+0000 7fc4c5400640 1 -- 172.21.15.71:0/1933753813 learned_addr learned my addr 172.21.15.71:0/1933753813 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5400640 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cc08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5e00640 1 -- 172.21.15.71:0/1933753813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 msgr2=0x556b66cc08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cc08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5e00640 1 -- 172.21.15.71:0/1933753813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 msgr2=0x556b66cb36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5e00640 1 -- 172.21.15.71:0/1933753813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b66928580 con 0x556b66cb3c30 2024-09-05T12:40:41.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4bea00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4ac0a2cf0 0x7fc4ac0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.321 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbdfb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8005940 tx=0x7fc4b8005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a3458e83cb641b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.319+0000 7fc4bea00640 1 -- 172.21.15.71:0/1103761057 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7fc4c00d0200 con 0x7fc4c8109140 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4b3400640 1 -- 172.21.15.71:0/1933753813 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8035040 con 0x556b66cb3c30 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4b3400640 1 -- 172.21.15.71:0/1933753813 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8031830 con 0x556b66cb3c30 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4c5400640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4ac0a2cf0 0x7fc4ac0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4b3400640 1 -- 172.21.15.71:0/1933753813 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8004750 con 0x556b66cb3c30 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4cf874740 1 -- 172.21.15.71:0/1933753813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 msgr2=0x556b66cbdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4cf874740 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbdfb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8005940 tx=0x7fc4b8005d50 comp rx=0 tx=0).stop 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.320+0000 7fc4c5400640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4ac0a2cf0 0x7fc4ac0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x556b66cb0c20 tx=0x7fc4b4009c10 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:41.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 -- 172.21.15.71:0/1933753813 shutdown_connections 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cc08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbdfb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 --2- 172.21.15.71:0/1933753813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 -- 172.21.15.71:0/1933753813 >> 172.21.15.71:0/1933753813 conn(0x556b66caf8a0 msgr2=0x556b66cafc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 -- 172.21.15.71:0/1933753813 shutdown_connections 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 -- 172.21.15.71:0/1933753813 wait complete. 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 -- start start 2024-09-05T12:40:41.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.321+0000 7fc4cf874740 1 Processor -- start 2024-09-05T12:40:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cbb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cbbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556b66cc10b0 con 0x556b66cb3310 2024-09-05T12:40:41.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b66bd4350 con 0x556b66cb3c30 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b66cc1230 con 0x556b66cbe4f0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4cf874740 10 client.0 _ll_register_callbacks cb 0x556b66bf7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cbb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cbb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40838/0 (socket says 172.21.15.71:40838) 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4c5e00640 1 -- 172.21.15.71:0/2972347368 learned_addr learned my addr 172.21.15.71:0/2972347368 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4c5400640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.322+0000 7fc4c6800640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cbbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4c6800640 1 -- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 msgr2=0x556b66cbb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4c6800640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4c6800640 1 -- 172.21.15.71:0/2972347368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 msgr2=0x556b66cbb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4c6800640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cbb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4c6800640 1 -- 172.21.15.71:0/2972347368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b669f2960 con 0x556b66cbe4f0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4c6800640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cbbe00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00cd560 tx=0x7fc4c009caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6b654507ddd4f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:41.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4a7e00640 1 -- 172.21.15.71:0/2972347368 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00a4fa0 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b8031830 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.323+0000 7fc4a7e00640 1 -- 172.21.15.71:0/2972347368 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7fc4c009b780 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4a7e00640 1 -- 172.21.15.71:0/2972347368 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc4c00b5240 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 10 client.26944 fetch_fsmap learned FSMap version 52 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556b66ccccc0 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4a7e00640 1 -- 172.21.15.71:0/2972347368 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8031830 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4a7e00640 1 -- 172.21.15.71:0/2972347368 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7fc4c0020070 con 0x556b66cbe4f0 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 10 client.26944 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 10 client.26944 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 20 client.26944 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66cce120 0x556b66cd0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 4 client.26944 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:41.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.324+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b66c26bc0 con 0x556b66cce120 2024-09-05T12:40:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.325+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66cce120 0x556b66cd0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.325+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66cce120 0x556b66cd0550 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab6a56a4f72c458 server_cookie=11d1ea9791b9aa39 in_seq=0 out_seq=0 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.325+0000 7fc4a7e00640 10 client.26944 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4a7e00640 1 -- 172.21.15.71:0/2972347368 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b66c26bc0 con 0x556b66cce120 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4a7e00640 10 client.26944 handle_command_reply: tid=0 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 client.26944 shutdown 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 msgr2=0x556b66cbbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cbbe00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00cd560 tx=0x7fc4c009caf0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 shutdown_connections 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66cce120 0x556b66cd0550 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cbe4f0 0x556b66cbbe00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cb3c30 0x556b66cbb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 --2- 172.21.15.71:0/2972347368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cbb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4a7e00640 0 client.26944 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:41.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.326+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 >> 172.21.15.71:0/2972347368 conn(0x556b66caf8a0 msgr2=0x556b66caeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:41.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.327+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 shutdown_connections 2024-09-05T12:40:41.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.327+0000 7fc4cf874740 1 -- 172.21.15.71:0/2972347368 wait complete. 2024-09-05T12:40:41.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.327+0000 7fc4cf874740 20 client.26944 trim_cache size 0 max 16384 2024-09-05T12:40:41.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.344+0000 7fc4cf874740 1 Processor -- start 2024-09-05T12:40:41.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.344+0000 7fc4cf874740 1 -- start start 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3c30 0x556b66cae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cd5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4cf874740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b1210 con 0x556b66cb3310 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4cf874740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b2d50 con 0x556b66cbe4f0 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4cf874740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b18e0 con 0x556b66cb3c30 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40850/0 (socket says 172.21.15.71:40850) 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 -- 172.21.15.71:0/299717998 learned_addr learned my addr 172.21.15.71:0/299717998 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:41.346 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c6800640 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cd5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5400640 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3c30 0x556b66cae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 -- 172.21.15.71:0/299717998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3c30 msgr2=0x556b66cae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3c30 0x556b66cae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 -- 172.21.15.71:0/299717998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 msgr2=0x556b66cd5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cd5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c5e00640 1 -- 172.21.15.71:0/299717998 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b66caf6d0 con 0x556b66cb3310 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.345+0000 7fc4c6800640 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cd5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.346+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb5dc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8031550 tx=0x7fc4b80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82891c60e529e6d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.346+0000 7fc4b2a00640 1 -- 172.21.15.71:0/299717998 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8035050 con 0x556b66cb3310 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.346+0000 7fc4b2a00640 1 -- 172.21.15.71:0/299717998 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b80317b0 con 0x556b66cb3310 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.346+0000 7fc4b2a00640 1 -- 172.21.15.71:0/299717998 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8046450 con 0x556b66cb3310 2024-09-05T12:40:41.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 -- 172.21.15.71:0/299717998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 msgr2=0x556b66cb5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb5dc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8031550 tx=0x7fc4b80058d0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 -- 172.21.15.71:0/299717998 shutdown_connections 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cd5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3c30 0x556b66cae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 --2- 172.21.15.71:0/299717998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3310 0x556b66cb5dc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 -- 172.21.15.71:0/299717998 >> 172.21.15.71:0/299717998 conn(0x556b66caf8a0 msgr2=0x556b66cae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 -- 172.21.15.71:0/299717998 shutdown_connections 2024-09-05T12:40:41.349 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 -- 172.21.15.71:0/299717998 wait complete. 2024-09-05T12:40:41.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.347+0000 7fc4cf874740 1 -- start start 2024-09-05T12:40:41.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 Processor -- start 2024-09-05T12:40:41.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3310 0x556b66cc2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3c30 0x556b66cc28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cc2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b2680 con 0x556b66cb3c30 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81b1fb0 con 0x556b66cbe4f0 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81af210 con 0x556b66cb3310 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4cf874740 10 client.0 _ll_register_callbacks cb 0x7fc4c0005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3c30 0x556b66cc28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4c5e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3310 0x556b66cc2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4c5400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3c30 0x556b66cc28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40858/0 (socket says 172.21.15.71:40858) 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4c5400640 1 -- 172.21.15.71:0/4011426045 learned_addr learned my addr 172.21.15.71:0/4011426045 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.348+0000 7fc4c6800640 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cc2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4c5e00640 1 -- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 msgr2=0x556b66cc2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cc2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4c5e00640 1 -- 172.21.15.71:0/4011426045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3c30 msgr2=0x556b66cc28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3c30 0x556b66cc28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4c5e00640 1 -- 172.21.15.71:0/4011426045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b66d37f50 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4c5e00640 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3310 0x556b66cc2380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8052040 tx=0x7fc4b8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743851dd6a2dc1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4a7e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4a7e00640 1 -- 172.21.15.71:0/4011426045 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8035040 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.349+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4b80317b0 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4a7e00640 1 -- 172.21.15.71:0/4011426045 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7fc4b80035f0 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4a7e00640 1 -- 172.21.15.71:0/4011426045 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fc4b806c050 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 10 client.26950 fetch_fsmap learned FSMap version 52 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x556b66d39250 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4a7e00640 1 -- 172.21.15.71:0/4011426045 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b80317b0 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4a7e00640 1 -- 172.21.15.71:0/4011426045 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7fc4b80469c0 con 0x556b66cb3310 2024-09-05T12:40:41.353 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 10 client.26950 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 10 client.26950 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 20 client.26950 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66d394f0 0x556b66d3b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 4 client.26950 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "358ac758", "release": true, "await": true}] 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.350+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "358ac758", "release": true, "await": true}) -- 0x556b66c210d0 con 0x556b66d394f0 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.351+0000 7fc4c5400640 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66d394f0 0x556b66d3b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.351+0000 7fc4a7e00640 10 client.26950 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:41.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.351+0000 7fc4c5400640 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66d394f0 0x556b66d3b8e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b0e10e5c4ee76d server_cookie=e4acfcf09f73ebb8 in_seq=0 out_seq=0 2024-09-05T12:40:41.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4a7e00640 1 -- 172.21.15.71:0/4011426045 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b66c210d0 con 0x556b66d394f0 2024-09-05T12:40:41.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4a7e00640 10 client.26950 handle_command_reply: tid=0 2024-09-05T12:40:41.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 client.26950 shutdown 2024-09-05T12:40:41.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3310 msgr2=0x556b66cc2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.359 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3310 0x556b66cc2380 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc4b8052040 tx=0x7fc4b8009560 comp rx=0 tx=0).stop 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 shutdown_connections 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b66d394f0 0x556b66d3b8e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b66cbe4f0 0x556b66cc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b66cb3c30 0x556b66cc28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4a7e00640 0 client.26950 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 --2- 172.21.15.71:0/4011426045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b66cb3310 0x556b66cc2380 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 >> 172.21.15.71:0/4011426045 conn(0x556b66caf8a0 msgr2=0x556b66caeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 shutdown_connections 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.358+0000 7fc4cf874740 1 -- 172.21.15.71:0/4011426045 wait complete. 2024-09-05T12:40:41.360 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.359+0000 7fc4cf874740 20 client.26950 trim_cache size 0 max 16384 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 52, 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 44, 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "358ac758": { 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 44, 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:40:41.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 150.0, 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:40:41.362 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.359+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4ac0a2cf0 msgr2=0x7fc4ac0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.359+0000 7fc4cce00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4ac0a2cf0 0x7fc4ac0a51b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x556b66cb0c20 tx=0x7fc4b4009c10 comp rx=0 tx=0).stop 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.359+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8109140 msgr2=0x7fc4c819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.359+0000 7fc4cce00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8109140 0x7fc4c819ef00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009fd0 tx=0x7fc4c000ef10 comp rx=0 tx=0).stop 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 shutdown_connections 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8109140 0x7fc4c819ef00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c81049c0 0x7fc4c819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.363 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc4ac0a2cf0 0x7fc4ac0a51b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 --2- 172.21.15.71:0/1103761057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4c8104010 0x7fc4c819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 >> 172.21.15.71:0/1103761057 conn(0x7fc4c80ffa50 msgr2=0x7fc4c8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 shutdown_connections 2024-09-05T12:40:41.364 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:40:41.360+0000 7fc4cce00640 1 -- 172.21.15.71:0/1103761057 wait complete. 2024-09-05T12:40:42.257 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '358ac758', total seconds elapsed: 33.9 2024-09-05T12:40:42.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 407953 2.91 MB/sec execute 578 sec latency 29663.890 ms 2024-09-05T12:40:42.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:42 smithi071 ceph-mon[29845]: pgmap v406: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-09-05T12:40:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:42 smithi123 ceph-mon[34321]: pgmap v406: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-09-05T12:40:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:42 smithi174 ceph-mon[34313]: pgmap v406: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-09-05T12:40:43.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 408987 2.91 MB/sec execute 579 sec latency 12.069 ms 2024-09-05T12:40:44.298 INFO:tasks.workunit.client.0.smithi071.stdout: 1 409641 2.91 MB/sec execute 580 sec latency 413.174 ms 2024-09-05T12:40:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:44 smithi071 ceph-mon[29845]: pgmap v407: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-05T12:40:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:44 smithi123 ceph-mon[34321]: pgmap v407: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-05T12:40:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:44 smithi174 ceph-mon[34313]: pgmap v407: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-09-05T12:40:45.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 409641 2.91 MB/sec execute 581 sec latency 1413.222 ms 2024-09-05T12:40:46.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 409641 2.90 MB/sec execute 582 sec latency 2413.367 ms 2024-09-05T12:40:46.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:46 smithi071 ceph-mon[29845]: pgmap v408: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-05T12:40:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:46 smithi123 ceph-mon[34321]: pgmap v408: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-05T12:40:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:46 smithi174 ceph-mon[34313]: pgmap v408: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 13 op/s 2024-09-05T12:40:47.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 411599 2.92 MB/sec execute 583 sec latency 2503.745 ms 2024-09-05T12:40:48.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 412539 2.92 MB/sec execute 584 sec latency 26.133 ms 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.470+0000 7f302b200640 1 -- 172.21.15.71:0/1336221419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c103ae0 msgr2=0x7f302c103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.470+0000 7f302b200640 1 --2- 172.21.15.71:0/1336221419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c103ae0 0x7f302c103f40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3018009930 tx=0x7f301802a420 comp rx=0 tx=0).stop 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 -- 172.21.15.71:0/1336221419 shutdown_connections 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 --2- 172.21.15.71:0/1336221419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f302c104480 0x7f302c1109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 --2- 172.21.15.71:0/1336221419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c103ae0 0x7f302c103f40 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 --2- 172.21.15.71:0/1336221419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c109b80 0x7f302c109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 -- 172.21.15.71:0/1336221419 >> 172.21.15.71:0/1336221419 conn(0x7f302c0ff7c0 msgr2=0x7f302c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 -- 172.21.15.71:0/1336221419 shutdown_connections 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 -- 172.21.15.71:0/1336221419 wait complete. 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.471+0000 7f302b200640 1 Processor -- start 2024-09-05T12:40:48.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 -- start start 2024-09-05T12:40:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 0x7f302c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c104480 0x7f302c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f302c109b80 0x7f302c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f302c113f40 con 0x7f302c104480 2024-09-05T12:40:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f302c113dc0 con 0x7f302c109b80 2024-09-05T12:40:48.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1140c0 con 0x7f302c103ae0 2024-09-05T12:40:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f3029e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 0x7f302c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f3029400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c104480 0x7f302c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f3029e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 0x7f302c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47164/0 (socket says 172.21.15.71:47164) 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f3029e00640 1 -- 172.21.15.71:0/2682045987 learned_addr learned my addr 172.21.15.71:0/2682045987 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.472+0000 7f302a800640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f302c109b80 0x7f302c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3029e00640 1 -- 172.21.15.71:0/2682045987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f302c109b80 msgr2=0x7f302c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3029e00640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f302c109b80 0x7f302c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3029e00640 1 -- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c104480 msgr2=0x7f302c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3029e00640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c104480 0x7f302c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3029e00640 1 -- 172.21.15.71:0/2682045987 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c19f4a0 con 0x7f302c103ae0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3029e00640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 0x7f302c19a4b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f301c00e9f0 tx=0x7f301c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3022a00640 1 -- 172.21.15.71:0/2682045987 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c00cd80 con 0x7f302c103ae0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c19f790 con 0x7f302c103ae0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3022a00640 1 -- 172.21.15.71:0/2682045987 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c010910 con 0x7f302c103ae0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f3022a00640 1 -- 172.21.15.71:0/2682045987 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0188f0 con 0x7f302c103ae0 2024-09-05T12:40:48.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.473+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c1a7020 con 0x7f302c103ae0 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.476+0000 7f30338a7740 1 Processor -- start 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.476+0000 7f3022a00640 1 -- 172.21.15.71:0/2682045987 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f301c010ac0 con 0x7f302c103ae0 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.476+0000 7f30338a7740 1 -- start start 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f30338a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375b3b0 0x55b18375b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f30338a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183766030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f30338a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183766570 0x55b183768960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f30338a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b1836c1100 con 0x55b18375b3b0 2024-09-05T12:40:48.479 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f30338a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b1836691b0 con 0x55b183766570 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f30338a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b1836bd0c0 con 0x55b18375bcd0 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f3029e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183766030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f302a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375b3b0 0x55b18375b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f3029e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183766030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47180/0 (socket says 172.21.15.71:47180) 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f3029e00640 1 -- 172.21.15.71:0/580045003 learned_addr learned my addr 172.21.15.71:0/580045003 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f302a800640 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375b3b0 0x55b18375b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50160/0 (socket says 172.21.15.71:50160) 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f3029400640 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183766570 0x55b183768960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f3029e00640 1 -- 172.21.15.71:0/580045003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183766570 msgr2=0x55b183768960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.480 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.477+0000 7f3029e00640 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183766570 0x55b183768960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3029e00640 1 -- 172.21.15.71:0/580045003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375b3b0 msgr2=0x55b18375b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3029e00640 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375b3b0 0x55b18375b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3029e00640 1 -- 172.21.15.71:0/580045003 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1833d4210 con 0x55b18375bcd0 2024-09-05T12:40:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3029400640 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183766570 0x55b183768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:48.484 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3029e00640 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183766030 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f301c0cce20 tx=0x7f301c0d8270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9106172209d737 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3017000640 1 -- 172.21.15.71:0/580045003 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0df020 con 0x55b18375bcd0 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3017000640 1 -- 172.21.15.71:0/580045003 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c0d8cb0 con 0x55b18375bcd0 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3017000640 1 -- 172.21.15.71:0/580045003 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0da660 con 0x55b18375bcd0 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3022a00640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30100a2cf0 0x7f30100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.478+0000 7f3022a00640 1 -- 172.21.15.71:0/2682045987 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f301c0cfb80 con 0x7f302c103ae0 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f302a800640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30100a2cf0 0x7f30100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 -- 172.21.15.71:0/580045003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 msgr2=0x55b183766030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183766030 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f301c0cce20 tx=0x7f301c0d8270 comp rx=0 tx=0).stop 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 -- 172.21.15.71:0/580045003 shutdown_connections 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183766570 0x55b183768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183766030 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 --2- 172.21.15.71:0/580045003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375b3b0 0x55b18375b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 -- 172.21.15.71:0/580045003 >> 172.21.15.71:0/580045003 conn(0x55b183757920 msgr2=0x55b183757d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f302a800640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30100a2cf0 0x7f30100a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3024005fd0 tx=0x7f3024009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 -- 172.21.15.71:0/580045003 shutdown_connections 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.479+0000 7f30338a7740 1 -- 172.21.15.71:0/580045003 wait complete. 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 -- start start 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 Processor -- start 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b1837636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183763bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b183764130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b183768ff0 con 0x55b183766570 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b18364a090 con 0x55b18375b3b0 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b183769170 con 0x55b18375bcd0 2024-09-05T12:40:48.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.480+0000 7f30338a7740 10 client.0 _ll_register_callbacks cb 0x55b18369ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183763bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f302a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b183764130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183763bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47182/0 (socket says 172.21.15.71:47182) 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029400640 1 -- 172.21.15.71:0/3342697520 learned_addr learned my addr 172.21.15.71:0/3342697520 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b1837636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 -- 172.21.15.71:0/3342697520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 msgr2=0x55b183763bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183763bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 -- 172.21.15.71:0/3342697520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 msgr2=0x55b183764130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b183764130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 -- 172.21.15.71:0/3342697520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1834a0620 con 0x55b18375b3b0 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029400640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183763bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:48.487 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f302a800640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b183764130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f3029e00640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b1837636b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f301c09ba60 tx=0x7f301c0dbbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7056156985d8e878 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f300b400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f300b400640 1 -- 172.21.15.71:0/3342697520 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0df030 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.481+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301c0d8cb0 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f300b400640 1 -- 172.21.15.71:0/3342697520 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f301c0cd160 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f300b400640 1 -- 172.21.15.71:0/3342697520 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f301c098000 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 10 client.26927 fetch_fsmap learned FSMap version 52 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b183774dd0 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f300b400640 1 -- 172.21.15.71:0/3342697520 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c0d8cb0 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f300b400640 1 -- 172.21.15.71:0/3342697520 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7f301c0a4490 con 0x55b18375b3b0 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 10 client.26927 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:48.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 20 client.26927 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b183776120 0x55b183778510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 4 client.26927 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:40:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b1836ce830 con 0x55b183776120 2024-09-05T12:40:48.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.482+0000 7f3029400640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b183776120 0x55b183778510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.483+0000 7f3029400640 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b183776120 0x55b183778510 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be02e1696630f517 server_cookie=d13b9656f92b6142 in_seq=0 out_seq=0 2024-09-05T12:40:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.483+0000 7f300b400640 10 client.26927 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f300b400640 1 -- 172.21.15.71:0/3342697520 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b1836ce830 con 0x55b183776120 2024-09-05T12:40:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f300b400640 10 client.26927 handle_command_reply: tid=0 2024-09-05T12:40:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 client.26927 shutdown 2024-09-05T12:40:48.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 msgr2=0x55b1837636b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b1837636b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f301c09ba60 tx=0x7f301c0dbbb0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 shutdown_connections 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b183776120 0x55b183778510 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b183764130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375bcd0 0x55b183763bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 --2- 172.21.15.71:0/3342697520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b1837636b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f300b400640 0 client.26927 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 >> 172.21.15.71:0/3342697520 conn(0x55b183757920 msgr2=0x55b18375d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 shutdown_connections 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 1 -- 172.21.15.71:0/3342697520 wait complete. 2024-09-05T12:40:48.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.484+0000 7f30338a7740 20 client.26927 trim_cache size 0 max 16384 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 Processor -- start 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 -- start start 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b183757d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375bcd0 0x55b183763d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183766570 0x55b18377db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b10d0 con 0x55b18375bcd0 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b2c10 con 0x55b18375b3b0 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f30338a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b17a0 con 0x55b183766570 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f3029e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b183757d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f3029400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375bcd0 0x55b183763d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f3029e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b183757d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55690/0 (socket says 172.21.15.71:55690) 2024-09-05T12:40:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f3029e00640 1 -- 172.21.15.71:0/2242156121 learned_addr learned my addr 172.21.15.71:0/2242156121 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f302a800640 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183766570 0x55b18377db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f3029e00640 1 -- 172.21.15.71:0/2242156121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183766570 msgr2=0x55b18377db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.501+0000 7f3029e00640 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183766570 0x55b18377db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3029e00640 1 -- 172.21.15.71:0/2242156121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375bcd0 msgr2=0x55b183763d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3029e00640 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375bcd0 0x55b183763d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3029e00640 1 -- 172.21.15.71:0/2242156121 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b183758610 con 0x55b18375b3b0 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3029e00640 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b183757d00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c6a0 tx=0x7f301c09a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a08e652d46fcbd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3016600640 1 -- 172.21.15.71:0/2242156121 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0df070 con 0x55b18375b3b0 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3016600640 1 -- 172.21.15.71:0/2242156121 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c0d3720 con 0x55b18375b3b0 2024-09-05T12:40:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.502+0000 7f3016600640 1 -- 172.21.15.71:0/2242156121 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c09b6c0 con 0x55b18375b3b0 2024-09-05T12:40:48.508 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 -- 172.21.15.71:0/2242156121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 msgr2=0x55b183757d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b183757d00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c6a0 tx=0x7f301c09a270 comp rx=0 tx=0).stop 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 -- 172.21.15.71:0/2242156121 shutdown_connections 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183766570 0x55b18377db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b18375bcd0 0x55b183763d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 --2- 172.21.15.71:0/2242156121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375b3b0 0x55b183757d00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 -- 172.21.15.71:0/2242156121 >> 172.21.15.71:0/2242156121 conn(0x55b183757920 msgr2=0x55b183763670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 -- 172.21.15.71:0/2242156121 shutdown_connections 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 -- 172.21.15.71:0/2242156121 wait complete. 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 -- start start 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.503+0000 7f30338a7740 1 Processor -- start 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 0x55b18376e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.509 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375bcd0 0x55b1837602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b1837607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b2540 con 0x55b183766570 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1b1e70 con 0x55b18375bcd0 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f302c1af0d0 con 0x55b18375b3b0 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f30338a7740 10 client.0 _ll_register_callbacks cb 0x7f3024009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 0x55b18376e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 0x55b18376e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:47200/0 (socket says 172.21.15.71:47200) 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 -- 172.21.15.71:0/4145789460 learned_addr learned my addr 172.21.15.71:0/4145789460 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029400640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375bcd0 0x55b1837602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f302a800640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b1837607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 -- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375bcd0 msgr2=0x55b1837602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375bcd0 0x55b1837602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 -- 172.21.15.71:0/4145789460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 msgr2=0x55b1837607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:40:48.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.504+0000 7f3029e00640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b1837607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f3029e00640 1 -- 172.21.15.71:0/4145789460 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b183760ed0 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f302a800640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b1837607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f3029e00640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 0x55b18376e7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f301c0981c0 tx=0x7f301c09bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d774e87f1aea7f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f300b400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f300b400640 1 -- 172.21.15.71:0/4145789460 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0df030 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f301c0d3720 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f300b400640 1 -- 172.21.15.71:0/4145789460 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7f301c0cd160 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f300b400640 1 -- 172.21.15.71:0/4145789460 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f301c09b8a0 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f30338a7740 10 client.26968 fetch_fsmap learned FSMap version 52 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55b1837e0a00 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.505+0000 7f300b400640 1 -- 172.21.15.71:0/4145789460 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f301c0d3720 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f300b400640 1 -- 172.21.15.71:0/4145789460 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7f301c09b1b0 con 0x55b18375b3b0 2024-09-05T12:40:48.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f30338a7740 10 client.26968 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f30338a7740 10 client.26968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f30338a7740 20 client.26968 populate_metadata read hostname 'smithi071' 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f30338a7740 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b1837e0f20 0x55b1837e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f30338a7740 4 client.26968 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b1836c8b80 con 0x55b1837e0f20 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.506+0000 7f3029400640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b1837e0f20 0x55b1837e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.507+0000 7f3029400640 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b1837e0f20 0x55b1837e3310 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4727893d4ca93237 server_cookie=3e65327c521710ca in_seq=0 out_seq=0 2024-09-05T12:40:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.507+0000 7f300b400640 10 client.26968 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.531+0000 7f300b400640 1 -- 172.21.15.71:0/4145789460 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b1836c8b80 con 0x55b1837e0f20 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.531+0000 7f300b400640 10 client.26968 handle_command_reply: tid=0 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.531+0000 7f30338a7740 1 client.26968 shutdown 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.531+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 msgr2=0x55b18376e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.531+0000 7f30338a7740 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 0x55b18376e7c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f301c0981c0 tx=0x7f301c09bee0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 shutdown_connections 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b1837e0f20 0x55b1837e3310 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b183766570 0x55b1837607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b18375bcd0 0x55b1837602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 --2- 172.21.15.71:0/4145789460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b18375b3b0 0x55b18376e7c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f300b400640 0 client.26968 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 >> 172.21.15.71:0/4145789460 conn(0x55b183757920 msgr2=0x55b18375d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 shutdown_connections 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 1 -- 172.21.15.71:0/4145789460 wait complete. 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.532+0000 7f30338a7740 20 client.26968 trim_cache size 0 max 16384 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:40:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:40:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:40:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30100a2cf0 msgr2=0x7f30100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30100a2cf0 0x7f30100a51b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3024005fd0 tx=0x7f3024009500 comp rx=0 tx=0).stop 2024-09-05T12:40:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 msgr2=0x7f302c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:40:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 0x7f302c19a4b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f301c00e9f0 tx=0x7f301c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 shutdown_connections 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f302c109b80 0x7f302c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30100a2cf0 0x7f30100a51b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f302c104480 0x7f302c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 --2- 172.21.15.71:0/2682045987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f302c103ae0 0x7f302c19a4b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.533+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 >> 172.21.15.71:0/2682045987 conn(0x7f302c0ff7c0 msgr2=0x7f302c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.534+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 shutdown_connections 2024-09-05T12:40:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:40:48.534+0000 7f302b200640 1 -- 172.21.15.71:0/2682045987 wait complete. 2024-09-05T12:40:48.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:48 smithi071 ceph-mon[29845]: pgmap v409: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 336 KiB/s wr, 63 op/s 2024-09-05T12:40:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:48 smithi123 ceph-mon[34321]: pgmap v409: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 336 KiB/s wr, 63 op/s 2024-09-05T12:40:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:48 smithi174 ceph-mon[34313]: pgmap v409: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 336 KiB/s wr, 63 op/s 2024-09-05T12:40:49.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 413692 2.92 MB/sec execute 585 sec latency 358.627 ms 2024-09-05T12:40:50.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 415361 2.94 MB/sec execute 586 sec latency 13.907 ms 2024-09-05T12:40:50.562 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:50 smithi174 ceph-mon[34313]: pgmap v410: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 315 KiB/s wr, 54 op/s 2024-09-05T12:40:50.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:50 smithi071 ceph-mon[29845]: pgmap v410: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 315 KiB/s wr, 54 op/s 2024-09-05T12:40:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:50 smithi123 ceph-mon[34321]: pgmap v410: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 315 KiB/s wr, 54 op/s 2024-09-05T12:40:51.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 416508 2.94 MB/sec execute 587 sec latency 18.162 ms 2024-09-05T12:40:52.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 417791 2.95 MB/sec execute 588 sec latency 268.896 ms 2024-09-05T12:40:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:52 smithi071 ceph-mon[29845]: pgmap v411: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 818 KiB/s wr, 132 op/s 2024-09-05T12:40:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:52 smithi123 ceph-mon[34321]: pgmap v411: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 818 KiB/s wr, 132 op/s 2024-09-05T12:40:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:52 smithi174 ceph-mon[34313]: pgmap v411: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 818 KiB/s wr, 132 op/s 2024-09-05T12:40:53.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 419127 2.96 MB/sec execute 589 sec latency 14.419 ms 2024-09-05T12:40:54.299 INFO:tasks.workunit.client.0.smithi071.stdout: 1 420275 2.96 MB/sec execute 590 sec latency 134.632 ms 2024-09-05T12:40:54.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:54 smithi071 ceph-mon[29845]: pgmap v412: 129 pgs: 129 active+clean; 138 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 855 KiB/s wr, 136 op/s 2024-09-05T12:40:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:54 smithi123 ceph-mon[34321]: pgmap v412: 129 pgs: 129 active+clean; 138 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 855 KiB/s wr, 136 op/s 2024-09-05T12:40:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:54 smithi174 ceph-mon[34313]: pgmap v412: 129 pgs: 129 active+clean; 138 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 855 KiB/s wr, 136 op/s 2024-09-05T12:40:55.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 420275 2.96 MB/sec execute 591 sec latency 1064.650 ms 2024-09-05T12:40:56.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 420275 2.95 MB/sec execute 592 sec latency 2064.791 ms 2024-09-05T12:40:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:56 smithi071 ceph-mon[29845]: pgmap v413: 129 pgs: 129 active+clean; 138 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 855 KiB/s wr, 134 op/s 2024-09-05T12:40:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:56 smithi123 ceph-mon[34321]: pgmap v413: 129 pgs: 129 active+clean; 138 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 855 KiB/s wr, 134 op/s 2024-09-05T12:40:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:56 smithi174 ceph-mon[34313]: pgmap v413: 129 pgs: 129 active+clean; 138 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 855 KiB/s wr, 134 op/s 2024-09-05T12:40:57.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 422235 2.97 MB/sec execute 593 sec latency 2158.835 ms 2024-09-05T12:40:58.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 423180 2.97 MB/sec execute 594 sec latency 49.627 ms 2024-09-05T12:40:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:40:58 smithi071 ceph-mon[29845]: pgmap v414: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-09-05T12:40:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:40:58 smithi123 ceph-mon[34321]: pgmap v414: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-09-05T12:40:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:40:58 smithi174 ceph-mon[34313]: pgmap v414: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-09-05T12:40:59.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 423824 2.97 MB/sec execute 595 sec latency 420.478 ms 2024-09-05T12:41:00.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 423824 2.96 MB/sec execute 596 sec latency 1420.592 ms 2024-09-05T12:41:00.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:00 smithi071 ceph-mon[29845]: pgmap v415: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 981 KiB/s wr, 156 op/s 2024-09-05T12:41:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:00 smithi123 ceph-mon[34321]: pgmap v415: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 981 KiB/s wr, 156 op/s 2024-09-05T12:41:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:00 smithi174 ceph-mon[34313]: pgmap v415: 129 pgs: 129 active+clean; 142 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 981 KiB/s wr, 156 op/s 2024-09-05T12:41:01.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 424406 2.96 MB/sec execute 597 sec latency 1923.870 ms 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.659+0000 7fe57d400640 1 -- 172.21.15.71:0/2339956696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe57810bf00 msgr2=0x7fe57810e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.659+0000 7fe57d400640 1 --2- 172.21.15.71:0/2339956696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe57810bf00 0x7fe57810e370 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe570009ca0 tx=0x7fe57002a950 comp rx=0 tx=0).stop 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 -- 172.21.15.71:0/2339956696 shutdown_connections 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 --2- 172.21.15.71:0/2339956696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe57810bf00 0x7fe57810e370 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 --2- 172.21.15.71:0/2339956696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe578103400 0x7fe57810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 --2- 172.21.15.71:0/2339956696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe578102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 -- 172.21.15.71:0/2339956696 >> 172.21.15.71:0/2339956696 conn(0x7fe5780777c0 msgr2=0x7fe5780feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 -- 172.21.15.71:0/2339956696 shutdown_connections 2024-09-05T12:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.660+0000 7fe57d400640 1 -- 172.21.15.71:0/2339956696 wait complete. 2024-09-05T12:41:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 Processor -- start 2024-09-05T12:41:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 -- start start 2024-09-05T12:41:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe57819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578103400 0x7fe57819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe57810bf00 0x7fe57819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe578111c90 con 0x7fe578103400 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe578111b10 con 0x7fe578102ae0 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57d400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe578111e10 con 0x7fe57810bf00 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe577e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe57819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe577e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe57819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40304/0 (socket says 172.21.15.71:40304) 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe577e00640 1 -- 172.21.15.71:0/388875216 learned_addr learned my addr 172.21.15.71:0/388875216 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe577400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578103400 0x7fe57819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.661+0000 7fe57ca00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe57810bf00 0x7fe57819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe577e00640 1 -- 172.21.15.71:0/388875216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe57810bf00 msgr2=0x7fe57819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe577e00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe57810bf00 0x7fe57819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe577e00640 1 -- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578103400 msgr2=0x7fe57819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe577e00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578103400 0x7fe57819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe577e00640 1 -- 172.21.15.71:0/388875216 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57819f380 con 0x7fe578102ae0 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe577e00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe57819a390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe56800ea60 tx=0x7fe56800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe574c00640 1 -- 172.21.15.71:0/388875216 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56800cde0 con 0x7fe578102ae0 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe574c00640 1 -- 172.21.15.71:0/388875216 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5680035f0 con 0x7fe578102ae0 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe574c00640 1 -- 172.21.15.71:0/388875216 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe568010670 con 0x7fe578102ae0 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57819f670 con 0x7fe578102ae0 2024-09-05T12:41:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.662+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5781a6f00 con 0x7fe578102ae0 2024-09-05T12:41:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 Processor -- start 2024-09-05T12:41:01.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe574c00640 1 -- 172.21.15.71:0/388875216 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe5680037a0 con 0x7fe578102ae0 2024-09-05T12:41:01.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 -- start start 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 0x563d48a56790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a56cd0 0x563d48a61030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a63960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563d489bc100 con 0x563d48a56cd0 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563d489641b0 con 0x563d48a563b0 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe58185b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d489b80c0 con 0x563d48a61570 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.666+0000 7fe57ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 0x563d48a56790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 0x563d48a56790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40306/0 (socket says 172.21.15.71:40306) 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 -- 172.21.15.71:0/1461078344 learned_addr learned my addr 172.21.15.71:0/1461078344 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe577400640 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a63960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe577e00640 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a56cd0 0x563d48a61030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 -- 172.21.15.71:0/1461078344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 msgr2=0x563d48a63960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a63960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 -- 172.21.15.71:0/1461078344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a56cd0 msgr2=0x563d48a61030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a56cd0 0x563d48a61030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 -- 172.21.15.71:0/1461078344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d486cf210 con 0x563d48a563b0 2024-09-05T12:41:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe57ca00640 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 0x563d48a56790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe570035000 tx=0x7fe5700093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5589f15b5062f54a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe56d200640 1 -- 172.21.15.71:0/1461078344 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe570034040 con 0x563d48a563b0 2024-09-05T12:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe574c00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe55c0a2d70 0x7fe55c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe56d200640 1 -- 172.21.15.71:0/1461078344 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57002f940 con 0x563d48a563b0 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.667+0000 7fe56d200640 1 -- 172.21.15.71:0/1461078344 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5700042a0 con 0x563d48a563b0 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe574c00640 1 -- 172.21.15.71:0/388875216 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7fe5680cfab0 con 0x7fe578102ae0 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe577e00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe55c0a2d70 0x7fe55c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 -- 172.21.15.71:0/1461078344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 msgr2=0x563d48a56790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 0x563d48a56790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe570035000 tx=0x7fe5700093e0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 -- 172.21.15.71:0/1461078344 shutdown_connections 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a63960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a56cd0 0x563d48a61030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 --2- 172.21.15.71:0/1461078344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a563b0 0x563d48a56790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 -- 172.21.15.71:0/1461078344 >> 172.21.15.71:0/1461078344 conn(0x563d48a52920 msgr2=0x563d48a52d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.668+0000 7fe58185b740 1 -- 172.21.15.71:0/1461078344 shutdown_connections 2024-09-05T12:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.669+0000 7fe58185b740 1 -- 172.21.15.71:0/1461078344 wait complete. 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.669+0000 7fe577e00640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe55c0a2d70 0x7fe55c0a5230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe568098040 tx=0x7fe568003f30 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.669+0000 7fe58185b740 1 -- start start 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.669+0000 7fe58185b740 1 Processor -- start 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a5e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 0x563d48a5eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a5f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563d48a63ff0 con 0x563d48a563b0 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563d48945090 con 0x563d48a56cd0 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d48a64170 con 0x563d48a61570 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe58185b740 10 client.0 _ll_register_callbacks cb 0x563d48999f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 0x563d48a5eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 0x563d48a5eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40316/0 (socket says 172.21.15.71:40316) 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 -- 172.21.15.71:0/626702485 learned_addr learned my addr 172.21.15.71:0/626702485 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe57ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a5e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe57ca00640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a5f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 -- 172.21.15.71:0/626702485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 msgr2=0x563d48a5f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a5f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 -- 172.21.15.71:0/626702485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 msgr2=0x563d48a5e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a5e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe577400640 1 -- 172.21.15.71:0/626702485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4879b620 con 0x563d48a56cd0 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.670+0000 7fe57ca00640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a5e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe577400640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 0x563d48a5eb90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe564012700 tx=0x7fe564012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b969f6b8d1e7586 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe561600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe561600640 1 -- 172.21.15.71:0/626702485 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400f8c0 con 0x563d48a56cd0 2024-09-05T12:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57002f940 con 0x563d48a56cd0 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe561600640 1 -- 172.21.15.71:0/626702485 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7fe56400fa60 con 0x563d48a56cd0 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe561600640 1 -- 172.21.15.71:0/626702485 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe56400fc60 con 0x563d48a56cd0 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe58185b740 10 client.26945 fetch_fsmap learned FSMap version 52 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563d48a786c0 con 0x563d48a56cd0 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe561600640 1 -- 172.21.15.71:0/626702485 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57002f940 con 0x563d48a56cd0 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe561600640 1 -- 172.21.15.71:0/626702485 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7fe564010550 con 0x563d48a56cd0 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe58185b740 10 client.26945 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.671+0000 7fe58185b740 10 client.26945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe58185b740 20 client.26945 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe58185b740 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48a79b80 0x563d48a7bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe58185b740 4 client.26945 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d489c9830 con 0x563d48a79b80 2024-09-05T12:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe57ca00640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48a79b80 0x563d48a7bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe57ca00640 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48a79b80 0x563d48a7bfd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca911dfb504d7bd3 server_cookie=b7c72ad20a7ea067 in_seq=0 out_seq=0 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.672+0000 7fe561600640 10 client.26945 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe561600640 1 -- 172.21.15.71:0/626702485 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d489c9830 con 0x563d48a79b80 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe561600640 10 client.26945 handle_command_reply: tid=0 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 client.26945 shutdown 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 msgr2=0x563d48a5eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 0x563d48a5eb90 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe564012700 tx=0x7fe564012bc0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 shutdown_connections 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48a79b80 0x563d48a7bfd0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a61570 0x563d48a5f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a56cd0 0x563d48a5eb90 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 --2- 172.21.15.71:0/626702485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a5e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe561600640 0 client.26945 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.673+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 >> 172.21.15.71:0/626702485 conn(0x563d48a52920 msgr2=0x563d48a58680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.674+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 shutdown_connections 2024-09-05T12:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.674+0000 7fe58185b740 1 -- 172.21.15.71:0/626702485 wait complete. 2024-09-05T12:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.674+0000 7fe58185b740 20 client.26945 trim_cache size 0 max 16384 2024-09-05T12:41:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 Processor -- start 2024-09-05T12:41:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 -- start start 2024-09-05T12:41:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a81e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781acb10 con 0x563d48a563b0 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781ae650 con 0x563d48a61570 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe58185b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781ad1e0 con 0x563d48a56cd0 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe577400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe577400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a818e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40292/0 (socket says 172.21.15.71:40292) 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe57ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.690+0000 7fe577400640 1 -- 172.21.15.71:0/2788303679 learned_addr learned my addr 172.21.15.71:0/2788303679 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe57ca00640 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a81e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe577400640 1 -- 172.21.15.71:0/2788303679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 msgr2=0x563d48a81e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe577400640 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a81e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe577400640 1 -- 172.21.15.71:0/2788303679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 msgr2=0x563d48a604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe577400640 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe577400640 1 -- 172.21.15.71:0/2788303679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d48a63cd0 con 0x563d48a56cd0 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe577400640 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a818e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe564010b60 tx=0x7fe56400ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e400f16fe3d594bd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe563e00640 1 -- 172.21.15.71:0/2788303679 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564010be0 con 0x563d48a56cd0 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe563e00640 1 -- 172.21.15.71:0/2788303679 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe564010d80 con 0x563d48a56cd0 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.691+0000 7fe563e00640 1 -- 172.21.15.71:0/2788303679 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640035f0 con 0x563d48a56cd0 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 -- 172.21.15.71:0/2788303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 msgr2=0x563d48a818e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a818e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe564010b60 tx=0x7fe56400ff90 comp rx=0 tx=0).stop 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 -- 172.21.15.71:0/2788303679 shutdown_connections 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a81e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a818e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 --2- 172.21.15.71:0/2788303679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 -- 172.21.15.71:0/2788303679 >> 172.21.15.71:0/2788303679 conn(0x563d48a52920 msgr2=0x563d48a5fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 -- 172.21.15.71:0/2788303679 shutdown_connections 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 -- 172.21.15.71:0/2788303679 wait complete. 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.692+0000 7fe58185b740 1 -- start start 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 Processor -- start 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a71c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a72190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a6ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781adf80 con 0x563d48a563b0 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781ad8b0 con 0x563d48a61570 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5781aab90 con 0x563d48a56cd0 2024-09-05T12:41:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe58185b740 10 client.0 _ll_register_callbacks cb 0x7fe564009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe57ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a71c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe577400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a72190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe577400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a72190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40300/0 (socket says 172.21.15.71:40300) 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.693+0000 7fe57ca00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a71c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52588/0 (socket says 172.21.15.71:52588) 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 -- 172.21.15.71:0/461192665 learned_addr learned my addr 172.21.15.71:0/461192665 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe57ca00640 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a6ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 -- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 msgr2=0x563d48a6ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a6ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 -- 172.21.15.71:0/461192665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 msgr2=0x563d48a71c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a71c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 -- 172.21.15.71:0/461192665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d48a72820 con 0x563d48a56cd0 2024-09-05T12:41:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe577400640 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a72190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5640109f0 tx=0x7fe564009280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7de4cc7478fc67f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe561600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe561600640 1 -- 172.21.15.71:0/461192665 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640050c0 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe564010d80 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.694+0000 7fe561600640 1 -- 172.21.15.71:0/461192665 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9438+0+0 (secure 0 0 0) 0x7fe564010640 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe561600640 1 -- 172.21.15.71:0/461192665 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fe564010b90 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 10 client.26986 fetch_fsmap learned FSMap version 52 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x563d48ae4e60 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe561600640 1 -- 172.21.15.71:0/461192665 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe564010d80 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe561600640 1 -- 172.21.15.71:0/461192665 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8374+0+0 (secure 0 0 0) 0x7fe56401c470 con 0x563d48a56cd0 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 10 client.26986 fetch_fsmap finished waiting for FSMap version 52 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 10 client.26986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 20 client.26986 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48ae5640 0x563d48ae7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 4 client.26986 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:41:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.695+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d489c3b80 con 0x563d48ae5640 2024-09-05T12:41:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.696+0000 7fe57ca00640 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48ae5640 0x563d48ae7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.696+0000 7fe57ca00640 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48ae5640 0x563d48ae7a60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1a91fe426384b74 server_cookie=c2722e6ba2c5c584 in_seq=0 out_seq=0 2024-09-05T12:41:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.696+0000 7fe561600640 10 client.26986 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.734+0000 7fe561600640 1 -- 172.21.15.71:0/461192665 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d489c3b80 con 0x563d48ae5640 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.734+0000 7fe561600640 10 client.26986 handle_command_reply: tid=0 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.734+0000 7fe58185b740 1 client.26986 shutdown 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.734+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 msgr2=0x563d48a72190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.734+0000 7fe58185b740 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a72190 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5640109f0 tx=0x7fe564009280 comp rx=0 tx=0).stop 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 shutdown_connections 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563d48ae5640 0x563d48ae7a60 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563d48a61570 0x563d48a6ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d48a56cd0 0x563d48a72190 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 --2- 172.21.15.71:0/461192665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d48a563b0 0x563d48a71c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 >> 172.21.15.71:0/461192665 conn(0x563d48a52920 msgr2=0x563d48a752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe561600640 0 client.26986 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 shutdown_connections 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 1 -- 172.21.15.71:0/461192665 wait complete. 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.735+0000 7fe58185b740 20 client.26986 trim_cache size 0 max 16384 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:41:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:41:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe55c0a2d70 msgr2=0x7fe55c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe55c0a2d70 0x7fe55c0a5230 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe568098040 tx=0x7fe568003f30 comp rx=0 tx=0).stop 2024-09-05T12:41:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 msgr2=0x7fe57819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe57819a390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe56800ea60 tx=0x7fe56800ef20 comp rx=0 tx=0).stop 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 shutdown_connections 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe57810bf00 0x7fe57819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe55c0a2d70 0x7fe55c0a5230 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe578103400 0x7fe57819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 --2- 172.21.15.71:0/388875216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe578102ae0 0x7fe57819a390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 >> 172.21.15.71:0/388875216 conn(0x7fe5780777c0 msgr2=0x7fe578100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 shutdown_connections 2024-09-05T12:41:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:01.736+0000 7fe57d400640 1 -- 172.21.15.71:0/388875216 wait complete. 2024-09-05T12:41:02.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 425129 2.97 MB/sec execute 598 sec latency 78.849 ms 2024-09-05T12:41:02.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:02 smithi071 ceph-mon[29845]: pgmap v416: 129 pgs: 129 active+clean; 145 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-05T12:41:02.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:02 smithi123 ceph-mon[34321]: pgmap v416: 129 pgs: 129 active+clean; 145 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-05T12:41:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:02 smithi174 ceph-mon[34313]: pgmap v416: 129 pgs: 129 active+clean; 145 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 1.2 MiB/s wr, 211 op/s 2024-09-05T12:41:03.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 426340 2.97 MB/sec execute 599 sec latency 30.713 ms 2024-09-05T12:41:03.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:03 smithi071 ceph-mon[29845]: osdmap e111: 12 total, 12 up, 12 in 2024-09-05T12:41:03.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:03 smithi123 ceph-mon[34321]: osdmap e111: 12 total, 12 up, 12 in 2024-09-05T12:41:03.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:03 smithi174 ceph-mon[34313]: osdmap e111: 12 total, 12 up, 12 in 2024-09-05T12:41:04.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 cleanup 600 sec 2024-09-05T12:41:04.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:04 smithi071 ceph-mon[29845]: pgmap v418: 129 pgs: 129 active+clean; 146 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 167 op/s 2024-09-05T12:41:04.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:04 smithi123 ceph-mon[34321]: pgmap v418: 129 pgs: 129 active+clean; 146 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 167 op/s 2024-09-05T12:41:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:04 smithi174 ceph-mon[34313]: pgmap v418: 129 pgs: 129 active+clean; 146 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 167 op/s 2024-09-05T12:41:05.300 INFO:tasks.workunit.client.0.smithi071.stdout: 1 cleanup 601 sec 2024-09-05T12:41:06.301 INFO:tasks.workunit.client.0.smithi071.stdout: 1 cleanup 602 sec 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: 0 cleanup 602 sec 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: Operation Count AvgLat MaxLat 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: ---------------------------------------- 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: NTCreateX 57410 5.756 33851.357 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: Close 42134 0.005 0.784 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: Rename 2433 45.333 4477.899 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: Unlink 11621 2.519 5097.947 2024-09-05T12:41:06.642 INFO:tasks.workunit.client.0.smithi071.stdout: Qpathinfo 52028 1.386 4273.018 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: Qfileinfo 9057 0.002 0.049 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: Qfsinfo 9538 0.207 6.947 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: Sfileinfo 4684 0.262 525.685 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: Find 20084 0.628 6002.807 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: WriteX 28332 0.252 203.111 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: ReadX 89825 0.201 554.671 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: LockX 186 0.635 2.411 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: UnlockX 186 0.581 1.412 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: Flush 4026 4.555 1298.961 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-05T12:41:06.643 INFO:tasks.workunit.client.0.smithi071.stdout:Throughput 2.97139 MB/sec 1 clients 1 procs max_latency=33851.604 ms 2024-09-05T12:41:06.644 INFO:tasks.workunit.client.0.smithi071.stderr:+ dbench 10 2024-09-05T12:41:06.644 INFO:tasks.workunit.client.0.smithi071.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-05T12:41:06.644 INFO:tasks.workunit.client.0.smithi071.stdout: 2024-09-05T12:41:06.644 INFO:tasks.workunit.client.0.smithi071.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-05T12:41:06.646 INFO:tasks.workunit.client.0.smithi071.stdout:9 of 10 processes prepared for launch 0 sec 2024-09-05T12:41:06.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:06 smithi071 ceph-mon[29845]: pgmap v419: 129 pgs: 129 active+clean; 146 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 167 op/s 2024-09-05T12:41:06.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:06 smithi123 ceph-mon[34321]: pgmap v419: 129 pgs: 129 active+clean; 146 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 167 op/s 2024-09-05T12:41:06.746 INFO:tasks.workunit.client.0.smithi071.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-05T12:41:06.746 INFO:tasks.workunit.client.0.smithi071.stdout:releasing clients 2024-09-05T12:41:06.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:06 smithi174 ceph-mon[34313]: pgmap v419: 129 pgs: 129 active+clean; 146 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 953 KiB/s wr, 167 op/s 2024-09-05T12:41:07.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:07 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:41:07.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:07 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:41:07.747 INFO:tasks.workunit.client.0.smithi071.stdout: 10 448 157.70 MB/sec warmup 1 sec latency 614.475 ms 2024-09-05T12:41:07.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:07 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:41:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:08 smithi071 ceph-mon[29845]: pgmap v420: 129 pgs: 129 active+clean; 144 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 995 KiB/s wr, 191 op/s 2024-09-05T12:41:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:41:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:41:08.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:41:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:08 smithi123 ceph-mon[34321]: pgmap v420: 129 pgs: 129 active+clean; 144 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 995 KiB/s wr, 191 op/s 2024-09-05T12:41:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:41:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:41:08.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:41:08.747 INFO:tasks.workunit.client.0.smithi071.stdout: 10 1070 106.84 MB/sec warmup 2 sec latency 1614.550 ms 2024-09-05T12:41:08.773 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:08 smithi174 ceph-mon[34313]: pgmap v420: 129 pgs: 129 active+clean; 144 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 995 KiB/s wr, 191 op/s 2024-09-05T12:41:08.773 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:41:08.773 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:41:08.774 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:41:09.747 INFO:tasks.workunit.client.0.smithi071.stdout: 10 2066 83.98 MB/sec warmup 3 sec latency 2614.593 ms 2024-09-05T12:41:10.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:10 smithi071 ceph-mon[29845]: pgmap v421: 129 pgs: 129 active+clean; 144 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 995 KiB/s wr, 191 op/s 2024-09-05T12:41:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:10 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:10 smithi071 ceph-mon[29845]: osdmap e112: 12 total, 12 up, 12 in 2024-09-05T12:41:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:10 smithi123 ceph-mon[34321]: pgmap v421: 129 pgs: 129 active+clean; 144 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 995 KiB/s wr, 191 op/s 2024-09-05T12:41:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:10 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:10.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:10 smithi123 ceph-mon[34321]: osdmap e112: 12 total, 12 up, 12 in 2024-09-05T12:41:10.747 INFO:tasks.workunit.client.0.smithi071.stdout: 10 2737 77.35 MB/sec warmup 4 sec latency 3614.634 ms 2024-09-05T12:41:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:10 smithi174 ceph-mon[34313]: pgmap v421: 129 pgs: 129 active+clean; 144 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 995 KiB/s wr, 191 op/s 2024-09-05T12:41:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:10 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:10 smithi174 ceph-mon[34313]: osdmap e112: 12 total, 12 up, 12 in 2024-09-05T12:41:11.747 INFO:tasks.workunit.client.0.smithi071.stdout: 10 3429 72.88 MB/sec warmup 5 sec latency 4614.682 ms 2024-09-05T12:41:12.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:12 smithi071 ceph-mon[29845]: pgmap v423: 129 pgs: 129 active+clean; 156 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2024-09-05T12:41:12.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:12 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:12 smithi123 ceph-mon[34321]: pgmap v423: 129 pgs: 129 active+clean; 156 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2024-09-05T12:41:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:12 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:12.747 INFO:tasks.workunit.client.0.smithi071.stdout: 10 3585 61.18 MB/sec warmup 6 sec latency 5614.754 ms 2024-09-05T12:41:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:12 smithi174 ceph-mon[34313]: pgmap v423: 129 pgs: 129 active+clean; 156 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 276 op/s 2024-09-05T12:41:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:12 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:13.614 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:13 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:13.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:13 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:13.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 3792 52.59 MB/sec warmup 7 sec latency 6616.358 ms 2024-09-05T12:41:13.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:13 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:14 smithi123 ceph-mon[34321]: pgmap v424: 129 pgs: 129 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T12:41:14.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 4241 48.16 MB/sec warmup 8 sec latency 7616.434 ms 2024-09-05T12:41:14.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:14 smithi174 ceph-mon[34313]: pgmap v424: 129 pgs: 129 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T12:41:14.871 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 -- 172.21.15.71:0/2946955321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 msgr2=0x7f6f04104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/2946955321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04104330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8009900 tx=0x7f6ef802a410 comp rx=0 tx=0).stop 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 -- 172.21.15.71:0/2946955321 shutdown_connections 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/2946955321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f041090a0 0x7f6f04110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/2946955321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f04104900 0x7f6f04108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/2946955321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04104330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 -- 172.21.15.71:0/2946955321 >> 172.21.15.71:0/2946955321 conn(0x7f6f040ff9e0 msgr2=0x7f6f04101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 -- 172.21.15.71:0/2946955321 shutdown_connections 2024-09-05T12:41:14.872 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.870+0000 7f6f0bc00640 1 -- 172.21.15.71:0/2946955321 wait complete. 2024-09-05T12:41:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 Processor -- start 2024-09-05T12:41:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 -- start start 2024-09-05T12:41:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04110ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f041090a0 0x7f6f0410db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f0410e090 0x7f6f0410e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041140f0 con 0x7f6f0410e090 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04113f70 con 0x7f6f04103f50 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f0bc00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f04114270 con 0x7f6f041090a0 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f09200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04110ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f09200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04110ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55238/0 (socket says 172.21.15.71:55238) 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.872+0000 7f6f09200640 1 -- 172.21.15.71:0/3857205511 learned_addr learned my addr 172.21.15.71:0/3857205511 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f0410e090 0x7f6f0410e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f041090a0 0x7f6f0410db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09200640 1 -- 172.21.15.71:0/3857205511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f041090a0 msgr2=0x7f6f0410db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09200640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f041090a0 0x7f6f0410db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09200640 1 -- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f0410e090 msgr2=0x7f6f0410e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09200640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f0410e090 0x7f6f0410e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09200640 1 -- 172.21.15.71:0/3857205511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f041ab040 con 0x7f6f04103f50 2024-09-05T12:41:14.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f09200640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04110ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8006010 tx=0x7f6ef8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f01600640 1 -- 172.21.15.71:0/3857205511 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef803f030 con 0x7f6f04103f50 2024-09-05T12:41:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f01600640 1 -- 172.21.15.71:0/3857205511 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ef8004440 con 0x7f6f04103f50 2024-09-05T12:41:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f01600640 1 -- 172.21.15.71:0/3857205511 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef8035070 con 0x7f6f04103f50 2024-09-05T12:41:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.873+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f041ab2d0 con 0x7f6f04103f50 2024-09-05T12:41:14.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.874+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f041ab770 con 0x7f6f04103f50 2024-09-05T12:41:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f01600640 1 -- 172.21.15.71:0/3857205511 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6ef8042030 con 0x7f6f04103f50 2024-09-05T12:41:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 Processor -- start 2024-09-05T12:41:14.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 -- start start 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 0x5582376595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558237659b00 0x558237663e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376643a0 0x558237666790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5582375bf100 con 0x5582376643a0 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582375671b0 con 0x5582376591e0 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5582375bb0c0 con 0x558237659b00 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 0x5582376595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 0x5582376595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55242/0 (socket says 172.21.15.71:55242) 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.877+0000 7f6f09c00640 1 -- 172.21.15.71:0/3922833348 learned_addr learned my addr 172.21.15.71:0/3922833348 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376643a0 0x558237666790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09200640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558237659b00 0x558237663e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09c00640 1 -- 172.21.15.71:0/3922833348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558237659b00 msgr2=0x558237663e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558237659b00 0x558237663e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09c00640 1 -- 172.21.15.71:0/3922833348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376643a0 msgr2=0x558237666790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376643a0 0x558237666790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09c00640 1 -- 172.21.15.71:0/3922833348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582372d2210 con 0x5582376591e0 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09200640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558237659b00 0x558237663e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376643a0 0x558237666790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 0x5582376595c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009f50 tx=0x7f6efc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f1a8b1612fab19 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6ef6000640 1 -- 172.21.15.71:0/3922833348 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc004d20 con 0x5582376591e0 2024-09-05T12:41:14.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6ef6000640 1 -- 172.21.15.71:0/3922833348 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc004050 con 0x5582376591e0 2024-09-05T12:41:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6f01600640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6eec0a2e50 0x7f6eec0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.878+0000 7f6ef6000640 1 -- 172.21.15.71:0/3922833348 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc005280 con 0x5582376591e0 2024-09-05T12:41:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f01600640 1 -- 172.21.15.71:0/3857205511 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7f6ef80efad0 con 0x7f6f04103f50 2024-09-05T12:41:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f09200640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6eec0a2e50 0x7f6eec0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3922833348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 msgr2=0x5582376595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 0x5582376595c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009f50 tx=0x7f6efc0118d0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3922833348 shutdown_connections 2024-09-05T12:41:14.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376643a0 0x558237666790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558237659b00 0x558237663e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3922833348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376591e0 0x5582376595c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3922833348 >> 172.21.15.71:0/3922833348 conn(0x558237655750 msgr2=0x558237655b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f09200640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6eec0a2e50 0x7f6eec0a5310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000c00 tx=0x7f6ef802a410 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3922833348 shutdown_connections 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.879+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3922833348 wait complete. 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 -- start start 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 Processor -- start 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 0x5582376612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 0x558237661810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376643a0 0x558237661d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558237666fd0 con 0x5582376591e0 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558237548090 con 0x558237659b00 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558237667150 con 0x5582376643a0 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f0ea85740 10 client.0 _ll_register_callbacks cb 0x55823759cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.880+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 0x5582376612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 0x558237661810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 0x558237661810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55258/0 (socket says 172.21.15.71:55258) 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 0x5582376612d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58748/0 (socket says 172.21.15.71:58748) 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 -- 172.21.15.71:0/1071338955 learned_addr learned my addr 172.21.15.71:0/1071338955 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376643a0 0x558237661d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 -- 172.21.15.71:0/1071338955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376643a0 msgr2=0x558237661d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376643a0 0x558237661d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 -- 172.21.15.71:0/1071338955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 msgr2=0x5582376612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 0x5582376612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 -- 172.21.15.71:0/1071338955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55823739e620 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 0x5582376612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6f03e00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 0x558237661810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0012730 tx=0x7f6ef0012b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f958a5b2706e0f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6eeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.881+0000 7f6eeaa00640 1 -- 172.21.15.71:0/1071338955 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef0004050 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc004050 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6eeaa00640 1 -- 172.21.15.71:0/1071338955 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7f6ef00041f0 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6eeaa00640 1 -- 172.21.15.71:0/1071338955 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6ef00043f0 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 10 client.26960 fetch_fsmap learned FSMap version 55 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x558237672c60 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6eeaa00640 1 -- 172.21.15.71:0/1071338955 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc004050 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6eeaa00640 1 -- 172.21.15.71:0/1071338955 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 55) ==== 8386+0+0 (secure 0 0 0) 0x7f6ef0003bf0 con 0x558237659b00 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 10 client.26960 fetch_fsmap finished waiting for FSMap version 55 2024-09-05T12:41:14.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 10 client.26960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 20 client.26960 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376740d0 0x5582376764c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 4 client.26960 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.882+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582375cc830 con 0x5582376740d0 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.883+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376740d0 0x5582376764c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.883+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376740d0 0x5582376764c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c7ee28dcaa382e server_cookie=bf551bfddbefe2df in_seq=0 out_seq=0 2024-09-05T12:41:14.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.883+0000 7f6eeaa00640 10 client.26960 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6eeaa00640 1 -- 172.21.15.71:0/1071338955 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582375cc830 con 0x5582376740d0 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6eeaa00640 10 client.26960 handle_command_reply: tid=0 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 client.26960 shutdown 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 msgr2=0x558237661810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 0x558237661810 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6ef0012730 tx=0x7f6ef0012b30 comp rx=0 tx=0).stop 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 shutdown_connections 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376740d0 0x5582376764c0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376643a0 0x558237661d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558237659b00 0x558237661810 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1071338955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582376591e0 0x5582376612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.885+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 >> 172.21.15.71:0/1071338955 conn(0x558237655750 msgr2=0x55823765b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.886+0000 7f6eeaa00640 0 client.26960 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.886+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 shutdown_connections 2024-09-05T12:41:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.886+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1071338955 wait complete. 2024-09-05T12:41:14.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.886+0000 7f6f0ea85740 20 client.26960 trim_cache size 0 max 16384 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.904+0000 7f6f0ea85740 1 Processor -- start 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.904+0000 7f6f0ea85740 1 -- start start 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823765c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823765cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x55823767bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041b5560 con 0x558237659b00 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041b7100 con 0x5582376643a0 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041b5c60 con 0x5582376591e0 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f03e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823765cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x55823767bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x55823767bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55274/0 (socket says 172.21.15.71:55274) 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 -- 172.21.15.71:0/1491192664 learned_addr learned my addr 172.21.15.71:0/1491192664 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823765c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 -- 172.21.15.71:0/1491192664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 msgr2=0x55823765c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823765c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 -- 172.21.15.71:0/1491192664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 msgr2=0x55823765cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823765cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.905+0000 7f6f09c00640 1 -- 172.21.15.71:0/1491192664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558237670230 con 0x5582376643a0 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.906+0000 7f6f09c00640 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x55823767bdd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0095e0 tx=0x7f6efc004700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5285b9a0c8a666c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.906+0000 7f6ef5600640 1 -- 172.21.15.71:0/1491192664 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc003c30 con 0x5582376643a0 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.906+0000 7f6ef5600640 1 -- 172.21.15.71:0/1491192664 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc003dd0 con 0x5582376643a0 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.906+0000 7f6ef5600640 1 -- 172.21.15.71:0/1491192664 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc019690 con 0x5582376643a0 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1491192664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 msgr2=0x55823767bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x55823767bdd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0095e0 tx=0x7f6efc004700 comp rx=0 tx=0).stop 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1491192664 shutdown_connections 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x55823767bdd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823765cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/1491192664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823765c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1491192664 >> 172.21.15.71:0/1491192664 conn(0x558237655750 msgr2=0x55823765a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1491192664 shutdown_connections 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 -- 172.21.15.71:0/1491192664 wait complete. 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.907+0000 7f6f0ea85740 1 -- start start 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 Processor -- start 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823766c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823766ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x558237667920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041b6a30 con 0x558237659b00 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041b6360 con 0x5582376643a0 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f041b3560 con 0x5582376591e0 2024-09-05T12:41:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f0ea85740 10 client.0 _ll_register_callbacks cb 0x7f6ef0009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823766c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f09c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823766c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:41530/0 (socket says 172.21.15.71:41530) 2024-09-05T12:41:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f09c00640 1 -- 172.21.15.71:0/3871822820 learned_addr learned my addr 172.21.15.71:0/3871822820 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.908+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x558237667920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823766ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f09c00640 1 -- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 msgr2=0x558237667920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x558237667920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f09c00640 1 -- 172.21.15.71:0/3871822820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 msgr2=0x55823766ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823766ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f09c00640 1 -- 172.21.15.71:0/3871822820 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558237668150 con 0x5582376591e0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823766ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6f09c00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823766c550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00ffa0 tx=0x7f6efc00fea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec46c32e0258b9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6eeaa00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.909+0000 7f6eeaa00640 1 -- 172.21.15.71:0/3871822820 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc00f680 con 0x5582376591e0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.910+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6efc003dd0 con 0x5582376591e0 2024-09-05T12:41:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.910+0000 7f6eeaa00640 1 -- 172.21.15.71:0/3871822820 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7f6efc00f820 con 0x5582376591e0 2024-09-05T12:41:14.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:14 smithi071 ceph-mon[29845]: pgmap v424: 129 pgs: 129 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.910+0000 7f6eeaa00640 1 -- 172.21.15.71:0/3871822820 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6efc00fa20 con 0x5582376591e0 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.910+0000 7f6f0ea85740 10 client.27004 fetch_fsmap learned FSMap version 55 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.910+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5582376de8e0 con 0x5582376591e0 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.910+0000 7f6eeaa00640 1 -- 172.21.15.71:0/3871822820 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6efc00fbc0 con 0x5582376591e0 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6eeaa00640 1 -- 172.21.15.71:0/3871822820 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 55) ==== 8386+0+0 (secure 0 0 0) 0x7f6efc03c500 con 0x5582376591e0 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f0ea85740 10 client.27004 fetch_fsmap finished waiting for FSMap version 55 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f0ea85740 10 client.27004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f0ea85740 20 client.27004 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376ded10 0x5582376e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f0ea85740 4 client.27004 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:41:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f6efc003dd0 con 0x5582376ded10 2024-09-05T12:41:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.911+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376ded10 0x5582376e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.912+0000 7f6eeaa00640 10 client.27004 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:14.912+0000 7f6f03e00640 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376ded10 0x5582376e1100 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa59ff0b4bc61e7 server_cookie=33f678c02509e019 in_seq=0 out_seq=0 2024-09-05T12:41:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6eeaa00640 1 -- 172.21.15.71:0/3871822820 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f6efc003dd0 con 0x5582376ded10 2024-09-05T12:41:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6eeaa00640 10 client.27004 handle_command_reply: tid=0 2024-09-05T12:41:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 client.27004 shutdown 2024-09-05T12:41:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 msgr2=0x55823766c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823766c550 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00ffa0 tx=0x7f6efc00fea0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 shutdown_connections 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5582376ded10 0x5582376e1100 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582376643a0 0x558237667920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558237659b00 0x55823766ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 --2- 172.21.15.71:0/3871822820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582376591e0 0x55823766c550 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.545+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 >> 172.21.15.71:0/3871822820 conn(0x558237655750 msgr2=0x55823765f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.546+0000 7f6eeaa00640 0 client.27004 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:15.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.546+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 shutdown_connections 2024-09-05T12:41:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.546+0000 7f6f0ea85740 1 -- 172.21.15.71:0/3871822820 wait complete. 2024-09-05T12:41:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.546+0000 7f6f0ea85740 20 client.27004 trim_cache size 0 max 16384 2024-09-05T12:41:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:41:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:41:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:41:15.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6eec0a2e50 msgr2=0x7f6eec0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6eec0a2e50 0x7f6eec0a5310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8000c00 tx=0x7f6ef802a410 comp rx=0 tx=0).stop 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 msgr2=0x7f6f04110ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04110ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8006010 tx=0x7f6ef8004040 comp rx=0 tx=0).stop 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 shutdown_connections 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6eec0a2e50 0x7f6eec0a5310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f0410e090 0x7f6f0410e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f041090a0 0x7f6f0410db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 --2- 172.21.15.71:0/3857205511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f04103f50 0x7f6f04110ab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 >> 172.21.15.71:0/3857205511 conn(0x7f6f040ff9e0 msgr2=0x7f6f04100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 shutdown_connections 2024-09-05T12:41:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:15.548+0000 7f6f0bc00640 1 -- 172.21.15.71:0/3857205511 wait complete. 2024-09-05T12:41:15.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 4643 45.97 MB/sec warmup 9 sec latency 8616.538 ms 2024-09-05T12:41:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:16 smithi123 ceph-mon[34321]: pgmap v425: 129 pgs: 129 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T12:41:16.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 5471 43.75 MB/sec warmup 10 sec latency 9616.578 ms 2024-09-05T12:41:16.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:16 smithi174 ceph-mon[34313]: pgmap v425: 129 pgs: 129 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T12:41:16.851 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:16 smithi071 ceph-mon[29845]: pgmap v425: 129 pgs: 129 active+clean; 158 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-09-05T12:41:17.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6079 44.72 MB/sec warmup 11 sec latency 9752.217 ms 2024-09-05T12:41:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:18 smithi123 ceph-mon[34321]: pgmap v426: 129 pgs: 129 active+clean; 157 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 671 op/s 2024-09-05T12:41:18.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6085 41.00 MB/sec warmup 12 sec latency 1682.366 ms 2024-09-05T12:41:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:18 smithi174 ceph-mon[34313]: pgmap v426: 129 pgs: 129 active+clean; 157 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 671 op/s 2024-09-05T12:41:18.903 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:18 smithi071 ceph-mon[29845]: pgmap v426: 129 pgs: 129 active+clean; 157 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 671 op/s 2024-09-05T12:41:19.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6086 37.85 MB/sec warmup 13 sec latency 2682.467 ms 2024-09-05T12:41:20.603 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:20 smithi071 ceph-mon[29845]: pgmap v427: 129 pgs: 129 active+clean; 157 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 671 op/s 2024-09-05T12:41:20.603 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:20 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:20 smithi123 ceph-mon[34321]: pgmap v427: 129 pgs: 129 active+clean; 157 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 671 op/s 2024-09-05T12:41:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:20 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:20.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6539 37.93 MB/sec warmup 14 sec latency 3682.526 ms 2024-09-05T12:41:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:20 smithi174 ceph-mon[34313]: pgmap v427: 129 pgs: 129 active+clean; 157 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 671 op/s 2024-09-05T12:41:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:20 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:21.749 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6765 36.32 MB/sec warmup 15 sec latency 3753.272 ms 2024-09-05T12:41:22.518 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:22 smithi071 ceph-mon[29845]: pgmap v428: 129 pgs: 129 active+clean; 201 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 738 op/s 2024-09-05T12:41:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:22 smithi123 ceph-mon[34321]: pgmap v428: 129 pgs: 129 active+clean; 201 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 738 op/s 2024-09-05T12:41:22.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6927 34.16 MB/sec warmup 16 sec latency 712.484 ms 2024-09-05T12:41:22.759 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:22 smithi174 ceph-mon[34313]: pgmap v428: 129 pgs: 129 active+clean; 201 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 738 op/s 2024-09-05T12:41:23.580 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:23 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:23.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 6972 32.20 MB/sec warmup 17 sec latency 1712.548 ms 2024-09-05T12:41:23.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:23 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:23.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:23 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:24.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:24 smithi071 ceph-mon[29845]: pgmap v429: 129 pgs: 129 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 571 op/s 2024-09-05T12:41:24.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7174 30.98 MB/sec warmup 18 sec latency 2712.601 ms 2024-09-05T12:41:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:24 smithi174 ceph-mon[34313]: pgmap v429: 129 pgs: 129 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 571 op/s 2024-09-05T12:41:24.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:24 smithi123 ceph-mon[34321]: pgmap v429: 129 pgs: 129 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.4 MiB/s wr, 571 op/s 2024-09-05T12:41:25.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7597 30.72 MB/sec warmup 19 sec latency 3712.644 ms 2024-09-05T12:41:26.641 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:26 smithi071 ceph-mon[29845]: pgmap v430: 129 pgs: 129 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 560 op/s 2024-09-05T12:41:26.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7630 29.25 MB/sec warmup 20 sec latency 4712.693 ms 2024-09-05T12:41:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:26 smithi174 ceph-mon[34313]: pgmap v430: 129 pgs: 129 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 560 op/s 2024-09-05T12:41:26.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:26 smithi123 ceph-mon[34321]: pgmap v430: 129 pgs: 129 active+clean; 219 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.2 MiB/s wr, 560 op/s 2024-09-05T12:41:27.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7630 27.86 MB/sec warmup 21 sec latency 5712.762 ms 2024-09-05T12:41:28.633 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:28 smithi071 ceph-mon[29845]: pgmap v431: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 24 MiB/s wr, 775 op/s 2024-09-05T12:41:28.633 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:28 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 -- 172.21.15.71:0/2259110514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c10e0f0 msgr2=0x7fc73c110540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 --2- 172.21.15.71:0/2259110514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c10e0f0 0x7fc73c110540 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc734009ca0 tx=0x7fc73402a950 comp rx=0 tx=0).stop 2024-09-05T12:41:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 -- 172.21.15.71:0/2259110514 shutdown_connections 2024-09-05T12:41:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 --2- 172.21.15.71:0/2259110514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c10e0f0 0x7fc73c110540 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 --2- 172.21.15.71:0/2259110514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc73c074f40 0x7fc73c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 --2- 172.21.15.71:0/2259110514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074620 0x7fc73c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.666+0000 7fc73b200640 1 -- 172.21.15.71:0/2259110514 >> 172.21.15.71:0/2259110514 conn(0x7fc73c0fd5d0 msgr2=0x7fc73c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.667+0000 7fc73b200640 1 -- 172.21.15.71:0/2259110514 shutdown_connections 2024-09-05T12:41:28.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.667+0000 7fc73b200640 1 -- 172.21.15.71:0/2259110514 wait complete. 2024-09-05T12:41:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.667+0000 7fc73b200640 1 Processor -- start 2024-09-05T12:41:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 -- start start 2024-09-05T12:41:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 0x7fc73c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074f40 0x7fc73c19a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc73c10e0f0 0x7fc73c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c113ef0 con 0x7fc73c074620 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c113d70 con 0x7fc73c074f40 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c114070 con 0x7fc73c10e0f0 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 0x7fc73c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 0x7fc73c19a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44684/0 (socket says 172.21.15.71:44684) 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc739e00640 1 -- 172.21.15.71:0/2682430203 learned_addr learned my addr 172.21.15.71:0/2682430203 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc731400640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074f40 0x7fc73c19a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.668+0000 7fc73a800640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc73c10e0f0 0x7fc73c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc731400640 1 -- 172.21.15.71:0/2682430203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc73c10e0f0 msgr2=0x7fc73c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc731400640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc73c10e0f0 0x7fc73c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc731400640 1 -- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 msgr2=0x7fc73c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc731400640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 0x7fc73c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc731400640 1 -- 172.21.15.71:0/2682430203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc73c19f3b0 con 0x7fc73c074f40 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc739e00640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 0x7fc73c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc731400640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074f40 0x7fc73c19a900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d720 tx=0x7fc72400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc733400640 1 -- 172.21.15.71:0/2682430203 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc724017070 con 0x7fc73c074f40 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc733400640 1 -- 172.21.15.71:0/2682430203 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72400de30 con 0x7fc73c074f40 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc733400640 1 -- 172.21.15.71:0/2682430203 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7240047a0 con 0x7fc73c074f40 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc73c19f6a0 con 0x7fc73c074f40 2024-09-05T12:41:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.669+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc73c1a6f30 con 0x7fc73c074f40 2024-09-05T12:41:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.672+0000 7fc743bbf740 1 Processor -- start 2024-09-05T12:41:28.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.672+0000 7fc743bbf740 1 -- start start 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 0x55a7434e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc743bbf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a74344f100 con 0x55a7434e9cd0 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc743bbf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7433f71b0 con 0x55a7434e93b0 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc743bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a74344b0c0 con 0x55a7434f4570 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 0x55a7434e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 0x55a7434e9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:44458/0 (socket says 172.21.15.71:44458) 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 -- 172.21.15.71:0/3162392323 learned_addr learned my addr 172.21.15.71:0/3162392323 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc731400640 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 -- 172.21.15.71:0/3162392323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 msgr2=0x55a7434f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 -- 172.21.15.71:0/3162392323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 msgr2=0x55a7434f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc73a800640 1 -- 172.21.15.71:0/3162392323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a743162210 con 0x55a7434e93b0 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.673+0000 7fc739e00640 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.674+0000 7fc73a800640 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 0x55a7434e9790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc734009ca0 tx=0x7fc73402f450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7874710869d852 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.674+0000 7fc72b400640 1 -- 172.21.15.71:0/3162392323 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734048070 con 0x55a7434e93b0 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.674+0000 7fc72b400640 1 -- 172.21.15.71:0/3162392323 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc734003e50 con 0x55a7434e93b0 2024-09-05T12:41:28.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.674+0000 7fc72b400640 1 -- 172.21.15.71:0/3162392323 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734034040 con 0x55a7434e93b0 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 -- 172.21.15.71:0/3162392323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 msgr2=0x55a7434e9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 0x55a7434e9790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc734009ca0 tx=0x7fc73402f450 comp rx=0 tx=0).stop 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 -- 172.21.15.71:0/3162392323 shutdown_connections 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3162392323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e93b0 0x55a7434e9790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 -- 172.21.15.71:0/3162392323 >> 172.21.15.71:0/3162392323 conn(0x55a7434e5920 msgr2=0x55a7434e5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 -- 172.21.15.71:0/3162392323 shutdown_connections 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 -- 172.21.15.71:0/3162392323 wait complete. 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.675+0000 7fc743bbf740 1 -- start start 2024-09-05T12:41:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 Processor -- start 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434e93b0 0x55a7434f1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434f4570 0x55a7434f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a7434f6ff0 con 0x55a7434e9cd0 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7433d8090 con 0x55a7434f4570 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a7434f7170 con 0x55a7434e93b0 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc743bbf740 10 client.0 _ll_register_callbacks cb 0x55a74342cf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44698/0 (socket says 172.21.15.71:44698) 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc739e00640 1 -- 172.21.15.71:0/3995819868 learned_addr learned my addr 172.21.15.71:0/3995819868 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc73a800640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434e93b0 0x55a7434f1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.676+0000 7fc731400640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434f4570 0x55a7434f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc739e00640 1 -- 172.21.15.71:0/3995819868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434e93b0 msgr2=0x55a7434f1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc739e00640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434e93b0 0x55a7434f1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc739e00640 1 -- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434f4570 msgr2=0x55a7434f20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc739e00640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434f4570 0x55a7434f20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc739e00640 1 -- 172.21.15.71:0/3995819868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74322e620 con 0x55a7434e9cd0 2024-09-05T12:41:28.681 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc731400640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434f4570 0x55a7434f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc739e00640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f1b70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc72c012d20 tx=0x7fc72c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6841f633e40f7a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc71fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc71fe00640 1 -- 172.21.15.71:0/3995819868 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c010430 con 0x55a7434e9cd0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc734003e50 con 0x55a7434e9cd0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.677+0000 7fc71fe00640 1 -- 172.21.15.71:0/3995819868 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fc72c0105d0 con 0x55a7434e9cd0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc71fe00640 1 -- 172.21.15.71:0/3995819868 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc72c010810 con 0x55a7434e9cd0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 10 client.16959 fetch_fsmap learned FSMap version 58 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a743502e50 con 0x55a7434e9cd0 2024-09-05T12:41:28.684 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc71fe00640 1 -- 172.21.15.71:0/3995819868 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc734003e50 con 0x55a7434e9cd0 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc71fe00640 1 -- 172.21.15.71:0/3995819868 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7fc72c01c910 con 0x55a7434e9cd0 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 10 client.16959 fetch_fsmap finished waiting for FSMap version 58 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 20 client.16959 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a743504380 0x55a743506790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 4 client.16959 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.678+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74345c830 con 0x55a743504380 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.679+0000 7fc73a800640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a743504380 0x55a743506790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.679+0000 7fc73a800640 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a743504380 0x55a743506790 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56a1c5e3d345cb0 server_cookie=3e4872b413a448dd in_seq=0 out_seq=0 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.679+0000 7fc71fe00640 10 client.16959 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc71fe00640 1 -- 172.21.15.71:0/3995819868 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a74345c830 con 0x55a743504380 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc71fe00640 10 client.16959 handle_command_reply: tid=0 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc743bbf740 1 client.16959 shutdown 2024-09-05T12:41:28.685 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 msgr2=0x55a7434f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f1b70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc72c012d20 tx=0x7fc72c00f790 comp rx=0 tx=0).stop 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 shutdown_connections 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.680+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a743504380 0x55a743506790 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434f4570 0x55a7434f20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e9cd0 0x55a7434f1b70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3995819868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434e93b0 0x55a7434f1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 >> 172.21.15.71:0/3995819868 conn(0x55a7434e5920 msgr2=0x55a7434eb680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc71fe00640 0 client.16959 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc733400640 1 -- 172.21.15.71:0/2682430203 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc7240099d0 con 0x7fc73c074f40 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 shutdown_connections 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 1 -- 172.21.15.71:0/3995819868 wait complete. 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.681+0000 7fc743bbf740 20 client.16959 trim_cache size 0 max 16384 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.682+0000 7fc733400640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7200a2bc0 0x7fc7200a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.686 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.683+0000 7fc733400640 1 -- 172.21.15.71:0/2682430203 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fc7240cf600 con 0x7fc73c074f40 2024-09-05T12:41:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.683+0000 7fc739e00640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7200a2bc0 0x7fc7200a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.687 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.683+0000 7fc739e00640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7200a2bc0 0x7fc7200a5080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001eb0 tx=0x7fc72c000bc0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.708+0000 7fc743bbf740 1 Processor -- start 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 -- start start 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434eb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a74350c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b10f0 con 0x55a7434e93b0 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b2cf0 con 0x55a7434e9cd0 2024-09-05T12:41:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc743bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b17f0 con 0x55a7434f4570 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44708/0 (socket says 172.21.15.71:44708) 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc739e00640 1 -- 172.21.15.71:0/2657300399 learned_addr learned my addr 172.21.15.71:0/2657300399 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc731400640 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434eb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.709+0000 7fc73a800640 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a74350c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc739e00640 1 -- 172.21.15.71:0/2657300399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 msgr2=0x55a74350c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc739e00640 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a74350c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc739e00640 1 -- 172.21.15.71:0/2657300399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 msgr2=0x55a7434eb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc739e00640 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434eb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc739e00640 1 -- 172.21.15.71:0/2657300399 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7434e0c40 con 0x55a7434e93b0 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc731400640 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc739e00640 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f1ee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a7434f2c70 tx=0x7fc72c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14dc6b38d2d140a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc72aa00640 1 -- 172.21.15.71:0/2657300399 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c010e40 con 0x55a7434e93b0 2024-09-05T12:41:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc72aa00640 1 -- 172.21.15.71:0/2657300399 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c0104e0 con 0x55a7434e93b0 2024-09-05T12:41:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.710+0000 7fc72aa00640 1 -- 172.21.15.71:0/2657300399 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c003770 con 0x55a7434e93b0 2024-09-05T12:41:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 -- 172.21.15.71:0/2657300399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 msgr2=0x55a7434f1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f1ee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55a7434f2c70 tx=0x7fc72c014000 comp rx=0 tx=0).stop 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 -- 172.21.15.71:0/2657300399 shutdown_connections 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a74350c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434eb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 --2- 172.21.15.71:0/2657300399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f1ee0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 -- 172.21.15.71:0/2657300399 >> 172.21.15.71:0/2657300399 conn(0x55a7434e5920 msgr2=0x55a7434eae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 -- 172.21.15.71:0/2657300399 shutdown_connections 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 -- 172.21.15.71:0/2657300399 wait complete. 2024-09-05T12:41:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.711+0000 7fc743bbf740 1 -- start start 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 Processor -- start 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b2620 con 0x55a7434e93b0 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1b1ef0 con 0x55a7434e9cd0 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc73c1af0f0 con 0x55a7434f4570 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc743bbf740 10 client.0 _ll_register_callbacks cb 0x7fc72c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc739e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44718/0 (socket says 172.21.15.71:44718) 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc739e00640 1 -- 172.21.15.71:0/3911223822 learned_addr learned my addr 172.21.15.71:0/3911223822 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.712+0000 7fc731400640 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc739e00640 1 -- 172.21.15.71:0/3911223822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 msgr2=0x55a7434f8c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:41:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc739e00640 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f8c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc739e00640 1 -- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 msgr2=0x55a7434f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc739e00640 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc739e00640 1 -- 172.21.15.71:0/3911223822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74356e120 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc739e00640 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f81b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc72c01cbc0 tx=0x7fc72c01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5928ba149cffeb20 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc71fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc71fe00640 1 -- 172.21.15.71:0/3911223822 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc72c042060 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc72c0104e0 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc71fe00640 1 -- 172.21.15.71:0/3911223822 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fc72c032040 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc71fe00640 1 -- 172.21.15.71:0/3911223822 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc72c02e020 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc743bbf740 10 client.16965 fetch_fsmap learned FSMap version 58 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.713+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a74356f410 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc71fe00640 1 -- 172.21.15.71:0/3911223822 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc72c0104e0 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc71fe00640 1 -- 172.21.15.71:0/3911223822 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7fc72c02a510 con 0x55a7434e93b0 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc743bbf740 10 client.16965 fetch_fsmap finished waiting for FSMap version 58 2024-09-05T12:41:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc743bbf740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc743bbf740 20 client.16965 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a74356f950 0x55a743571d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc743bbf740 4 client.16965 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a743456b80 con 0x55a74356f950 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.714+0000 7fc731400640 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a74356f950 0x55a743571d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.715+0000 7fc731400640 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a74356f950 0x55a743571d70 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f532944dc2a07cb server_cookie=7749638d8929f201 in_seq=0 out_seq=0 2024-09-05T12:41:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.715+0000 7fc71fe00640 10 client.16965 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:28.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7630 26.59 MB/sec warmup 22 sec latency 6712.806 ms 2024-09-05T12:41:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:28 smithi174 ceph-mon[34313]: pgmap v431: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 24 MiB/s wr, 775 op/s 2024-09-05T12:41:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:28 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.889+0000 7fc71fe00640 1 -- 172.21.15.71:0/3911223822 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a743456b80 con 0x55a74356f950 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.889+0000 7fc71fe00640 10 client.16965 handle_command_reply: tid=0 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.889+0000 7fc743bbf740 1 client.16965 shutdown 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 msgr2=0x55a7434f81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f81b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc72c01cbc0 tx=0x7fc72c01c650 comp rx=0 tx=0).stop 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 shutdown_connections 2024-09-05T12:41:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a74356f950 0x55a743571d70 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7434f4570 0x55a7434f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7434e9cd0 0x55a7434f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 --2- 172.21.15.71:0/3911223822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7434e93b0 0x55a7434f81b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 >> 172.21.15.71:0/3911223822 conn(0x55a7434e5920 msgr2=0x55a7434f34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc71fe00640 0 client.16965 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 shutdown_connections 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 1 -- 172.21.15.71:0/3911223822 wait complete. 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.890+0000 7fc743bbf740 20 client.16965 trim_cache size 0 max 16384 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7200a2bc0 msgr2=0x7fc7200a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7200a2bc0 0x7fc7200a5080 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc72c001eb0 tx=0x7fc72c000bc0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074f40 msgr2=0x7fc73c19a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074f40 0x7fc73c19a900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc72400d720 tx=0x7fc72400dbe0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 shutdown_connections 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc73c10e0f0 0x7fc73c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc73c074f40 0x7fc73c19a900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7200a2bc0 0x7fc7200a5080 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 --2- 172.21.15.71:0/2682430203 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc73c074620 0x7fc73c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 >> 172.21.15.71:0/2682430203 conn(0x7fc73c0fd5d0 msgr2=0x7fc73c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 shutdown_connections 2024-09-05T12:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:28.891+0000 7fc73b200640 1 -- 172.21.15.71:0/2682430203 wait complete. 2024-09-05T12:41:28.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:28 smithi123 ceph-mon[34321]: pgmap v431: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 24 MiB/s wr, 775 op/s 2024-09-05T12:41:28.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:28 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:29.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7631 25.44 MB/sec warmup 23 sec latency 7712.849 ms 2024-09-05T12:41:30.641 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:30 smithi071 ceph-mon[29845]: pgmap v432: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 22 MiB/s wr, 328 op/s 2024-09-05T12:41:30.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7631 24.38 MB/sec warmup 24 sec latency 8712.890 ms 2024-09-05T12:41:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:30 smithi174 ceph-mon[34313]: pgmap v432: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 22 MiB/s wr, 328 op/s 2024-09-05T12:41:30.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:30 smithi123 ceph-mon[34321]: pgmap v432: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 22 MiB/s wr, 328 op/s 2024-09-05T12:41:31.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 7648 23.41 MB/sec warmup 25 sec latency 9658.671 ms 2024-09-05T12:41:32.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 8513 24.09 MB/sec warmup 26 sec latency 147.616 ms 2024-09-05T12:41:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:32 smithi174 ceph-mon[34313]: pgmap v433: 129 pgs: 129 active+clean; 353 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 22 MiB/s wr, 464 op/s 2024-09-05T12:41:32.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:32 smithi071 ceph-mon[29845]: pgmap v433: 129 pgs: 129 active+clean; 353 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 22 MiB/s wr, 464 op/s 2024-09-05T12:41:32.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:32 smithi123 ceph-mon[34321]: pgmap v433: 129 pgs: 129 active+clean; 353 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 22 MiB/s wr, 464 op/s 2024-09-05T12:41:33.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 10004 26.29 MB/sec warmup 27 sec latency 47.261 ms 2024-09-05T12:41:33.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:33 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:33.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:33 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:34.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:33 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:34.750 INFO:tasks.workunit.client.0.smithi071.stdout: 10 11075 27.72 MB/sec warmup 28 sec latency 48.867 ms 2024-09-05T12:41:34.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:34 smithi071 ceph-mon[29845]: pgmap v434: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 18 MiB/s wr, 426 op/s 2024-09-05T12:41:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:34 smithi123 ceph-mon[34321]: pgmap v434: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 18 MiB/s wr, 426 op/s 2024-09-05T12:41:35.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:34 smithi174 ceph-mon[34313]: pgmap v434: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 18 MiB/s wr, 426 op/s 2024-09-05T12:41:35.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 11928 27.99 MB/sec warmup 29 sec latency 88.490 ms 2024-09-05T12:41:36.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 13319 29.72 MB/sec warmup 30 sec latency 35.905 ms 2024-09-05T12:41:36.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:36 smithi071 ceph-mon[29845]: pgmap v435: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 17 MiB/s wr, 400 op/s 2024-09-05T12:41:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:36 smithi123 ceph-mon[34321]: pgmap v435: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 17 MiB/s wr, 400 op/s 2024-09-05T12:41:37.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:36 smithi174 ceph-mon[34313]: pgmap v435: 129 pgs: 129 active+clean; 340 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 17 MiB/s wr, 400 op/s 2024-09-05T12:41:37.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 14501 31.07 MB/sec warmup 31 sec latency 66.126 ms 2024-09-05T12:41:38.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 15466 31.42 MB/sec warmup 32 sec latency 42.416 ms 2024-09-05T12:41:38.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:38 smithi071 ceph-mon[29845]: pgmap v436: 129 pgs: 129 active+clean; 357 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 19 MiB/s wr, 646 op/s 2024-09-05T12:41:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:38 smithi123 ceph-mon[34321]: pgmap v436: 129 pgs: 129 active+clean; 357 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 19 MiB/s wr, 646 op/s 2024-09-05T12:41:39.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:38 smithi174 ceph-mon[34313]: pgmap v436: 129 pgs: 129 active+clean; 357 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 19 MiB/s wr, 646 op/s 2024-09-05T12:41:39.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 16622 32.59 MB/sec warmup 33 sec latency 80.231 ms 2024-09-05T12:41:40.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 17663 32.91 MB/sec warmup 34 sec latency 48.556 ms 2024-09-05T12:41:40.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:40 smithi071 ceph-mon[29845]: pgmap v437: 129 pgs: 129 active+clean; 357 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 430 op/s 2024-09-05T12:41:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:40 smithi123 ceph-mon[34321]: pgmap v437: 129 pgs: 129 active+clean; 357 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 430 op/s 2024-09-05T12:41:41.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:40 smithi174 ceph-mon[34313]: pgmap v437: 129 pgs: 129 active+clean; 357 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.6 MiB/s wr, 430 op/s 2024-09-05T12:41:41.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 18910 33.91 MB/sec warmup 35 sec latency 150.537 ms 2024-09-05T12:41:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 -- 172.21.15.71:0/1444825058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 msgr2=0x7fe3a8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 --2- 172.21.15.71:0/1444825058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a8100cb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe398009930 tx=0x7fe39802a4d0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 -- 172.21.15.71:0/1444825058 shutdown_connections 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 --2- 172.21.15.71:0/1444825058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a8105d00 0x7fe3a8106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 --2- 172.21.15.71:0/1444825058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a81011f0 0x7fe3a81056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 --2- 172.21.15.71:0/1444825058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a8100cb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.059+0000 7fe3aec00640 1 -- 172.21.15.71:0/1444825058 >> 172.21.15.71:0/1444825058 conn(0x7fe3a80777c0 msgr2=0x7fe3a80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.060+0000 7fe3aec00640 1 -- 172.21.15.71:0/1444825058 shutdown_connections 2024-09-05T12:41:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.060+0000 7fe3aec00640 1 -- 172.21.15.71:0/1444825058 wait complete. 2024-09-05T12:41:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 Processor -- start 2024-09-05T12:41:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 -- start start 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a81a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a81011f0 0x7fe3a81a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a8105d00 0x7fe3a819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a8118320 con 0x7fe3a81011f0 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81181a0 con 0x7fe3a81008d0 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.062+0000 7fe3aec00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81184a0 con 0x7fe3a8105d00 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a81a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a81a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45910/0 (socket says 172.21.15.71:45910) 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3575774320 learned_addr learned my addr 172.21.15.71:0/3575774320 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3acc00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a8105d00 0x7fe3a819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7400640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a81011f0 0x7fe3a81a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3575774320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a8105d00 msgr2=0x7fe3a819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a8105d00 0x7fe3a819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a81011f0 msgr2=0x7fe3a81a4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a81011f0 0x7fe3a81a4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3575774320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a819ef60 con 0x7fe3a81008d0 2024-09-05T12:41:42.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.063+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a81a4620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe39802a9f0 tx=0x7fe398003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.064+0000 7fe3a4c00640 1 -- 172.21.15.71:0/3575774320 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39803f070 con 0x7fe3a81008d0 2024-09-05T12:41:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.064+0000 7fe3a4c00640 1 -- 172.21.15.71:0/3575774320 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe398031c90 con 0x7fe3a81008d0 2024-09-05T12:41:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.064+0000 7fe3a4c00640 1 -- 172.21.15.71:0/3575774320 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe398035030 con 0x7fe3a81008d0 2024-09-05T12:41:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.064+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a819f1f0 con 0x7fe3a81008d0 2024-09-05T12:41:42.066 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.064+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a81ab470 con 0x7fe3a81008d0 2024-09-05T12:41:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.068+0000 7fe3b1c46740 1 Processor -- start 2024-09-05T12:41:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 -- start start 2024-09-05T12:41:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3a4c00640 1 -- 172.21.15.71:0/3575774320 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe398042070 con 0x7fe3a81008d0 2024-09-05T12:41:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb073b0 0x562adcb07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb12030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb14960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562adca6d100 con 0x562adcb07cd0 2024-09-05T12:41:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562adca151b0 con 0x562adcb12570 2024-09-05T12:41:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.069+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562adca690c0 con 0x562adcb073b0 2024-09-05T12:41:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.070+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb12030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.070+0000 7fe3acc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb073b0 0x562adcb07790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.070+0000 7fe3acc00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb073b0 0x562adcb07790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48548/0 (socket says 172.21.15.71:48548) 2024-09-05T12:41:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.070+0000 7fe3acc00640 1 -- 172.21.15.71:0/465002889 learned_addr learned my addr 172.21.15.71:0/465002889 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.070+0000 7fe3a7400640 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb14960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7400640 1 -- 172.21.15.71:0/465002889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb073b0 msgr2=0x562adcb07790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7400640 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb073b0 0x562adcb07790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7400640 1 -- 172.21.15.71:0/465002889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 msgr2=0x562adcb12030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7400640 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb12030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7400640 1 -- 172.21.15.71:0/465002889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adc780210 con 0x562adcb12570 2024-09-05T12:41:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb12030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.071+0000 7fe3a7400640 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb14960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3a819fae0 tx=0x7fe394011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19cd8ae0a37ba429 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.072+0000 7fe39d200640 1 -- 172.21.15.71:0/465002889 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394009d70 con 0x562adcb12570 2024-09-05T12:41:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.072+0000 7fe39d200640 1 -- 172.21.15.71:0/465002889 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe394005020 con 0x562adcb12570 2024-09-05T12:41:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.072+0000 7fe39d200640 1 -- 172.21.15.71:0/465002889 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe394004c00 con 0x562adcb12570 2024-09-05T12:41:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.074+0000 7fe3a4c00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe38c0a2cf0 0x7fe38c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.075+0000 7fe3acc00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe38c0a2cf0 0x7fe38c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.075+0000 7fe3a4c00640 1 -- 172.21.15.71:0/3575774320 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe3980f00f0 con 0x7fe3a81008d0 2024-09-05T12:41:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.075+0000 7fe3b1c46740 1 -- 172.21.15.71:0/465002889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 msgr2=0x562adcb14960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.075+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb14960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe3a819fae0 tx=0x7fe394011810 comp rx=0 tx=0).stop 2024-09-05T12:41:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.076+0000 7fe3b1c46740 1 -- 172.21.15.71:0/465002889 shutdown_connections 2024-09-05T12:41:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.076+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb14960 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.076+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb12030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.076+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/465002889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb073b0 0x562adcb07790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.076+0000 7fe3b1c46740 1 -- 172.21.15.71:0/465002889 >> 172.21.15.71:0/465002889 conn(0x562adcb03920 msgr2=0x562adcb03d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.077+0000 7fe3acc00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe38c0a2cf0 0x7fe38c0a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0012560 tx=0x7fe3a0009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.078+0000 7fe3b1c46740 1 -- 172.21.15.71:0/465002889 shutdown_connections 2024-09-05T12:41:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.078+0000 7fe3b1c46740 1 -- 172.21.15.71:0/465002889 wait complete. 2024-09-05T12:41:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.079+0000 7fe3b1c46740 1 -- start start 2024-09-05T12:41:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 Processor -- start 2024-09-05T12:41:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 0x562adcb0f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb0fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb12570 0x562adcb0fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562adcb15050 con 0x562adcb07cd0 2024-09-05T12:41:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562adc9f6090 con 0x562adcb073b0 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562adcb151d0 con 0x562adcb12570 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3b1c46740 10 client.0 _ll_register_callbacks cb 0x562adca4af30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 0x562adcb0f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.080+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 0x562adcb0f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45932/0 (socket says 172.21.15.71:45932) 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7e00640 1 -- 172.21.15.71:0/972397197 learned_addr learned my addr 172.21.15.71:0/972397197 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3acc00640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb12570 0x562adcb0fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7400640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb0fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7e00640 1 -- 172.21.15.71:0/972397197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb12570 msgr2=0x562adcb0fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb12570 0x562adcb0fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7e00640 1 -- 172.21.15.71:0/972397197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 msgr2=0x562adcb0fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb0fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7e00640 1 -- 172.21.15.71:0/972397197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adc84c620 con 0x562adcb073b0 2024-09-05T12:41:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3acc00640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb12570 0x562adcb0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.081+0000 7fe3a7400640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb0fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 0x562adcb0f570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe3980b86e0 tx=0x7fe398031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e7f42944fdc938b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe391600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe391600640 1 -- 172.21.15.71:0/972397197 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980bbbe0 con 0x562adcb073b0 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe394005020 con 0x562adcb073b0 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe391600640 1 -- 172.21.15.71:0/972397197 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe3980ed730 con 0x562adcb073b0 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe391600640 1 -- 172.21.15.71:0/972397197 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fe398005d50 con 0x562adcb073b0 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe3b1c46740 10 client.26996 fetch_fsmap learned FSMap version 60 2024-09-05T12:41:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.082+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x562adcb20ec0 con 0x562adcb073b0 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.083+0000 7fe391600640 1 -- 172.21.15.71:0/972397197 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe394005020 con 0x562adcb073b0 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe391600640 1 -- 172.21.15.71:0/972397197 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 60) ==== 8418+0+0 (secure 0 0 0) 0x7fe3980b88f0 con 0x562adcb073b0 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3b1c46740 10 client.26996 fetch_fsmap finished waiting for FSMap version 60 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3b1c46740 10 client.26996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3b1c46740 20 client.26996 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb22420 0x562adcb24880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3b1c46740 4 client.26996 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562adca7a830 con 0x562adcb22420 2024-09-05T12:41:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.084+0000 7fe3a7400640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb22420 0x562adcb24880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.085+0000 7fe3a7400640 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb22420 0x562adcb24880 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633bea0058115208 server_cookie=b292802215ed3430 in_seq=0 out_seq=0 2024-09-05T12:41:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.085+0000 7fe391600640 10 client.26996 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.087+0000 7fe391600640 1 -- 172.21.15.71:0/972397197 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562adca7a830 con 0x562adcb22420 2024-09-05T12:41:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.087+0000 7fe391600640 10 client.26996 handle_command_reply: tid=0 2024-09-05T12:41:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.087+0000 7fe3b1c46740 1 client.26996 shutdown 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 msgr2=0x562adcb0f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 0x562adcb0f570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe3980b86e0 tx=0x7fe398031490 comp rx=0 tx=0).stop 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 shutdown_connections 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb22420 0x562adcb24880 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb12570 0x562adcb0fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb07cd0 0x562adcb0fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/972397197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb073b0 0x562adcb0f570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 >> 172.21.15.71:0/972397197 conn(0x562adcb03920 msgr2=0x562adcb09680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.088+0000 7fe391600640 0 client.26996 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.089+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 shutdown_connections 2024-09-05T12:41:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.089+0000 7fe3b1c46740 1 -- 172.21.15.71:0/972397197 wait complete. 2024-09-05T12:41:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.089+0000 7fe3b1c46740 20 client.26996 trim_cache size 0 max 16384 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 Processor -- start 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 -- start start 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb07cd0 0x562adcb11cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb09630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb2a190 0x562adcb2a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b54f0 con 0x562adcb2a190 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b7150 con 0x562adcb12570 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b5bf0 con 0x562adcb07cd0 2024-09-05T12:41:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3acc00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb2a190 0x562adcb2a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb09630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb07cd0 0x562adcb11cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3a7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb09630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45938/0 (socket says 172.21.15.71:45938) 2024-09-05T12:41:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.116+0000 7fe3a7400640 1 -- 172.21.15.71:0/3880718270 learned_addr learned my addr 172.21.15.71:0/3880718270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3880718270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 msgr2=0x562adcb09630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb09630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3880718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb2a190 msgr2=0x562adcb2a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb2a190 0x562adcb2a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7e00640 1 -- 172.21.15.71:0/3880718270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562adcb0e500 con 0x562adcb07cd0 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7400640 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb09630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3acc00640 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb2a190 0x562adcb2a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb07cd0 0x562adcb11cd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe3980b8c50 tx=0x7fe398003f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f085d25701551b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.117+0000 7fe393e00640 1 -- 172.21.15.71:0/3880718270 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980bc980 con 0x562adcb07cd0 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.118+0000 7fe393e00640 1 -- 172.21.15.71:0/3880718270 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3980c4ef0 con 0x562adcb07cd0 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.118+0000 7fe393e00640 1 -- 172.21.15.71:0/3880718270 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980ed730 con 0x562adcb07cd0 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.118+0000 7fe3b1c46740 1 -- 172.21.15.71:0/3880718270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb07cd0 msgr2=0x562adcb11cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.118+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb07cd0 0x562adcb11cd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe3980b8c50 tx=0x7fe398003f50 comp rx=0 tx=0).stop 2024-09-05T12:41:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 -- 172.21.15.71:0/3880718270 shutdown_connections 2024-09-05T12:41:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb2a190 0x562adcb2a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb12570 0x562adcb09630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/3880718270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb07cd0 0x562adcb11cd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 -- 172.21.15.71:0/3880718270 >> 172.21.15.71:0/3880718270 conn(0x562adcb03920 msgr2=0x562adcafe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 -- 172.21.15.71:0/3880718270 shutdown_connections 2024-09-05T12:41:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.119+0000 7fe3b1c46740 1 -- 172.21.15.71:0/3880718270 wait complete. 2024-09-05T12:41:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.120+0000 7fe3b1c46740 1 -- start start 2024-09-05T12:41:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.120+0000 7fe3b1c46740 1 Processor -- start 2024-09-05T12:41:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 0x562adcb161e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb12570 0x562adcb16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 0x562adcb161e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb2a190 0x562adcb16c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3a7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 0x562adcb161e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45940/0 (socket says 172.21.15.71:45940) 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b6a80 con 0x562adcb12570 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b6320 con 0x562adcb07cd0 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a81b34f0 con 0x562adcb2a190 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3b1c46740 10 client.0 _ll_register_callbacks cb 0x7fe398009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3a7e00640 1 -- 172.21.15.71:0/1565688803 learned_addr learned my addr 172.21.15.71:0/1565688803 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.121+0000 7fe3acc00640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb2a190 0x562adcb16c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7400640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb12570 0x562adcb16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7e00640 1 -- 172.21.15.71:0/1565688803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb2a190 msgr2=0x562adcb16c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb2a190 0x562adcb16c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7e00640 1 -- 172.21.15.71:0/1565688803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb12570 msgr2=0x562adcb16720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb12570 0x562adcb16720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7e00640 1 -- 172.21.15.71:0/1565688803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562adcb8c160 con 0x562adcb07cd0 2024-09-05T12:41:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe3a7e00640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 0x562adcb161e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe3980c49c0 tx=0x7fe39803b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e8444a5815ef67b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe391600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:41:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.122+0000 7fe391600640 1 -- 172.21.15.71:0/1565688803 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3980c4c60 con 0x562adcb07cd0 2024-09-05T12:41:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.123+0000 7fe391600640 1 -- 172.21.15.71:0/1565688803 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe3980dc0b0 con 0x562adcb07cd0 2024-09-05T12:41:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.123+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3980c4ef0 con 0x562adcb07cd0 2024-09-05T12:41:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.123+0000 7fe391600640 1 -- 172.21.15.71:0/1565688803 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fe3980bc280 con 0x562adcb07cd0 2024-09-05T12:41:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.123+0000 7fe3b1c46740 10 client.27002 fetch_fsmap learned FSMap version 60 2024-09-05T12:41:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.123+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x562adcb8c160 con 0x562adcb07cd0 2024-09-05T12:41:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.123+0000 7fe391600640 1 -- 172.21.15.71:0/1565688803 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3980c4ef0 con 0x562adcb07cd0 2024-09-05T12:41:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe391600640 1 -- 172.21.15.71:0/1565688803 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 60) ==== 8418+0+0 (secure 0 0 0) 0x7fe3980870d0 con 0x562adcb07cd0 2024-09-05T12:41:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe3b1c46740 10 client.27002 fetch_fsmap finished waiting for FSMap version 60 2024-09-05T12:41:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe3b1c46740 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe3b1c46740 20 client.27002 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb8d5a0 0x562adcb8f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe3b1c46740 4 client.27002 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:41:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.124+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562adca74b80 con 0x562adcb8d5a0 2024-09-05T12:41:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.125+0000 7fe3a7400640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb8d5a0 0x562adcb8f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.126+0000 7fe3a7400640 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb8d5a0 0x562adcb8f9c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3de5e8ba8ab6c1a server_cookie=ae6046e52c1e6d40 in_seq=0 out_seq=0 2024-09-05T12:41:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.126+0000 7fe391600640 10 client.27002 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.582+0000 7fe391600640 1 -- 172.21.15.71:0/1565688803 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562adca74b80 con 0x562adcb8d5a0 2024-09-05T12:41:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.582+0000 7fe391600640 10 client.27002 handle_command_reply: tid=0 2024-09-05T12:41:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.582+0000 7fe3b1c46740 1 client.27002 shutdown 2024-09-05T12:41:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 msgr2=0x562adcb161e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 0x562adcb161e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe3980c49c0 tx=0x7fe39803b690 comp rx=0 tx=0).stop 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 shutdown_connections 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x562adcb8d5a0 0x562adcb8f9c0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562adcb2a190 0x562adcb16c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562adcb12570 0x562adcb16720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 --2- 172.21.15.71:0/1565688803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562adcb07cd0 0x562adcb161e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 >> 172.21.15.71:0/1565688803 conn(0x562adcb03920 msgr2=0x562adcb021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe391600640 0 client.27002 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.583+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 shutdown_connections 2024-09-05T12:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.584+0000 7fe3b1c46740 1 -- 172.21.15.71:0/1565688803 wait complete. 2024-09-05T12:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.584+0000 7fe3b1c46740 20 client.27002 trim_cache size 0 max 16384 2024-09-05T12:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:41:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.585+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe38c0a2cf0 msgr2=0x7fe38c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.585+0000 7fe3aec00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe38c0a2cf0 0x7fe38c0a51b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0012560 tx=0x7fe3a0009500 comp rx=0 tx=0).stop 2024-09-05T12:41:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.585+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 msgr2=0x7fe3a81a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.585+0000 7fe3aec00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a81a4620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe39802a9f0 tx=0x7fe398003b20 comp rx=0 tx=0).stop 2024-09-05T12:41:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.586+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 shutdown_connections 2024-09-05T12:41:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.587+0000 7fe3aec00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3a8105d00 0x7fe3a819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.587+0000 7fe3aec00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe38c0a2cf0 0x7fe38c0a51b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.587+0000 7fe3aec00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3a81011f0 0x7fe3a81a4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.587+0000 7fe3aec00640 1 --2- 172.21.15.71:0/3575774320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3a81008d0 0x7fe3a81a4620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.587+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 >> 172.21.15.71:0/3575774320 conn(0x7fe3a80777c0 msgr2=0x7fe3a810f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.589+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 shutdown_connections 2024-09-05T12:41:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:42.589+0000 7fe3aec00640 1 -- 172.21.15.71:0/3575774320 wait complete. 2024-09-05T12:41:42.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 19712 34.52 MB/sec warmup 36 sec latency 248.831 ms 2024-09-05T12:41:42.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:42 smithi071 ceph-mon[29845]: pgmap v438: 129 pgs: 129 active+clean; 385 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 712 op/s 2024-09-05T12:41:42.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:42 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:42 smithi123 ceph-mon[34321]: pgmap v438: 129 pgs: 129 active+clean; 385 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 712 op/s 2024-09-05T12:41:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:42 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:43.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:42 smithi174 ceph-mon[34313]: pgmap v438: 129 pgs: 129 active+clean; 385 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 712 op/s 2024-09-05T12:41:43.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:42 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:43.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 20467 34.40 MB/sec warmup 37 sec latency 45.410 ms 2024-09-05T12:41:43.850 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:43 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:43.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:43 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:44.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:43 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:44.752 INFO:tasks.workunit.client.0.smithi071.stdout: 10 21803 35.40 MB/sec warmup 38 sec latency 42.793 ms 2024-09-05T12:41:44.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:44 smithi071 ceph-mon[29845]: pgmap v439: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-09-05T12:41:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:44 smithi123 ceph-mon[34321]: pgmap v439: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-09-05T12:41:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:44 smithi174 ceph-mon[34313]: pgmap v439: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2024-09-05T12:41:45.751 INFO:tasks.workunit.client.0.smithi071.stdout: 10 22938 36.26 MB/sec warmup 39 sec latency 63.701 ms 2024-09-05T12:41:46.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 23809 36.32 MB/sec warmup 40 sec latency 82.516 ms 2024-09-05T12:41:46.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:46 smithi071 ceph-mon[29845]: pgmap v440: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 552 op/s 2024-09-05T12:41:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:46 smithi123 ceph-mon[34321]: pgmap v440: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 552 op/s 2024-09-05T12:41:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:46 smithi174 ceph-mon[34313]: pgmap v440: 129 pgs: 129 active+clean; 389 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 552 op/s 2024-09-05T12:41:47.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 24751 36.66 MB/sec warmup 41 sec latency 103.413 ms 2024-09-05T12:41:47.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:47 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:47.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:47 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:48.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:47 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:48.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25358 36.44 MB/sec warmup 42 sec latency 196.571 ms 2024-09-05T12:41:48.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:48 smithi071 ceph-mon[29845]: pgmap v441: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 1.07k op/s 2024-09-05T12:41:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:48 smithi123 ceph-mon[34321]: pgmap v441: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 1.07k op/s 2024-09-05T12:41:49.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:48 smithi174 ceph-mon[34313]: pgmap v441: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.5 MiB/s wr, 1.07k op/s 2024-09-05T12:41:49.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25888 36.42 MB/sec warmup 43 sec latency 485.200 ms 2024-09-05T12:41:50.655 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:50 smithi071 ceph-mon[29845]: pgmap v442: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.1 MiB/s wr, 822 op/s 2024-09-05T12:41:50.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25927 35.62 MB/sec warmup 44 sec latency 1470.812 ms 2024-09-05T12:41:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:50 smithi123 ceph-mon[34321]: pgmap v442: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.1 MiB/s wr, 822 op/s 2024-09-05T12:41:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:50 smithi174 ceph-mon[34313]: pgmap v442: 129 pgs: 129 active+clean; 404 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 6.1 MiB/s wr, 822 op/s 2024-09-05T12:41:51.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25936 34.86 MB/sec warmup 45 sec latency 2275.191 ms 2024-09-05T12:41:52.753 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25936 34.10 MB/sec warmup 46 sec latency 3275.239 ms 2024-09-05T12:41:52.871 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:52 smithi071 ceph-mon[29845]: pgmap v443: 129 pgs: 129 active+clean; 453 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 11 MiB/s wr, 981 op/s 2024-09-05T12:41:52.871 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:52 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:52 smithi123 ceph-mon[34321]: pgmap v443: 129 pgs: 129 active+clean; 453 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 11 MiB/s wr, 981 op/s 2024-09-05T12:41:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:52 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:52 smithi174 ceph-mon[34313]: pgmap v443: 129 pgs: 129 active+clean; 453 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 11 MiB/s wr, 981 op/s 2024-09-05T12:41:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:52 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:41:53.754 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25939 33.37 MB/sec warmup 47 sec latency 4275.292 ms 2024-09-05T12:41:54.754 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25940 32.68 MB/sec warmup 48 sec latency 5275.360 ms 2024-09-05T12:41:54.856 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:54 smithi071 ceph-mon[29845]: pgmap v444: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.7 MiB/s wr, 797 op/s 2024-09-05T12:41:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:54 smithi123 ceph-mon[34321]: pgmap v444: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.7 MiB/s wr, 797 op/s 2024-09-05T12:41:55.009 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:54 smithi174 ceph-mon[34313]: pgmap v444: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.7 MiB/s wr, 797 op/s 2024-09-05T12:41:55.768 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25940 32.00 MB/sec warmup 49 sec latency 6254.216 ms 2024-09-05T12:41:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 -- 172.21.15.71:0/673111666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200a4820 msgr2=0x7f8b200a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:55.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 --2- 172.21.15.71:0/673111666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200a4820 0x7f8b200a4c00 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8b18009900 tx=0x7f8b1802a480 comp rx=0 tx=0).stop 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 -- 172.21.15.71:0/673111666 shutdown_connections 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 --2- 172.21.15.71:0/673111666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200b7540 0x7f8b200b9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 --2- 172.21.15.71:0/673111666 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b200a5140 0x7f8b200b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 --2- 172.21.15.71:0/673111666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200a4820 0x7f8b200a4c00 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.788+0000 7f8b27e00640 1 -- 172.21.15.71:0/673111666 >> 172.21.15.71:0/673111666 conn(0x7f8b2001a110 msgr2=0x7f8b2001a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 -- 172.21.15.71:0/673111666 shutdown_connections 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 -- 172.21.15.71:0/673111666 wait complete. 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 Processor -- start 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 -- start start 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b200a4820 0x7f8b200b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200a5140 0x7f8b200ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200b7540 0x7f8b200aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200bd010 con 0x7f8b200a4820 2024-09-05T12:41:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200bce90 con 0x7f8b200a5140 2024-09-05T12:41:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200bd190 con 0x7f8b200b7540 2024-09-05T12:41:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b26000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200a5140 0x7f8b200ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b26000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200a5140 0x7f8b200ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34686/0 (socket says 172.21.15.71:34686) 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b26000640 1 -- 172.21.15.71:0/1085682428 learned_addr learned my addr 172.21.15.71:0/1085682428 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27400640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200b7540 0x7f8b200aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b26a00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b200a4820 0x7f8b200b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27400640 1 -- 172.21.15.71:0/1085682428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200a5140 msgr2=0x7f8b200ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27400640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200a5140 0x7f8b200ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27400640 1 -- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b200a4820 msgr2=0x7f8b200b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27400640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b200a4820 0x7f8b200b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.793+0000 7f8b27400640 1 -- 172.21.15.71:0/1085682428 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b200af260 con 0x7f8b200b7540 2024-09-05T12:41:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.800+0000 7f8b27400640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200b7540 0x7f8b200aec20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8b28067f00 tx=0x7f8b28071d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.800+0000 7f8b1f400640 1 -- 172.21.15.71:0/1085682428 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2806fa00 con 0x7f8b200b7540 2024-09-05T12:41:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.800+0000 7f8b1f400640 1 -- 172.21.15.71:0/1085682428 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b2806fba0 con 0x7f8b200b7540 2024-09-05T12:41:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.800+0000 7f8b1f400640 1 -- 172.21.15.71:0/1085682428 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b280746b0 con 0x7f8b200b7540 2024-09-05T12:41:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.800+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b200b3180 con 0x7f8b200b7540 2024-09-05T12:41:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.800+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b200b3650 con 0x7f8b200b7540 2024-09-05T12:41:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 Processor -- start 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 -- start start 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5f3b0 0x5563a5d5f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d6c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5563a5cc5100 con 0x5563a5d6a570 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563a5c6d1b0 con 0x5563a5d5f3b0 2024-09-05T12:41:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b304cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563a5cc10c0 con 0x5563a5d5fcd0 2024-09-05T12:41:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b26000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d6c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b26000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d6c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38906/0 (socket says 172.21.15.71:38906) 2024-09-05T12:41:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.805+0000 7f8b26000640 1 -- 172.21.15.71:0/3379063575 learned_addr learned my addr 172.21.15.71:0/3379063575 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.810+0000 7f8b26a00640 1 -- 172.21.15.71:0/3379063575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5f3b0 msgr2=0x5563a5d5f790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:41:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.810+0000 7f8b26a00640 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5f3b0 0x5563a5d5f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.810+0000 7f8b26a00640 1 -- 172.21.15.71:0/3379063575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 msgr2=0x5563a5d6c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.810+0000 7f8b26a00640 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d6c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.810+0000 7f8b26a00640 1 -- 172.21.15.71:0/3379063575 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a59d8210 con 0x5563a5d5fcd0 2024-09-05T12:41:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.810+0000 7f8b26a00640 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6a030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b1802aa70 tx=0x7f8b18004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ca278316a79786d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b13400640 1 -- 172.21.15.71:0/3379063575 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18035030 con 0x5563a5d5fcd0 2024-09-05T12:41:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b13400640 1 -- 172.21.15.71:0/3379063575 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18005520 con 0x5563a5d5fcd0 2024-09-05T12:41:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b13400640 1 -- 172.21.15.71:0/3379063575 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18003850 con 0x5563a5d5fcd0 2024-09-05T12:41:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- 172.21.15.71:0/3379063575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 msgr2=0x5563a5d6a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6a030 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8b1802aa70 tx=0x7f8b18004170 comp rx=0 tx=0).stop 2024-09-05T12:41:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- 172.21.15.71:0/3379063575 shutdown_connections 2024-09-05T12:41:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d6c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6a030 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- 172.21.15.71:0/3379063575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5f3b0 0x5563a5d5f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- 172.21.15.71:0/3379063575 >> 172.21.15.71:0/3379063575 conn(0x5563a5d5b920 msgr2=0x5563a5d5bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- 172.21.15.71:0/3379063575 shutdown_connections 2024-09-05T12:41:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- 172.21.15.71:0/3379063575 wait complete. 2024-09-05T12:41:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- start start 2024-09-05T12:41:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 Processor -- start 2024-09-05T12:41:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5f3b0 0x5563a5d67500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 0x5563a5d67a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d67f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5563a5d6cff0 con 0x5563a5d6a570 2024-09-05T12:41:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5563a5c4e090 con 0x5563a5d5fcd0 2024-09-05T12:41:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563a5d6d170 con 0x5563a5d5f3b0 2024-09-05T12:41:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.811+0000 7f8b304cb740 10 client.0 _ll_register_callbacks cb 0x5563a5ca2f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d67f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d67f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38910/0 (socket says 172.21.15.71:38910) 2024-09-05T12:41:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 -- 172.21.15.71:0/82806669 learned_addr learned my addr 172.21.15.71:0/82806669 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5f3b0 0x5563a5d67500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.821 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 -- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 msgr2=0x5563a5d67a40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:41:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 0x5563a5d67a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 -- 172.21.15.71:0/82806669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 msgr2=0x5563a5d67f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d67f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.822 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.817+0000 7f8b26000640 1 -- 172.21.15.71:0/82806669 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a5aa4620 con 0x5563a5d5f3b0 2024-09-05T12:41:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b26000640 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5f3b0 0x5563a5d67500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b14014dc0 tx=0x7f8b140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f09854577ffd7a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:55.827 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b07e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:41:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b07e00640 1 -- 172.21.15.71:0/82806669 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b14018020 con 0x5563a5d5f3b0 2024-09-05T12:41:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b07e00640 1 -- 172.21.15.71:0/82806669 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7f8b14014170 con 0x5563a5d5f3b0 2024-09-05T12:41:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b18005520 con 0x5563a5d5f3b0 2024-09-05T12:41:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b07e00640 1 -- 172.21.15.71:0/82806669 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8b14026050 con 0x5563a5d5f3b0 2024-09-05T12:41:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 10 client.27049 fetch_fsmap learned FSMap version 63 2024-09-05T12:41:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5563a5d78da0 con 0x5563a5d5f3b0 2024-09-05T12:41:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b1f400640 1 -- 172.21.15.71:0/1085682428 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8b2806a5e0 con 0x7f8b200b7540 2024-09-05T12:41:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b07e00640 1 -- 172.21.15.71:0/82806669 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b14011920 con 0x5563a5d5f3b0 2024-09-05T12:41:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b07e00640 1 -- 172.21.15.71:0/82806669 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f8b14021400 con 0x5563a5d5f3b0 2024-09-05T12:41:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 10 client.27049 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:41:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 10 client.27049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 20 client.27049 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5d7a800 0x5563a5d7cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 4 client.27049 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.823+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8b18005520 con 0x5563a5d7a800 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.829+0000 7f8b1f400640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8b0c0a2ca0 0x7f8b0c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.829+0000 7f8b1f400640 1 -- 172.21.15.71:0/1085682428 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7f8b28132bd0 con 0x7f8b200b7540 2024-09-05T12:41:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.829+0000 7f8b26000640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8b0c0a2ca0 0x7f8b0c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.835+0000 7f8b26000640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8b0c0a2ca0 0x7f8b0c0a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009bd0 tx=0x7f8b1400bf70 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:55.850 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.847+0000 7f8b26a00640 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5d7a800 0x5563a5d7cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.918+0000 7f8b07e00640 10 client.27049 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:55.919 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.918+0000 7f8b26a00640 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5d7a800 0x5563a5d7cc80 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3149c52c7f397a38 server_cookie=d39215c869634ef7 in_seq=0 out_seq=0 2024-09-05T12:41:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b07e00640 1 -- 172.21.15.71:0/82806669 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8b18005520 con 0x5563a5d7a800 2024-09-05T12:41:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b07e00640 10 client.27049 handle_command_reply: tid=0 2024-09-05T12:41:55.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 client.27049 shutdown 2024-09-05T12:41:55.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5f3b0 msgr2=0x5563a5d67500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5f3b0 0x5563a5d67500 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8b14014dc0 tx=0x7f8b140046c0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 shutdown_connections 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5d7a800 0x5563a5d7cc80 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d6a570 0x5563a5d67f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 0x5563a5d67a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 --2- 172.21.15.71:0/82806669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5f3b0 0x5563a5d67500 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 >> 172.21.15.71:0/82806669 conn(0x5563a5d5b920 msgr2=0x5563a5d61630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.930+0000 7f8b07e00640 0 client.27049 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.931+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 shutdown_connections 2024-09-05T12:41:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.931+0000 7f8b304cb740 1 -- 172.21.15.71:0/82806669 wait complete. 2024-09-05T12:41:55.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.931+0000 7f8b304cb740 20 client.27049 trim_cache size 0 max 16384 2024-09-05T12:41:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 Processor -- start 2024-09-05T12:41:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 -- start start 2024-09-05T12:41:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d61e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d69360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 0x5563a5d82330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.973 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20151910 con 0x5563a5d5f3b0 2024-09-05T12:41:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20175640 con 0x5563a5d6a570 2024-09-05T12:41:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b304cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20152040 con 0x5563a5d5fcd0 2024-09-05T12:41:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b26a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d61e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b27400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 0x5563a5d82330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b27400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 0x5563a5d82330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:34710/0 (socket says 172.21.15.71:34710) 2024-09-05T12:41:55.974 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b27400640 1 -- 172.21.15.71:0/592728119 learned_addr learned my addr 172.21.15.71:0/592728119 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b27400640 1 -- 172.21.15.71:0/592728119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 msgr2=0x5563a5d69360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:41:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b27400640 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d69360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.975 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.971+0000 7f8b27400640 1 -- 172.21.15.71:0/592728119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 msgr2=0x5563a5d61e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.976+0000 7f8b27400640 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d61e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.976+0000 7f8b27400640 1 -- 172.21.15.71:0/592728119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563a5d5fb00 con 0x5563a5d6a570 2024-09-05T12:41:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.976+0000 7f8b26a00640 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d61e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:41:55.978 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b27400640 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 0x5563a5d82330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b28050d60 tx=0x7f8b280fec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d70d74580c78ce9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b12a00640 1 -- 172.21.15.71:0/592728119 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b281030b0 con 0x5563a5d6a570 2024-09-05T12:41:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b12a00640 1 -- 172.21.15.71:0/592728119 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28107970 con 0x5563a5d6a570 2024-09-05T12:41:55.979 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b12a00640 1 -- 172.21.15.71:0/592728119 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28107740 con 0x5563a5d6a570 2024-09-05T12:41:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- 172.21.15.71:0/592728119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 msgr2=0x5563a5d82330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 0x5563a5d82330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8b28050d60 tx=0x7f8b280fec00 comp rx=0 tx=0).stop 2024-09-05T12:41:55.980 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- 172.21.15.71:0/592728119 shutdown_connections 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d6a570 0x5563a5d82330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d5fcd0 0x5563a5d69360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- 172.21.15.71:0/592728119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d61e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- 172.21.15.71:0/592728119 >> 172.21.15.71:0/592728119 conn(0x5563a5d5b920 msgr2=0x5563a5d68c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- 172.21.15.71:0/592728119 shutdown_connections 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- 172.21.15.71:0/592728119 wait complete. 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- start start 2024-09-05T12:41:55.981 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 Processor -- start 2024-09-05T12:41:55.982 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d6dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.983 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 0x5563a5de32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8b20152f60 con 0x5563a5d5f3b0 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b201527d0 con 0x5563a5d5fcd0 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8b200a5b00 con 0x5563a5d6a570 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b304cb740 10 client.0 _ll_register_callbacks cb 0x7f8b1400bd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b27400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 0x5563a5de32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b27400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 0x5563a5de32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45576/0 (socket says 172.21.15.71:45576) 2024-09-05T12:41:55.984 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.977+0000 7f8b27400640 1 -- 172.21.15.71:0/1187610837 learned_addr learned my addr 172.21.15.71:0/1187610837 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:41:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b27400640 1 -- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 msgr2=0x5563a5d6e020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:41:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b27400640 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6e020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b27400640 1 -- 172.21.15.71:0/1187610837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 msgr2=0x5563a5d6dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:41:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b27400640 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d6dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:55.985 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b27400640 1 -- 172.21.15.71:0/1187610837 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563a5de3830 con 0x5563a5d6a570 2024-09-05T12:41:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b27400640 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 0x5563a5de32f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000b60 tx=0x7f8b280fedc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69531727f71a141 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:41:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b07e00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:41:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b07e00640 1 -- 172.21.15.71:0/1187610837 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2811b060 con 0x5563a5d6a570 2024-09-05T12:41:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b07e00640 1 -- 172.21.15.71:0/1187610837 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7f8b281074e0 con 0x5563a5d6a570 2024-09-05T12:41:55.987 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b28107970 con 0x5563a5d6a570 2024-09-05T12:41:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b07e00640 1 -- 172.21.15.71:0/1187610837 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8b281076e0 con 0x5563a5d6a570 2024-09-05T12:41:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b304cb740 10 client.27052 fetch_fsmap learned FSMap version 63 2024-09-05T12:41:55.988 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5563a5de4af0 con 0x5563a5d6a570 2024-09-05T12:41:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b07e00640 1 -- 172.21.15.71:0/1187610837 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b28107970 con 0x5563a5d6a570 2024-09-05T12:41:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.983+0000 7f8b07e00640 1 -- 172.21.15.71:0/1187610837 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f8b2806a170 con 0x5563a5d6a570 2024-09-05T12:41:55.989 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b304cb740 10 client.27052 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:41:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b304cb740 10 client.27052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:41:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b304cb740 20 client.27052 populate_metadata read hostname 'smithi071' 2024-09-05T12:41:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b304cb740 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5de51f0 0x5563a5de7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:41:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b304cb740 4 client.27052 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:41:55.990 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5563a5cd2830 con 0x5563a5de51f0 2024-09-05T12:41:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b26a00640 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5de51f0 0x5563a5de7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:41:55.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b26a00640 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5de51f0 0x5563a5de7610 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f766915a800b2391 server_cookie=c33e3614c5ab782f in_seq=0 out_seq=0 2024-09-05T12:41:55.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:55.988+0000 7f8b07e00640 10 client.27052 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b07e00640 1 -- 172.21.15.71:0/1187610837 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5563a5cd2830 con 0x5563a5de51f0 2024-09-05T12:41:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b07e00640 10 client.27052 handle_command_reply: tid=0 2024-09-05T12:41:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 client.27052 shutdown 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 msgr2=0x5563a5de32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 0x5563a5de32f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8b28000b60 tx=0x7f8b280fedc0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 shutdown_connections 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5563a5de51f0 0x5563a5de7610 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563a5d6a570 0x5563a5de32f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5563a5d5fcd0 0x5563a5d6e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 --2- 172.21.15.71:0/1187610837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5563a5d5f3b0 0x5563a5d6dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 >> 172.21.15.71:0/1187610837 conn(0x5563a5d5b920 msgr2=0x5563a5d5a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.733+0000 7f8b07e00640 0 client.27052 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:41:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 shutdown_connections 2024-09-05T12:41:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b304cb740 1 -- 172.21.15.71:0/1187610837 wait complete. 2024-09-05T12:41:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b304cb740 20 client.27052 trim_cache size 0 max 16384 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8b0c0a2ca0 msgr2=0x7f8b0c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8b0c0a2ca0 0x7f8b0c0a5160 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009bd0 tx=0x7f8b1400bf70 comp rx=0 tx=0).stop 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200b7540 msgr2=0x7f8b200aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:41:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200b7540 0x7f8b200aec20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8b28067f00 tx=0x7f8b28071d80 comp rx=0 tx=0).stop 2024-09-05T12:41:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 shutdown_connections 2024-09-05T12:41:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8b200b7540 0x7f8b200aec20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b200a5140 0x7f8b200ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8b0c0a2ca0 0x7f8b0c0a5160 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 --2- 172.21.15.71:0/1085682428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b200a4820 0x7f8b200b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:41:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 >> 172.21.15.71:0/1085682428 conn(0x7f8b2001a110 msgr2=0x7f8b200a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:41:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 shutdown_connections 2024-09-05T12:41:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:41:56.736+0000 7f8b27e00640 1 -- 172.21.15.71:0/1085682428 wait complete. 2024-09-05T12:41:56.768 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25953 31.38 MB/sec warmup 50 sec latency 6232.567 ms 2024-09-05T12:41:56.773 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:56 smithi071 ceph-mon[29845]: pgmap v445: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.3 MiB/s wr, 773 op/s 2024-09-05T12:41:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:56 smithi123 ceph-mon[34321]: pgmap v445: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.3 MiB/s wr, 773 op/s 2024-09-05T12:41:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:56 smithi174 ceph-mon[34313]: pgmap v445: 129 pgs: 129 active+clean; 468 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.3 MiB/s wr, 773 op/s 2024-09-05T12:41:57.768 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25961 30.78 MB/sec warmup 51 sec latency 6890.140 ms 2024-09-05T12:41:58.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 25984 30.19 MB/sec warmup 52 sec latency 7654.338 ms 2024-09-05T12:41:58.787 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:41:58 smithi071 ceph-mon[29845]: pgmap v446: 129 pgs: 129 active+clean; 485 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 961 op/s 2024-09-05T12:41:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:41:58 smithi123 ceph-mon[34321]: pgmap v446: 129 pgs: 129 active+clean; 485 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 961 op/s 2024-09-05T12:41:59.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:41:58 smithi174 ceph-mon[34313]: pgmap v446: 129 pgs: 129 active+clean; 485 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 14 MiB/s wr, 961 op/s 2024-09-05T12:41:59.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26043 29.71 MB/sec warmup 53 sec latency 8654.426 ms 2024-09-05T12:42:00.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26048 29.16 MB/sec warmup 54 sec latency 9654.491 ms 2024-09-05T12:42:00.772 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:00 smithi071 ceph-mon[29845]: pgmap v447: 129 pgs: 129 active+clean; 485 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 446 op/s 2024-09-05T12:42:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:00 smithi123 ceph-mon[34321]: pgmap v447: 129 pgs: 129 active+clean; 485 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 446 op/s 2024-09-05T12:42:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:00 smithi174 ceph-mon[34313]: pgmap v447: 129 pgs: 129 active+clean; 485 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 12 MiB/s wr, 446 op/s 2024-09-05T12:42:01.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 28.63 MB/sec warmup 55 sec latency 10654.538 ms 2024-09-05T12:42:02.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 28.12 MB/sec warmup 56 sec latency 11654.609 ms 2024-09-05T12:42:02.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:02 smithi071 ceph-mon[29845]: pgmap v448: 129 pgs: 129 active+clean; 442 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 747 op/s 2024-09-05T12:42:02.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:02 smithi123 ceph-mon[34321]: pgmap v448: 129 pgs: 129 active+clean; 442 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 747 op/s 2024-09-05T12:42:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:02 smithi174 ceph-mon[34313]: pgmap v448: 129 pgs: 129 active+clean; 442 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 747 op/s 2024-09-05T12:42:03.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 27.63 MB/sec warmup 57 sec latency 12654.684 ms 2024-09-05T12:42:04.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 27.15 MB/sec warmup 58 sec latency 13654.770 ms 2024-09-05T12:42:04.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:04 smithi071 ceph-mon[29845]: pgmap v449: 129 pgs: 129 active+clean; 421 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 674 op/s 2024-09-05T12:42:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:04 smithi123 ceph-mon[34321]: pgmap v449: 129 pgs: 129 active+clean; 421 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 674 op/s 2024-09-05T12:42:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:04 smithi174 ceph-mon[34313]: pgmap v449: 129 pgs: 129 active+clean; 421 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 12 MiB/s wr, 674 op/s 2024-09-05T12:42:05.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 26.69 MB/sec warmup 59 sec latency 14654.843 ms 2024-09-05T12:42:06.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 26.25 MB/sec warmup 60 sec latency 15654.911 ms 2024-09-05T12:42:06.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:06 smithi071 ceph-mon[29845]: pgmap v450: 129 pgs: 129 active+clean; 421 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 576 op/s 2024-09-05T12:42:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:06 smithi123 ceph-mon[34321]: pgmap v450: 129 pgs: 129 active+clean; 421 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 576 op/s 2024-09-05T12:42:07.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:06 smithi174 ceph-mon[34313]: pgmap v450: 129 pgs: 129 active+clean; 421 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 576 op/s 2024-09-05T12:42:07.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 25.82 MB/sec warmup 61 sec latency 16654.986 ms 2024-09-05T12:42:08.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 25.40 MB/sec warmup 62 sec latency 17655.025 ms 2024-09-05T12:42:08.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:08 smithi123 ceph-mon[34321]: pgmap v451: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 598 op/s 2024-09-05T12:42:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:42:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:08 smithi174 ceph-mon[34313]: pgmap v451: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 598 op/s 2024-09-05T12:42:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:42:09.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:08 smithi071 ceph-mon[29845]: pgmap v451: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 11 MiB/s wr, 598 op/s 2024-09-05T12:42:09.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:42:09.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 25.00 MB/sec warmup 63 sec latency 18655.065 ms 2024-09-05T12:42:09.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- 172.21.15.71:0/860849261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b4103f50 msgr2=0x7fe4b4104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- 172.21.15.71:0/860849261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b4103f50 0x7fe4b4104330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009900 tx=0x7fe4a402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- 172.21.15.71:0/860849261 shutdown_connections 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- 172.21.15.71:0/860849261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 0x7fe4b4110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- 172.21.15.71:0/860849261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b4104900 0x7fe4b4108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- 172.21.15.71:0/860849261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b4103f50 0x7fe4b4104330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- 172.21.15.71:0/860849261 >> 172.21.15.71:0/860849261 conn(0x7fe4b40ff9e0 msgr2=0x7fe4b4101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- 172.21.15.71:0/860849261 shutdown_connections 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- 172.21.15.71:0/860849261 wait complete. 2024-09-05T12:42:09.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 Processor -- start 2024-09-05T12:42:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- start start 2024-09-05T12:42:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b4103f50 0x7fe4b419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 0x7fe4b419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 0x7fe4b419f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4114130 con 0x7fe4b419eea0 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b4113fb0 con 0x7fe4b41090a0 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4ba600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41142b0 con 0x7fe4b4103f50 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 0x7fe4b419f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 0x7fe4b419f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39744/0 (socket says 172.21.15.71:39744) 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 -- 172.21.15.71:0/3095027825 learned_addr learned my addr 172.21.15.71:0/3095027825 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 0x7fe4b419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3400640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b4103f50 0x7fe4b419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 -- 172.21.15.71:0/3095027825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b4103f50 msgr2=0x7fe4b419a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b4103f50 0x7fe4b419a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 -- 172.21.15.71:0/3095027825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 msgr2=0x7fe4b419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 0x7fe4b419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b3e00640 1 -- 172.21.15.71:0/3095027825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b419f990 con 0x7fe4b419eea0 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.855+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 0x7fe4b419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.860+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 0x7fe4b419f350 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00efd0 tx=0x7fe4ac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.860+0000 7fe4abe00640 1 -- 172.21.15.71:0/3095027825 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac003b70 con 0x7fe4b419eea0 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.860+0000 7fe4abe00640 1 -- 172.21.15.71:0/3095027825 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4ac003d10 con 0x7fe4b419eea0 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.860+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b419fc80 con 0x7fe4b419eea0 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.860+0000 7fe4abe00640 1 -- 172.21.15.71:0/3095027825 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0109f0 con 0x7fe4b419eea0 2024-09-05T12:42:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.860+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b41a7470 con 0x7fe4b419eea0 2024-09-05T12:42:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 Processor -- start 2024-09-05T12:42:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4abe00640 1 -- 172.21.15.71:0/3095027825 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe4ac004020 con 0x7fe4b419eea0 2024-09-05T12:42:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- start start 2024-09-05T12:42:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb0e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5592eb04e100 con 0x5592eb0e81e0 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592eaff61b0 con 0x5592eb0e8b00 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592eb04a0c0 con 0x5592eb0f33a0 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb0e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb0e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39756/0 (socket says 172.21.15.71:39756) 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 -- 172.21.15.71:0/53135097 learned_addr learned my addr 172.21.15.71:0/53135097 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 -- 172.21.15.71:0/53135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 msgr2=0x5592eb0f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 -- 172.21.15.71:0/53135097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 msgr2=0x5592eb0e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb0e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 -- 172.21.15.71:0/53135097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592ead61210 con 0x5592eb0e8b00 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb0e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4abe00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4980a2e20 0x7fe4980a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b3400640 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f2e60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4a402a9e0 tx=0x7fe4a4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c952e1f89b9ada54 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4abe00640 1 -- 172.21.15.71:0/3095027825 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe4ac0cfb80 con 0x7fe4b419eea0 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4980a2e20 0x7fe4980a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4a8c00640 1 -- 172.21.15.71:0/53135097 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4035040 con 0x5592eb0e8b00 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4a8c00640 1 -- 172.21.15.71:0/53135097 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004400 con 0x5592eb0e8b00 2024-09-05T12:42:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4a8c00640 1 -- 172.21.15.71:0/53135097 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40035f0 con 0x5592eb0e8b00 2024-09-05T12:42:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- 172.21.15.71:0/53135097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 msgr2=0x5592eb0f2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4980a2e20 0x7fe4980a52e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5592eb0e5ad0 tx=0x7fe4a0009be0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f2e60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe4a402a9e0 tx=0x7fe4a4005de0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- 172.21.15.71:0/53135097 shutdown_connections 2024-09-05T12:42:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f2e60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/53135097 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb0e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- 172.21.15.71:0/53135097 >> 172.21.15.71:0/53135097 conn(0x5592eb0e4750 msgr2=0x5592eb0e4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- 172.21.15.71:0/53135097 shutdown_connections 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- 172.21.15.71:0/53135097 wait complete. 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 -- start start 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.863+0000 7fe4bd63b740 1 Processor -- start 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0e81e0 0x5592eb0f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 0x5592eb0f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5592eb0f5f70 con 0x5592eb0f33a0 2024-09-05T12:42:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592eafd7090 con 0x5592eb0e8b00 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592eb0f60f0 con 0x5592eb0e81e0 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 10 client.0 _ll_register_callbacks cb 0x5592eb02bf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 0x5592eb0f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 0x5592eb0f0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39764/0 (socket says 172.21.15.71:39764) 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0e81e0 0x5592eb0f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 -- 172.21.15.71:0/570543802 learned_addr learned my addr 172.21.15.71:0/570543802 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 -- 172.21.15.71:0/570543802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0e81e0 msgr2=0x5592eb0f03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0e81e0 0x5592eb0f03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 -- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 msgr2=0x5592eb0f0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 -- 172.21.15.71:0/570543802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eae2d620 con 0x5592eb0f33a0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 0x5592eb0f0e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0a43f0 tx=0x7fe4ac09b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce1c1ef9f83b4e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 1 -- 172.21.15.71:0/570543802 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac09bcb0 con 0x5592eb0f33a0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 1 -- 172.21.15.71:0/570543802 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe4ac004310 con 0x5592eb0f33a0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4004400 con 0x5592eb0f33a0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 1 -- 172.21.15.71:0/570543802 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe4ac0a0060 con 0x5592eb0f33a0 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 10 client.17007 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5592eb101bd0 con 0x5592eb0f33a0 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 1 -- 172.21.15.71:0/570543802 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004400 con 0x5592eb0f33a0 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 1 -- 172.21.15.71:0/570543802 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7fe4ac01f020 con 0x5592eb0f33a0 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 10 client.17007 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 20 client.17007 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb103150 0x5592eb1055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 4 client.17007 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592eb05b830 con 0x5592eb103150 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3400640 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb103150 0x5592eb1055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4b3400640 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb103150 0x5592eb1055d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b288f64ab1a2155b server_cookie=5bd650c6b74a9353 in_seq=0 out_seq=0 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 10 client.17007 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 1 -- 172.21.15.71:0/570543802 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592eb05b830 con 0x5592eb103150 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 10 client.17007 handle_command_reply: tid=0 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 client.17007 shutdown 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 msgr2=0x5592eb0f0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 0x5592eb0f0e70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac0a43f0 tx=0x7fe4ac09b610 comp rx=0 tx=0).stop 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 shutdown_connections 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb103150 0x5592eb1055d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0f33a0 0x5592eb0f0e70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb0f0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/570543802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0e81e0 0x5592eb0f03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe49d600640 0 client.17007 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:09.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 >> 172.21.15.71:0/570543802 conn(0x5592eb0e4750 msgr2=0x5592eb0ea4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 shutdown_connections 2024-09-05T12:42:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 1 -- 172.21.15.71:0/570543802 wait complete. 2024-09-05T12:42:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.868+0000 7fe4bd63b740 20 client.17007 trim_cache size 0 max 16384 2024-09-05T12:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 Processor -- start 2024-09-05T12:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- start start 2024-09-05T12:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb10aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb10b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41b0fe0 con 0x5592eb0e81e0 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41b2d00 con 0x5592eb0e8b00 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41b1710 con 0x5592eb0f33a0 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb10aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb10aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39768/0 (socket says 172.21.15.71:39768) 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 -- 172.21.15.71:0/3775522639 learned_addr learned my addr 172.21.15.71:0/3775522639 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb10b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 -- 172.21.15.71:0/3775522639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 msgr2=0x5592eb0ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 -- 172.21.15.71:0/3775522639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 msgr2=0x5592eb10b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb10b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 -- 172.21.15.71:0/3775522639 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592eb0dec20 con 0x5592eb0e81e0 2024-09-05T12:42:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb10aee0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe4a402a9e0 tx=0x7fe4a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b56a157bb9f239 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe49fe00640 1 -- 172.21.15.71:0/3775522639 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4035040 con 0x5592eb0e81e0 2024-09-05T12:42:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe49fe00640 1 -- 172.21.15.71:0/3775522639 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a40048c0 con 0x5592eb0e81e0 2024-09-05T12:42:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe49fe00640 1 -- 172.21.15.71:0/3775522639 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4003a20 con 0x5592eb0e81e0 2024-09-05T12:42:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- 172.21.15.71:0/3775522639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 msgr2=0x5592eb10aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb10aee0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe4a402a9e0 tx=0x7fe4a4000bc0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- 172.21.15.71:0/3775522639 shutdown_connections 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e8b00 0x5592eb10b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/3775522639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e81e0 0x5592eb10aee0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- 172.21.15.71:0/3775522639 >> 172.21.15.71:0/3775522639 conn(0x5592eb0e4750 msgr2=0x5592eb0ef4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- 172.21.15.71:0/3775522639 shutdown_connections 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- 172.21.15.71:0/3775522639 wait complete. 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- start start 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 Processor -- start 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e81e0 0x5592eb0f7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 0x5592eb0f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41b2630 con 0x5592eb0e8b00 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41b1ea0 con 0x5592eb0e81e0 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b41aefe0 con 0x5592eb0f33a0 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4bd63b740 10 client.0 _ll_register_callbacks cb 0x7fe4ac009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 0x5592eb0f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 0x5592eb0f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39770/0 (socket says 172.21.15.71:39770) 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 -- 172.21.15.71:0/1256334801 learned_addr learned my addr 172.21.15.71:0/1256334801 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3400640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e81e0 0x5592eb0f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b3e00640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 -- 172.21.15.71:0/1256334801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 msgr2=0x5592eb0f7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 -- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e81e0 msgr2=0x5592eb0f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e81e0 0x5592eb0f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 -- 172.21.15.71:0/1256334801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eb16d050 con 0x5592eb0e8b00 2024-09-05T12:42:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 0x5592eb0f76b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009710 tx=0x7fe4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af73c8c6f7389faf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.887+0000 7fe49d600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:42:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe49d600640 1 -- 172.21.15.71:0/1256334801 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0022d20 con 0x5592eb0e8b00 2024-09-05T12:42:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe49d600640 1 -- 172.21.15.71:0/1256334801 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe4a0005020 con 0x5592eb0e8b00 2024-09-05T12:42:09.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a40048c0 con 0x5592eb0e8b00 2024-09-05T12:42:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe49d600640 1 -- 172.21.15.71:0/1256334801 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe4a0005220 con 0x5592eb0e8b00 2024-09-05T12:42:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 10 client.17013 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5592eb16e390 con 0x5592eb0e8b00 2024-09-05T12:42:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe49d600640 1 -- 172.21.15.71:0/1256334801 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a40048c0 con 0x5592eb0e8b00 2024-09-05T12:42:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe49d600640 1 -- 172.21.15.71:0/1256334801 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7fe4a002c070 con 0x5592eb0e8b00 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 10 client.17013 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 20 client.17013 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb16ec80 0x5592eb1710a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 4 client.17013 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592eb055b80 con 0x5592eb16ec80 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4b3400640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb16ec80 0x5592eb1710a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe4b3400640 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb16ec80 0x5592eb1710a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1848221bee4e8aa7 server_cookie=fc050b5bee49313d in_seq=0 out_seq=0 2024-09-05T12:42:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.892+0000 7fe49d600640 10 client.17013 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe49d600640 1 -- 172.21.15.71:0/1256334801 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592eb055b80 con 0x5592eb16ec80 2024-09-05T12:42:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe49d600640 10 client.17013 handle_command_reply: tid=0 2024-09-05T12:42:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 client.17013 shutdown 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 msgr2=0x5592eb0f76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 0x5592eb0f76b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4a0009710 tx=0x7fe4a00046c0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 shutdown_connections 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5592eb16ec80 0x5592eb1710a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592eb0f33a0 0x5592eb0f7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5592eb0e8b00 0x5592eb0f76b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 --2- 172.21.15.71:0/1256334801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592eb0e81e0 0x5592eb0f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 >> 172.21.15.71:0/1256334801 conn(0x5592eb0e4750 msgr2=0x5592eb0fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe49d600640 0 client.17013 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 shutdown_connections 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 1 -- 172.21.15.71:0/1256334801 wait complete. 2024-09-05T12:42:09.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4bd63b740 20 client.17013 trim_cache size 0 max 16384 2024-09-05T12:42:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:42:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:42:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:42:09.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4980a2e20 msgr2=0x7fe4980a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4980a2e20 0x7fe4980a52e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x5592eb0e5ad0 tx=0x7fe4a0009be0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 msgr2=0x7fe4b419f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 0x7fe4b419f350 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00efd0 tx=0x7fe4ac00c6a0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 shutdown_connections 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4980a2e20 0x7fe4980a52e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4b419eea0 0x7fe4b419f350 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b41090a0 0x7fe4b419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 --2- 172.21.15.71:0/3095027825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b4103f50 0x7fe4b419a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 >> 172.21.15.71:0/3095027825 conn(0x7fe4b40ff9e0 msgr2=0x7fe4b4100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:09.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 shutdown_connections 2024-09-05T12:42:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:09.934+0000 7fe4ba600640 1 -- 172.21.15.71:0/3095027825 wait complete. 2024-09-05T12:42:10.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 24.61 MB/sec warmup 64 sec latency 19655.109 ms 2024-09-05T12:42:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:10 smithi123 ceph-mon[34321]: pgmap v452: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 5.5 MiB/s wr, 410 op/s 2024-09-05T12:42:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:10 smithi174 ceph-mon[34313]: pgmap v452: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 5.5 MiB/s wr, 410 op/s 2024-09-05T12:42:11.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:10 smithi071 ceph-mon[29845]: pgmap v452: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 5.5 MiB/s wr, 410 op/s 2024-09-05T12:42:11.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 24.23 MB/sec warmup 65 sec latency 20655.152 ms 2024-09-05T12:42:12.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 23.86 MB/sec warmup 66 sec latency 21655.193 ms 2024-09-05T12:42:12.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:12 smithi123 ceph-mon[34321]: pgmap v453: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 5.5 MiB/s wr, 424 op/s 2024-09-05T12:42:13.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:12 smithi174 ceph-mon[34313]: pgmap v453: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 5.5 MiB/s wr, 424 op/s 2024-09-05T12:42:13.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:12 smithi071 ceph-mon[29845]: pgmap v453: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 5.5 MiB/s wr, 424 op/s 2024-09-05T12:42:13.769 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 23.51 MB/sec warmup 67 sec latency 22655.236 ms 2024-09-05T12:42:14.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 23.16 MB/sec warmup 68 sec latency 23655.281 ms 2024-09-05T12:42:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:14 smithi123 ceph-mon[34321]: pgmap v454: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-05T12:42:15.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:14 smithi174 ceph-mon[34313]: pgmap v454: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-05T12:42:15.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:14 smithi071 ceph-mon[29845]: pgmap v454: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-09-05T12:42:15.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 22.83 MB/sec warmup 69 sec latency 24655.325 ms 2024-09-05T12:42:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:15 smithi174 ceph-mon[34313]: pgmap v455: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 159 KiB/s wr, 40 op/s 2024-09-05T12:42:16.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:15 smithi071 ceph-mon[29845]: pgmap v455: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 159 KiB/s wr, 40 op/s 2024-09-05T12:42:16.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:15 smithi123 ceph-mon[34321]: pgmap v455: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 159 KiB/s wr, 40 op/s 2024-09-05T12:42:16.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 22.50 MB/sec warmup 70 sec latency 25655.369 ms 2024-09-05T12:42:17.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 22.18 MB/sec warmup 71 sec latency 26655.412 ms 2024-09-05T12:42:18.089 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:17 smithi071 ceph-mon[29845]: pgmap v456: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 165 KiB/s wr, 46 op/s 2024-09-05T12:42:18.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:17 smithi123 ceph-mon[34321]: pgmap v456: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 165 KiB/s wr, 46 op/s 2024-09-05T12:42:18.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:17 smithi174 ceph-mon[34313]: pgmap v456: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 165 KiB/s wr, 46 op/s 2024-09-05T12:42:18.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 21.88 MB/sec warmup 72 sec latency 27655.460 ms 2024-09-05T12:42:19.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 21.58 MB/sec warmup 73 sec latency 28655.550 ms 2024-09-05T12:42:20.097 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:19 smithi071 ceph-mon[29845]: pgmap v457: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 23 op/s 2024-09-05T12:42:20.098 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:19 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:42:20.098 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:19 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:42:20.098 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:19 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:42:20.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:19 smithi123 ceph-mon[34321]: pgmap v457: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 23 op/s 2024-09-05T12:42:20.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:19 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:42:20.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:19 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:42:20.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:19 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:42:20.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:19 smithi174 ceph-mon[34313]: pgmap v457: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 23 op/s 2024-09-05T12:42:20.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:19 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:42:20.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:19 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:42:20.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:19 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:42:20.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26053 21.28 MB/sec warmup 74 sec latency 29655.617 ms 2024-09-05T12:42:21.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26054 21.00 MB/sec warmup 75 sec latency 30655.741 ms 2024-09-05T12:42:22.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:21 smithi071 ceph-mon[29845]: pgmap v458: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-05T12:42:22.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:21 smithi123 ceph-mon[34321]: pgmap v458: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-05T12:42:22.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:21 smithi174 ceph-mon[34313]: pgmap v458: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 26 op/s 2024-09-05T12:42:22.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26054 20.73 MB/sec warmup 76 sec latency 31655.817 ms 2024-09-05T12:42:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 -- 172.21.15.71:0/4102541846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac1048e0 msgr2=0x7fe4ac108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/4102541846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac108920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe498009900 tx=0x7fe49802a410 comp rx=0 tx=0).stop 2024-09-05T12:42:23.059 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 -- 172.21.15.71:0/4102541846 shutdown_connections 2024-09-05T12:42:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/4102541846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ac109080 0x7fe4ac110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/4102541846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac1048e0 0x7fe4ac108920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/4102541846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac103f30 0x7fe4ac104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 -- 172.21.15.71:0/4102541846 >> 172.21.15.71:0/4102541846 conn(0x7fe4ac0ff980 msgr2=0x7fe4ac101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 -- 172.21.15.71:0/4102541846 shutdown_connections 2024-09-05T12:42:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.058+0000 7fe4b2a00640 1 -- 172.21.15.71:0/4102541846 wait complete. 2024-09-05T12:42:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.059+0000 7fe4b2a00640 1 Processor -- start 2024-09-05T12:42:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.059+0000 7fe4b2a00640 1 -- start start 2024-09-05T12:42:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.059+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ac103f30 0x7fe4ac19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.059+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 0x7fe4ac19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4b2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac19ee50 0x7fe4ac19f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4b2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1140a0 con 0x7fe4ac109080 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4b2a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac113f20 con 0x7fe4ac19ee50 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4b2a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac114220 con 0x7fe4ac103f30 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 0x7fe4ac19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 0x7fe4ac19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49534/0 (socket says 172.21.15.71:49534) 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 -- 172.21.15.71:0/2262970690 learned_addr learned my addr 172.21.15.71:0/2262970690 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4abe00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ac103f30 0x7fe4ac19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac19ee50 0x7fe4ac19f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 -- 172.21.15.71:0/2262970690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ac103f30 msgr2=0x7fe4ac19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ac103f30 0x7fe4ac19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 -- 172.21.15.71:0/2262970690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac19ee50 msgr2=0x7fe4ac19f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac19ee50 0x7fe4ac19f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.060+0000 7fe4ab400640 1 -- 172.21.15.71:0/2262970690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac19f940 con 0x7fe4ac109080 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.061+0000 7fe4ab400640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 0x7fe4ac19aad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe498005940 tx=0x7fe4980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.061+0000 7fe4a8c00640 1 -- 172.21.15.71:0/2262970690 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49803f030 con 0x7fe4ac109080 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.061+0000 7fe4a8c00640 1 -- 172.21.15.71:0/2262970690 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4980044c0 con 0x7fe4ac109080 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.061+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac19fbd0 con 0x7fe4ac109080 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.061+0000 7fe4a8c00640 1 -- 172.21.15.71:0/2262970690 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498035070 con 0x7fe4ac109080 2024-09-05T12:42:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.061+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac1a7390 con 0x7fe4ac109080 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 Processor -- start 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 -- start start 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4a8c00640 1 -- 172.21.15.71:0/2262970690 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe498042030 con 0x7fe4ac109080 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e31917c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319186e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319189790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e3190e2100 con 0x55e31917c1e0 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e31908a1b0 con 0x55e31917cb00 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3190de0c0 con 0x55e3191873a0 2024-09-05T12:42:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e31917c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e31917c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49550/0 (socket says 172.21.15.71:49550) 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.065+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319186e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1284968702 learned_addr learned my addr 172.21.15.71:0/1284968702 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319189790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1284968702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 msgr2=0x55e319189790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319189790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1284968702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 msgr2=0x55e319186e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319186e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1284968702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e318df5210 con 0x55e31917c1e0 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319186e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e31917c5c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009f50 tx=0x7fe4a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1945afabfb0cae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4a8c00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4900a2da0 0x7fe4900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.066+0000 7fe4a1200640 1 -- 172.21.15.71:0/1284968702 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4004d20 con 0x55e31917c1e0 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4a1200640 1 -- 172.21.15.71:0/1284968702 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004050 con 0x55e31917c1e0 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4a1200640 1 -- 172.21.15.71:0/1284968702 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4005220 con 0x55e31917c1e0 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4a8c00640 1 -- 172.21.15.71:0/2262970690 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe4980ef840 con 0x7fe4ac109080 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4abe00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4900a2da0 0x7fe4900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1284968702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 msgr2=0x55e31917c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e31917c5c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009f50 tx=0x7fe4a40118d0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4abe00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4900a2da0 0x7fe4900a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009f70 tx=0x7fe49c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1284968702 shutdown_connections 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319189790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319186e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1284968702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e31917c5c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1284968702 >> 172.21.15.71:0/1284968702 conn(0x55e319178750 msgr2=0x55e319178b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1284968702 shutdown_connections 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.067+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1284968702 wait complete. 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 -- start start 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 Processor -- start 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e319184330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319184870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319184db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e319189fd0 con 0x55e31917c1e0 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e31906b090 con 0x55e31917cb00 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.068+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e31918a150 con 0x55e3191873a0 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4b58b3740 10 client.0 _ll_register_callbacks cb 0x55e3190bff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e319184330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e319184330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49552/0 (socket says 172.21.15.71:49552) 2024-09-05T12:42:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 -- 172.21.15.71:0/1757560051 learned_addr learned my addr 172.21.15.71:0/1757560051 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319184870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319184db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 -- 172.21.15.71:0/1757560051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 msgr2=0x55e319184db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319184db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 -- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 msgr2=0x55e319184870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319184870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4abe00640 1 -- 172.21.15.71:0/1757560051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e318ec1620 con 0x55e31917c1e0 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.069+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319184870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e319184330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004620 tx=0x7fe49c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731cb2247575ea5f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe495600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe495600640 1 -- 172.21.15.71:0/1757560051 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c01f640 con 0x55e31917c1e0 2024-09-05T12:42:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe495600640 1 -- 172.21.15.71:0/1757560051 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe49c01f7e0 con 0x55e31917c1e0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4004050 con 0x55e31917c1e0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe495600640 1 -- 172.21.15.71:0/1757560051 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe49c01f9e0 con 0x55e31917c1e0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe4b58b3740 10 client.17025 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55e319195c30 con 0x55e31917c1e0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.070+0000 7fe495600640 1 -- 172.21.15.71:0/1757560051 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004050 con 0x55e31917c1e0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe495600640 1 -- 172.21.15.71:0/1757560051 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7fe49c027070 con 0x55e31917c1e0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4b58b3740 10 client.17025 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4b58b3740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4b58b3740 20 client.17025 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3191971b0 0x55e3191995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4b58b3740 4 client.17025 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3190ef830 con 0x55e3191971b0 2024-09-05T12:42:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.071+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3191971b0 0x55e3191995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.072+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3191971b0 0x55e3191995e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e66e3b8f047ec80 server_cookie=b50c13b3039511a4 in_seq=0 out_seq=0 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.072+0000 7fe495600640 10 client.17025 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.072+0000 7fe495600640 1 -- 172.21.15.71:0/1757560051 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3190ef830 con 0x55e3191971b0 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.072+0000 7fe495600640 10 client.17025 handle_command_reply: tid=0 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 client.17025 shutdown 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 msgr2=0x55e319184330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e319184330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004620 tx=0x7fe49c004730 comp rx=0 tx=0).stop 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 shutdown_connections 2024-09-05T12:42:23.079 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e3191971b0 0x55e3191995e0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e319184db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e319184870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1757560051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917c1e0 0x55e319184330 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 >> 172.21.15.71:0/1757560051 conn(0x55e319178750 msgr2=0x55e31917e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe495600640 0 client.17025 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 shutdown_connections 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1757560051 wait complete. 2024-09-05T12:42:23.080 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.073+0000 7fe4b58b3740 20 client.17025 trim_cache size 0 max 16384 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 Processor -- start 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 -- start start 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31917c1e0 0x55e3191738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e31919eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 0x55e31919f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b1070 con 0x55e3191873a0 2024-09-05T12:42:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b2d90 con 0x55e31917cb00 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.088+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b17a0 con 0x55e31917c1e0 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 0x55e31919f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e31919eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 0x55e31919f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49566/0 (socket says 172.21.15.71:49566) 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1696737234 learned_addr learned my addr 172.21.15.71:0/1696737234 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31917c1e0 0x55e3191738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1696737234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31917c1e0 msgr2=0x55e3191738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31917c1e0 0x55e3191738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1696737234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 msgr2=0x55e31919eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e31919eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1696737234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e319195a40 con 0x55e3191873a0 2024-09-05T12:42:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e31919eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.089+0000 7fe4b0a00640 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 0x55e31919f430 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4011dd0 tx=0x7fe4a40059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fbb40145df8aaf server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe497e00640 1 -- 172.21.15.71:0/1696737234 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a402a070 con 0x55e3191873a0 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe497e00640 1 -- 172.21.15.71:0/1696737234 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004530 con 0x55e3191873a0 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe497e00640 1 -- 172.21.15.71:0/1696737234 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400fab0 con 0x55e3191873a0 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1696737234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 msgr2=0x55e31919f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 0x55e31919f430 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4011dd0 tx=0x7fe4a40059c0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1696737234 shutdown_connections 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3191873a0 0x55e31919f430 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917cb00 0x55e31919eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1696737234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e31917c1e0 0x55e3191738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1696737234 >> 172.21.15.71:0/1696737234 conn(0x55e319178750 msgr2=0x55e319183400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1696737234 shutdown_connections 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.090+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1696737234 wait complete. 2024-09-05T12:42:23.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 -- start start 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 Processor -- start 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917c1e0 0x55e31918f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917cb00 0x55e31918f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e31918a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b26c0 con 0x55e31917cb00 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b1f30 con 0x55e31917c1e0 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1af070 con 0x55e3191873a0 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.091+0000 7fe4b58b3740 10 client.0 _ll_register_callbacks cb 0x7fe49c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917cb00 0x55e31918f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e31918a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917c1e0 0x55e31918f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4b0a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e31918a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:34200/0 (socket says 172.21.15.71:34200) 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4b0a00640 1 -- 172.21.15.71:0/1810860337 learned_addr learned my addr 172.21.15.71:0/1810860337 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917c1e0 0x55e31918f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40716/0 (socket says 172.21.15.71:40716) 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 -- 172.21.15.71:0/1810860337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 msgr2=0x55e31918a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e31918a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 -- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917c1e0 msgr2=0x55e31918f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917c1e0 0x55e31918f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 -- 172.21.15.71:0/1810860337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e31918ae10 con 0x55e31917cb00 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4ab400640 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917cb00 0x55e31918f710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe4980ecba0 tx=0x7fe49803b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c17d9dc137383a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:23.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe495600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:42:23.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe495600640 1 -- 172.21.15.71:0/1810860337 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498049040 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe495600640 1 -- 172.21.15.71:0/1810860337 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9438+0+0 (secure 0 0 0) 0x7fe4980b8090 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.092+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4a4004530 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe495600640 1 -- 172.21.15.71:0/1810860337 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe4980bba20 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 10 client.17031 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55e319201970 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe495600640 1 -- 172.21.15.71:0/1810860337 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4a4004530 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe495600640 1 -- 172.21.15.71:0/1810860337 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7fe4980ee070 con 0x55e31917cb00 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 10 client.17031 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 20 client.17031 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e319202200 0x55e319204620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:23.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 4 client.17031 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:42:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e3190e9b80 con 0x55e319202200 2024-09-05T12:42:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.093+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e319202200 0x55e319204620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.094+0000 7fe4abe00640 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e319202200 0x55e319204620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e83ed582c96f3f95 server_cookie=d51984a22842a7e7 in_seq=0 out_seq=0 2024-09-05T12:42:23.100 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.094+0000 7fe495600640 10 client.17031 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.130+0000 7fe495600640 1 -- 172.21.15.71:0/1810860337 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e3190e9b80 con 0x55e319202200 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.130+0000 7fe495600640 10 client.17031 handle_command_reply: tid=0 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.130+0000 7fe4b58b3740 1 client.17031 shutdown 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917cb00 msgr2=0x55e31918f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917cb00 0x55e31918f710 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe4980ecba0 tx=0x7fe49803b640 comp rx=0 tx=0).stop 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 shutdown_connections 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55e319202200 0x55e319204620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3191873a0 0x55e31918a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e31917cb00 0x55e31918f710 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 --2- 172.21.15.71:0/1810860337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e31917c1e0 0x55e31918f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe495600640 0 client.17031 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 >> 172.21.15.71:0/1810860337 conn(0x55e319178750 msgr2=0x55e31917dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 shutdown_connections 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 1 -- 172.21.15.71:0/1810860337 wait complete. 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.131+0000 7fe4b58b3740 20 client.17031 trim_cache size 0 max 16384 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:42:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:42:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4900a2da0 msgr2=0x7fe4900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4900a2da0 0x7fe4900a5260 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009f70 tx=0x7fe49c005f90 comp rx=0 tx=0).stop 2024-09-05T12:42:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 msgr2=0x7fe4ac19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 0x7fe4ac19aad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe498005940 tx=0x7fe4980040c0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 shutdown_connections 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4ac19ee50 0x7fe4ac19f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe4900a2da0 0x7fe4900a5260 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4ac109080 0x7fe4ac19aad0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 --2- 172.21.15.71:0/2262970690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4ac103f30 0x7fe4ac19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 >> 172.21.15.71:0/2262970690 conn(0x7fe4ac0ff980 msgr2=0x7fe4ac100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 shutdown_connections 2024-09-05T12:42:23.135 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:23.132+0000 7fe4b2a00640 1 -- 172.21.15.71:0/2262970690 wait complete. 2024-09-05T12:42:23.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26054 20.46 MB/sec warmup 77 sec latency 32655.981 ms 2024-09-05T12:42:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:23 smithi071 ceph-mon[29845]: pgmap v459: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-05T12:42:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:23 smithi123 ceph-mon[34321]: pgmap v459: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-05T12:42:24.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:23 smithi174 ceph-mon[34313]: pgmap v459: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 15 op/s 2024-09-05T12:42:24.770 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26054 20.19 MB/sec warmup 78 sec latency 33656.142 ms 2024-09-05T12:42:25.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26054 19.94 MB/sec warmup 79 sec latency 34656.286 ms 2024-09-05T12:42:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:25 smithi071 ceph-mon[29845]: pgmap v460: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-05T12:42:26.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:25 smithi123 ceph-mon[34321]: pgmap v460: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-05T12:42:26.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:25 smithi174 ceph-mon[34313]: pgmap v460: 129 pgs: 129 active+clean; 414 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-09-05T12:42:26.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 26372 19.94 MB/sec warmup 80 sec latency 35280.638 ms 2024-09-05T12:42:27.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 27428 20.40 MB/sec warmup 81 sec latency 180.800 ms 2024-09-05T12:42:28.103 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:27 smithi071 ceph-mon[29845]: pgmap v461: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 289 KiB/s wr, 29 op/s 2024-09-05T12:42:28.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:27 smithi123 ceph-mon[34321]: pgmap v461: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 289 KiB/s wr, 29 op/s 2024-09-05T12:42:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:27 smithi174 ceph-mon[34313]: pgmap v461: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 289 KiB/s wr, 29 op/s 2024-09-05T12:42:28.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 27798 20.48 MB/sec warmup 82 sec latency 257.464 ms 2024-09-05T12:42:29.160 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:28 smithi071 ceph-mon[29845]: osdmap e113: 12 total, 12 up, 12 in 2024-09-05T12:42:29.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:28 smithi123 ceph-mon[34321]: osdmap e113: 12 total, 12 up, 12 in 2024-09-05T12:42:29.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:28 smithi174 ceph-mon[34313]: osdmap e113: 12 total, 12 up, 12 in 2024-09-05T12:42:29.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 28202 20.44 MB/sec warmup 83 sec latency 460.677 ms 2024-09-05T12:42:29.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:29 smithi071 ceph-mon[29845]: pgmap v463: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 339 KiB/s wr, 28 op/s 2024-09-05T12:42:30.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:29 smithi123 ceph-mon[34321]: pgmap v463: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 339 KiB/s wr, 28 op/s 2024-09-05T12:42:30.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:29 smithi174 ceph-mon[34313]: pgmap v463: 129 pgs: 129 active+clean; 416 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 339 KiB/s wr, 28 op/s 2024-09-05T12:42:30.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 29044 20.62 MB/sec warmup 84 sec latency 183.508 ms 2024-09-05T12:42:31.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 30081 21.15 MB/sec warmup 85 sec latency 60.246 ms 2024-09-05T12:42:32.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:31 smithi071 ceph-mon[29845]: pgmap v464: 129 pgs: 129 active+clean; 430 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-09-05T12:42:32.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:31 smithi123 ceph-mon[34321]: pgmap v464: 129 pgs: 129 active+clean; 430 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-09-05T12:42:32.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:31 smithi174 ceph-mon[34313]: pgmap v464: 129 pgs: 129 active+clean; 430 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-09-05T12:42:32.771 INFO:tasks.workunit.client.0.smithi071.stdout: 10 31018 21.40 MB/sec warmup 86 sec latency 113.049 ms 2024-09-05T12:42:33.687 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 14.5 2024-09-05T12:42:33.688 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:42:33.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 32003 21.79 MB/sec warmup 87 sec latency 48.732 ms 2024-09-05T12:42:34.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:33 smithi071 ceph-mon[29845]: pgmap v465: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-05T12:42:34.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:33 smithi123 ceph-mon[34321]: pgmap v465: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-05T12:42:34.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:33 smithi174 ceph-mon[34313]: pgmap v465: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-05T12:42:34.338 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:42:34.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 33138 22.32 MB/sec warmup 88 sec latency 47.150 ms 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.625+0000 7f7c7da00640 1 -- 172.21.15.71:0/3119354381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c781079c0 msgr2=0x7f7c78107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.625+0000 7f7c7da00640 1 --2- 172.21.15.71:0/3119354381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c781079c0 0x7f7c78107da0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009930 tx=0x7f7c6802a440 comp rx=0 tx=0).stop 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 -- 172.21.15.71:0/3119354381 shutdown_connections 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 --2- 172.21.15.71:0/3119354381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c78102360 0x7f7c7810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 --2- 172.21.15.71:0/3119354381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781019c0 0x7f7c78101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 --2- 172.21.15.71:0/3119354381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c781079c0 0x7f7c78107da0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 -- 172.21.15.71:0/3119354381 >> 172.21.15.71:0/3119354381 conn(0x7f7c780fd740 msgr2=0x7f7c780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:35.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 -- 172.21.15.71:0/3119354381 shutdown_connections 2024-09-05T12:42:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.626+0000 7f7c7da00640 1 -- 172.21.15.71:0/3119354381 wait complete. 2024-09-05T12:42:35.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.627+0000 7f7c7da00640 1 Processor -- start 2024-09-05T12:42:35.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.627+0000 7f7c7da00640 1 -- start start 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c781019c0 0x7f7c7819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7da00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c78102360 0x7f7c7819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7da00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 0x7f7c7819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7da00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7c78111e10 con 0x7f7c78102360 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c78111c90 con 0x7f7c781019c0 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7da00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c78111f90 con 0x7f7c781079c0 2024-09-05T12:42:35.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 0x7f7c7819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 0x7f7c7819edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59780/0 (socket says 172.21.15.71:59780) 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c7d000640 1 -- 172.21.15.71:0/1255811492 learned_addr learned my addr 172.21.15.71:0/1255811492 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.628+0000 7f7c77400640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c78102360 0x7f7c7819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.629+0000 7f7c7d000640 1 -- 172.21.15.71:0/1255811492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c781019c0 msgr2=0x7f7c7819a500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.629+0000 7f7c7d000640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c781019c0 0x7f7c7819a500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.629+0000 7f7c7d000640 1 -- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c78102360 msgr2=0x7f7c7819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:35.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.629+0000 7f7c7d000640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c78102360 0x7f7c7819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.629+0000 7f7c7d000640 1 -- 172.21.15.71:0/1255811492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7819f4b0 con 0x7f7c781079c0 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.629+0000 7f7c7d000640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 0x7f7c7819edd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000cad0 tx=0x7f7c7000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.630+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000ae20 con 0x7f7c781079c0 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.630+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c70005020 con 0x7f7c781079c0 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.630+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000f6f0 con 0x7f7c781079c0 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.631+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7819f7a0 con 0x7f7c781079c0 2024-09-05T12:42:35.635 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.631+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c78108820 con 0x7f7c781079c0 2024-09-05T12:42:35.636 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.632+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c78103100 con 0x7f7c781079c0 2024-09-05T12:42:35.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.636+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7c700051d0 con 0x7f7c781079c0 2024-09-05T12:42:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.638+0000 7f7c74c00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7c5c0a2d70 0x7f7c5c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:35.639 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.638+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f7c700cfb00 con 0x7f7c781079c0 2024-09-05T12:42:35.640 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.640+0000 7f7c77e00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7c5c0a2d70 0x7f7c5c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:35.642 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.642+0000 7f7c77e00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7c5c0a2d70 0x7f7c5c0a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c680096c0 tx=0x7f7c68005fd0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:35.644 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:35.643+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c7001d020 con 0x7f7c781079c0 2024-09-05T12:42:35.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 33894 22.52 MB/sec warmup 89 sec latency 72.181 ms 2024-09-05T12:42:36.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.030+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c78108ae0 con 0x7f7c781079c0 2024-09-05T12:42:36.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.032+0000 7f7c74c00640 1 -- 172.21.15.71:0/1255811492 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11998 (secure 0 0 0) 0x7f7c700cfe20 con 0x7f7c781079c0 2024-09-05T12:42:36.033 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 63 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.035+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7c5c0a2d70 msgr2=0x7f7c5c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.035+0000 7f7c7da00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7c5c0a2d70 0x7f7c5c0a5230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7c680096c0 tx=0x7f7c68005fd0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.035+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 msgr2=0x7f7c7819edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.035+0000 7f7c7da00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 0x7f7c7819edd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000cad0 tx=0x7f7c7000cf90 comp rx=0 tx=0).stop 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.036+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 shutdown_connections 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.036+0000 7f7c7da00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c781079c0 0x7f7c7819edd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.036+0000 7f7c7da00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7c5c0a2d70 0x7f7c5c0a5230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.036+0000 7f7c7da00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c78102360 0x7f7c7819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.036+0000 7f7c7da00640 1 --2- 172.21.15.71:0/1255811492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c781019c0 0x7f7c7819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.036+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 >> 172.21.15.71:0/1255811492 conn(0x7f7c780fd740 msgr2=0x7f7c7810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.039 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":63,"btime":"2024-09-05T12:41:51:592773+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:41:51.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:42:36.039 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.037+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 shutdown_connections 2024-09-05T12:42:36.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:36.037+0000 7f7c7da00640 1 -- 172.21.15.71:0/1255811492 wait complete. 2024-09-05T12:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.316+0000 7fa4ed000640 1 -- 172.21.15.71:0/3969967449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e8109b90 msgr2=0x7fa4e8109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.316+0000 7fa4ed000640 1 --2- 172.21.15.71:0/3969967449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e8109b90 0x7fa4e8109f70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009860 tx=0x7fa4d402a360 comp rx=0 tx=0).stop 2024-09-05T12:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.316+0000 7fa4ed000640 1 -- 172.21.15.71:0/3969967449 shutdown_connections 2024-09-05T12:42:36.317 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.316+0000 7fa4ed000640 1 --2- 172.21.15.71:0/3969967449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8104490 0x7fa4e81109c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.317+0000 7fa4ed000640 1 --2- 172.21.15.71:0/3969967449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8103af0 0x7fa4e8103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.317+0000 7fa4ed000640 1 --2- 172.21.15.71:0/3969967449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e8109b90 0x7fa4e8109f70 secure :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009860 tx=0x7fa4d402a360 comp rx=0 tx=0).stop 2024-09-05T12:42:36.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.317+0000 7fa4ed000640 1 -- 172.21.15.71:0/3969967449 >> 172.21.15.71:0/3969967449 conn(0x7fa4e80ff7d0 msgr2=0x7fa4e8101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.319+0000 7fa4ed000640 1 -- 172.21.15.71:0/3969967449 shutdown_connections 2024-09-05T12:42:36.320 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.319+0000 7fa4ed000640 1 -- 172.21.15.71:0/3969967449 wait complete. 2024-09-05T12:42:36.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.322+0000 7fa4ed000640 1 Processor -- start 2024-09-05T12:42:36.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.323+0000 7fa4ed000640 1 -- start start 2024-09-05T12:42:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.324+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8103af0 0x7fa4e810db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.324+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 0x7fa4e810e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.324+0000 7fa4ed000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 0x7fa4e81aac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.325+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 0x7fa4e810e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.325+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 0x7fa4e810e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59804/0 (socket says 172.21.15.71:59804) 2024-09-05T12:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.325+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 0x7fa4e81aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.325+0000 7fa4e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 0x7fa4e81aac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40648/0 (socket says 172.21.15.71:40648) 2024-09-05T12:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.326+0000 7fa4e6a00640 1 -- 172.21.15.71:0/2397025312 learned_addr learned my addr 172.21.15.71:0/2397025312 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.326+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8113f40 con 0x7fa4e8103af0 2024-09-05T12:42:36.327 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.326+0000 7fa4e7400640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8103af0 0x7fa4e810db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.326+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8113dc0 con 0x7fa4e810e5a0 2024-09-05T12:42:36.328 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.327+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81140c0 con 0x7fa4e8104490 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e7e00640 1 -- 172.21.15.71:0/2397025312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 msgr2=0x7fa4e810e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e7e00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 0x7fa4e810e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e7e00640 1 -- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8103af0 msgr2=0x7fa4e810db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e7e00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8103af0 0x7fa4e810db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e7e00640 1 -- 172.21.15.71:0/2397025312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e81ab160 con 0x7fa4e810e5a0 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 0x7fa4e810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:42:36.329 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.328+0000 7fa4e7400640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8103af0 0x7fa4e810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:42:36.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.329+0000 7fa4e7e00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 0x7fa4e81aac20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000ed70 tx=0x7fa4e000c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.330+0000 7fa4dfe00640 1 -- 172.21.15.71:0/2397025312 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e000ee60 con 0x7fa4e810e5a0 2024-09-05T12:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.330+0000 7fa4dfe00640 1 -- 172.21.15.71:0/2397025312 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e0003b70 con 0x7fa4e810e5a0 2024-09-05T12:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.330+0000 7fa4dfe00640 1 -- 172.21.15.71:0/2397025312 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00178e0 con 0x7fa4e810e5a0 2024-09-05T12:42:36.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.330+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e81ab390 con 0x7fa4e810e5a0 2024-09-05T12:42:36.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.330+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e81ab860 con 0x7fa4e810e5a0 2024-09-05T12:42:36.336 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.335+0000 7fa4f148a740 1 Processor -- start 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.335+0000 7fa4dfe00640 1 -- 172.21.15.71:0/2397025312 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa4e0017a80 con 0x7fa4e810e5a0 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.335+0000 7fa4f148a740 1 -- start start 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1ee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4f148a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe154100 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4f148a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe0fc1b0 con 0x55d0fe1eecd0 2024-09-05T12:42:36.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4f148a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe1500c0 con 0x55d0fe1f9570 2024-09-05T12:42:36.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.336+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40662/0 (socket says 172.21.15.71:40662) 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e7400640 1 -- 172.21.15.71:0/1189195894 learned_addr learned my addr 172.21.15.71:0/1189195894 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4dfe00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4cc0a2ec0 0x7fa4cc0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4dfe00640 1 -- 172.21.15.71:0/2397025312 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7fa4e00cf8c0 con 0x7fa4e810e5a0 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e7400640 1 -- 172.21.15.71:0/1189195894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 msgr2=0x55d0fe1fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e7400640 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1fb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.337+0000 7fa4e7400640 1 -- 172.21.15.71:0/1189195894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 msgr2=0x55d0fe1ee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.338+0000 7fa4e7400640 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1ee790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.338+0000 7fa4e7400640 1 -- 172.21.15.71:0/1189195894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fde67210 con 0x55d0fe1eecd0 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.338+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:42:36.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.338+0000 7fa4e7400640 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f9030 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8104f80 tx=0x7fa4d4009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c83b99bc60eea68 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.338+0000 7fa4e7e00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4cc0a2ec0 0x7fa4cc0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.339+0000 7fa4dcc00640 1 -- 172.21.15.71:0/1189195894 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4041070 con 0x55d0fe1eecd0 2024-09-05T12:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.339+0000 7fa4dcc00640 1 -- 172.21.15.71:0/1189195894 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4030590 con 0x55d0fe1eecd0 2024-09-05T12:42:36.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.339+0000 7fa4dcc00640 1 -- 172.21.15.71:0/1189195894 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4037040 con 0x55d0fe1eecd0 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.340+0000 7fa4f148a740 1 -- 172.21.15.71:0/1189195894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 msgr2=0x55d0fe1f9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.340+0000 7fa4f148a740 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f9030 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8104f80 tx=0x7fa4d4009400 comp rx=0 tx=0).stop 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.341+0000 7fa4f148a740 1 -- 172.21.15.71:0/1189195894 shutdown_connections 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.341+0000 7fa4f148a740 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1fb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.341+0000 7fa4f148a740 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f9030 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.341+0000 7fa4f148a740 1 --2- 172.21.15.71:0/1189195894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.341+0000 7fa4f148a740 1 -- 172.21.15.71:0/1189195894 >> 172.21.15.71:0/1189195894 conn(0x55d0fe1ea920 msgr2=0x55d0fe1ead00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.341+0000 7fa4e7e00640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4cc0a2ec0 0x7fa4cc0a5380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00984d0 tx=0x7fa4e0005a50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:36.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.342+0000 7fa4f148a740 1 -- 172.21.15.71:0/1189195894 shutdown_connections 2024-09-05T12:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.342+0000 7fa4f148a740 1 -- 172.21.15.71:0/1189195894 wait complete. 2024-09-05T12:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.343+0000 7fa4f148a740 1 -- start start 2024-09-05T12:42:36.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.343+0000 7fa4f148a740 1 Processor -- start 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe20d150 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe0dd090 con 0x55d0fe1eecd0 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d0fe20d2d0 con 0x55d0fe1f9570 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4f148a740 10 client.0 _ll_register_callbacks cb 0x55d0fe131f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:36.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:40674/0 (socket says 172.21.15.71:40674) 2024-09-05T12:42:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.344+0000 7fa4e7400640 1 -- 172.21.15.71:0/702703987 learned_addr learned my addr 172.21.15.71:0/702703987 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.345+0000 7fa4e7400640 1 -- 172.21.15.71:0/702703987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 msgr2=0x55d0fe1f7000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:42:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.345+0000 7fa4e7400640 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1f7000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.345+0000 7fa4e7400640 1 -- 172.21.15.71:0/702703987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 msgr2=0x55d0fe1f6580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:42:36.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.345+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.345+0000 7fa4e7400640 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.345+0000 7fa4e7400640 1 -- 172.21.15.71:0/702703987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fdf33620 con 0x55d0fe1eecd0 2024-09-05T12:42:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.346+0000 7fa4e7400640 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f6ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4035040 tx=0x7fa4d402a9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf2a6f3cf9332fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.346+0000 7fa4d1600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:42:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.347+0000 7fa4d1600640 1 -- 172.21.15.71:0/702703987 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4041070 con 0x55d0fe1eecd0 2024-09-05T12:42:36.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.347+0000 7fa4d1600640 1 -- 172.21.15.71:0/702703987 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7fa4d4004450 con 0x55d0fe1eecd0 2024-09-05T12:42:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.347+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d4030590 con 0x55d0fe1eecd0 2024-09-05T12:42:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.348+0000 7fa4d1600640 1 -- 172.21.15.71:0/702703987 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa4d4037040 con 0x55d0fe1eecd0 2024-09-05T12:42:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.348+0000 7fa4f148a740 10 client.27056 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:36.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.348+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d0fe218db0 con 0x55d0fe1eecd0 2024-09-05T12:42:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4d1600640 1 -- 172.21.15.71:0/702703987 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4030590 con 0x55d0fe1eecd0 2024-09-05T12:42:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4d1600640 1 -- 172.21.15.71:0/702703987 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7fa4d4004c20 con 0x55d0fe1eecd0 2024-09-05T12:42:36.350 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4f148a740 10 client.27056 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4f148a740 10 client.27056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4f148a740 20 client.27056 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:36.351 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4f148a740 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe21a100 0x55d0fe21c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4f148a740 4 client.27056 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.350+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0fe161830 con 0x55d0fe21a100 2024-09-05T12:42:36.352 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.351+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe21a100 0x55d0fe21c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.352+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe21a100 0x55d0fe21c4f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42fc4197f27cc799 server_cookie=5c279349e3d0d684 in_seq=0 out_seq=0 2024-09-05T12:42:36.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.354+0000 7fa4d1600640 10 client.27056 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.354+0000 7fa4d1600640 1 -- 172.21.15.71:0/702703987 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0fe161830 con 0x55d0fe21a100 2024-09-05T12:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.354+0000 7fa4d1600640 10 client.27056 handle_command_reply: tid=0 2024-09-05T12:42:36.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.355+0000 7fa4f148a740 1 client.27056 shutdown 2024-09-05T12:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 msgr2=0x55d0fe1f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f6ac0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4035040 tx=0x7fa4d402a9e0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 shutdown_connections 2024-09-05T12:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe21a100 0x55d0fe21c4f0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe1f6ac0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 --2- 172.21.15.71:0/702703987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe1f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.356+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 >> 172.21.15.71:0/702703987 conn(0x55d0fe1ea920 msgr2=0x55d0fe1f0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.357+0000 7fa4d1600640 0 client.27056 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.357+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 shutdown_connections 2024-09-05T12:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.357+0000 7fa4f148a740 1 -- 172.21.15.71:0/702703987 wait complete. 2024-09-05T12:42:36.359 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.358+0000 7fa4f148a740 20 client.27056 trim_cache size 0 max 16384 2024-09-05T12:42:36.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 Processor -- start 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 -- start start 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe221af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe222030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81ad240 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81aef60 con 0x55d0fe1eecd0 2024-09-05T12:42:36.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.382+0000 7fa4f148a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81ad970 con 0x55d0fe1f9570 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe221af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe221af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38170/0 (socket says 172.21.15.71:38170) 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 -- 172.21.15.71:0/3036038411 learned_addr learned my addr 172.21.15.71:0/3036038411 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 -- 172.21.15.71:0/3036038411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 msgr2=0x55d0fe1efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 -- 172.21.15.71:0/3036038411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 msgr2=0x55d0fe222030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:36.385 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe222030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.383+0000 7fa4e6a00640 1 -- 172.21.15.71:0/3036038411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0fe1e4c20 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.384+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe221af0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8004700 tx=0x7fa4d8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4733bb948df2860f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.384+0000 7fa4d3e00640 1 -- 172.21.15.71:0/3036038411 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8015170 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.384+0000 7fa4d3e00640 1 -- 172.21.15.71:0/3036038411 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d80159d0 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.384+0000 7fa4d3e00640 1 -- 172.21.15.71:0/3036038411 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d80038e0 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 -- 172.21.15.71:0/3036038411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 msgr2=0x55d0fe221af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe221af0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8004700 tx=0x7fa4d8005f90 comp rx=0 tx=0).stop 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 -- 172.21.15.71:0/3036038411 shutdown_connections 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1f9570 0x55d0fe1efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1eecd0 0x55d0fe222030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3036038411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe221af0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 -- 172.21.15.71:0/3036038411 >> 172.21.15.71:0/3036038411 conn(0x55d0fe1ea920 msgr2=0x55d0fe1f4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 -- 172.21.15.71:0/3036038411 shutdown_connections 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.385+0000 7fa4f148a740 1 -- 172.21.15.71:0/3036038411 wait complete. 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.386+0000 7fa4f148a740 1 -- start start 2024-09-05T12:42:36.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.386+0000 7fa4f148a740 1 Processor -- start 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe20dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 0x55d0fe20e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1f9570 0x55d0fe282ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81ae890 con 0x55d0fe1ee3b0 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e81ae100 con 0x55d0fe1f9570 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e8073ad0 con 0x55d0fe1eecd0 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4f148a740 10 client.0 _ll_register_callbacks cb 0x7fa4d403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe20dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 0x55d0fe20e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4e7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 0x55d0fe20e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59832/0 (socket says 172.21.15.71:59832) 2024-09-05T12:42:36.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4e7400640 1 -- 172.21.15.71:0/3592499297 learned_addr learned my addr 172.21.15.71:0/3592499297 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1f9570 0x55d0fe282ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.387+0000 7fa4e7400640 1 -- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1f9570 msgr2=0x55d0fe282ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4e7400640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1f9570 0x55d0fe282ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4e7400640 1 -- 172.21.15.71:0/3592499297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 msgr2=0x55d0fe20dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4e7400640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe20dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.391 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4e7400640 1 -- 172.21.15.71:0/3592499297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0fe2830a0 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1f9570 0x55d0fe282ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4e7400640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 0x55d0fe20e000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40036e0 tx=0x7fa4d4030f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ef3f136e70857f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4d1600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4d1600640 1 -- 172.21.15.71:0/3592499297 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4041040 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4d1600640 1 -- 172.21.15.71:0/3592499297 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7fa4d4004460 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.388+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d80159d0 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.389+0000 7fa4d1600640 1 -- 172.21.15.71:0/3592499297 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fa4d4030590 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.389+0000 7fa4f148a740 10 client.27106 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.389+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d0fe284390 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.389+0000 7fa4d1600640 1 -- 172.21.15.71:0/3592499297 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d4005550 con 0x55d0fe1eecd0 2024-09-05T12:42:36.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.389+0000 7fa4d1600640 1 -- 172.21.15.71:0/3592499297 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7fa4d4004930 con 0x55d0fe1eecd0 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4f148a740 10 client.27106 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4f148a740 10 client.27106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4f148a740 20 client.27106 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe284c00 0x55d0fe287020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4f148a740 4 client.27106 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fa4d80159d0 con 0x55d0fe284c00 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe284c00 0x55d0fe287020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4e6a00640 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe284c00 0x55d0fe287020 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b45995f7f9a91bb server_cookie=8578d6e6eae8a83a in_seq=0 out_seq=0 2024-09-05T12:42:36.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.390+0000 7fa4d1600640 10 client.27106 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:36.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:36 smithi071 ceph-mon[29845]: pgmap v466: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-05T12:42:36.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:36 smithi071 ceph-mon[29845]: osdmap e114: 12 total, 12 up, 12 in 2024-09-05T12:42:36.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:36 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1255811492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:42:36.615 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4d1600640 1 -- 172.21.15.71:0/3592499297 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fa4d80159d0 con 0x55d0fe284c00 2024-09-05T12:42:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4d1600640 10 client.27106 handle_command_reply: tid=0 2024-09-05T12:42:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 client.27106 shutdown 2024-09-05T12:42:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 msgr2=0x55d0fe20e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 0x55d0fe20e000 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa4d40036e0 tx=0x7fa4d4030f90 comp rx=0 tx=0).stop 2024-09-05T12:42:36.695 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:36 smithi123 ceph-mon[34321]: pgmap v466: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-05T12:42:36.695 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:36 smithi123 ceph-mon[34321]: osdmap e114: 12 total, 12 up, 12 in 2024-09-05T12:42:36.695 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:36 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1255811492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 shutdown_connections 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55d0fe284c00 0x55d0fe287020 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d0fe1f9570 0x55d0fe282ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d0fe1eecd0 0x55d0fe20e000 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 --2- 172.21.15.71:0/3592499297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d0fe1ee3b0 0x55d0fe20dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 >> 172.21.15.71:0/3592499297 conn(0x55d0fe1ea920 msgr2=0x55d0fe1f6520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4d1600640 0 client.27106 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.614+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 shutdown_connections 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.615+0000 7fa4f148a740 1 -- 172.21.15.71:0/3592499297 wait complete. 2024-09-05T12:42:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.615+0000 7fa4f148a740 20 client.27106 trim_cache size 0 max 16384 2024-09-05T12:42:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:42:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:42:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:42:36.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:42:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4cc0a2ec0 msgr2=0x7fa4cc0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4cc0a2ec0 0x7fa4cc0a5380 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00984d0 tx=0x7fa4e0005a50 comp rx=0 tx=0).stop 2024-09-05T12:42:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 msgr2=0x7fa4e81aac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:36.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 0x7fa4e81aac20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa4e000ed70 tx=0x7fa4e000c6e0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 shutdown_connections 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4e810e5a0 0x7fa4e81aac20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4e8104490 0x7fa4e810e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4cc0a2ec0 0x7fa4cc0a5380 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 --2- 172.21.15.71:0/2397025312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4e8103af0 0x7fa4e810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.616+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 >> 172.21.15.71:0/2397025312 conn(0x7fa4e80ff7d0 msgr2=0x7fa4e81010e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.617+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 shutdown_connections 2024-09-05T12:42:36.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:36.618+0000 7fa4ed000640 1 -- 172.21.15.71:0/2397025312 wait complete. 2024-09-05T12:42:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:36 smithi174 ceph-mon[34313]: pgmap v466: 129 pgs: 129 active+clean; 439 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-05T12:42:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:36 smithi174 ceph-mon[34313]: osdmap e114: 12 total, 12 up, 12 in 2024-09-05T12:42:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:36 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1255811492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:42:36.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 34626 22.57 MB/sec warmup 90 sec latency 154.552 ms 2024-09-05T12:42:37.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0' 2024-09-05T12:42:37.655 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 134.5 --await-for 45.0 2024-09-05T12:42:37.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 35943 23.23 MB/sec warmup 91 sec latency 60.483 ms 2024-09-05T12:42:38.269 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:42:38.618 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:38 smithi071 ceph-mon[29845]: pgmap v468: 129 pgs: 129 active+clean; 472 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.4 MiB/s wr, 762 op/s 2024-09-05T12:42:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:38 smithi123 ceph-mon[34321]: pgmap v468: 129 pgs: 129 active+clean; 472 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.4 MiB/s wr, 762 op/s 2024-09-05T12:42:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:38 smithi174 ceph-mon[34313]: pgmap v468: 129 pgs: 129 active+clean; 472 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.4 MiB/s wr, 762 op/s 2024-09-05T12:42:38.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36796 23.43 MB/sec warmup 92 sec latency 76.333 ms 2024-09-05T12:42:39.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 -- 172.21.15.71:0/783353187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc0745c0 msgr2=0x7f14dc0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 --2- 172.21.15.71:0/783353187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc0745c0 0x7f14dc0749a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f14cc009900 tx=0x7f14cc02a480 comp rx=0 tx=0).stop 2024-09-05T12:42:39.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 -- 172.21.15.71:0/783353187 shutdown_connections 2024-09-05T12:42:39.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 --2- 172.21.15.71:0/783353187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc10e430 0x7f14dc110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 --2- 172.21.15.71:0/783353187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc074ee0 0x7f14dc10def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 --2- 172.21.15.71:0/783353187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc0745c0 0x7f14dc0749a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 -- 172.21.15.71:0/783353187 >> 172.21.15.71:0/783353187 conn(0x7f14dc0fd8b0 msgr2=0x7f14dc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 -- 172.21.15.71:0/783353187 shutdown_connections 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.003+0000 7f14e3000640 1 -- 172.21.15.71:0/783353187 wait complete. 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 Processor -- start 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 -- start start 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 0x7f14dc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc074ee0 0x7f14dc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc10e430 0x7f14dc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1141f0 con 0x7f14dc10e430 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc114070 con 0x7f14dc074ee0 2024-09-05T12:42:39.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.004+0000 7f14e3000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc114370 con 0x7f14dc0745c0 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 0x7f14dc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14e1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc10e430 0x7f14dc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 0x7f14dc19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59862/0 (socket says 172.21.15.71:59862) 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 -- 172.21.15.71:0/782757438 learned_addr learned my addr 172.21.15.71:0/782757438 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14db400640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc074ee0 0x7f14dc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 -- 172.21.15.71:0/782757438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc074ee0 msgr2=0x7f14dc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc074ee0 0x7f14dc19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 -- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc10e430 msgr2=0x7f14dc19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc10e430 0x7f14dc19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 -- 172.21.15.71:0/782757438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14dc19f620 con 0x7f14dc0745c0 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14db400640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc074ee0 0x7f14dc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14e1000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc10e430 0x7f14dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.005+0000 7f14dbe00640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 0x7f14dc19a670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14cc004770 tx=0x7f14cc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:39.182 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.006+0000 7f14d8c00640 1 -- 172.21.15.71:0/782757438 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc03f070 con 0x7f14dc0745c0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.006+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14dc19f8b0 con 0x7f14dc0745c0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.006+0000 7f14d8c00640 1 -- 172.21.15.71:0/782757438 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14cc031ca0 con 0x7f14dc0745c0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.006+0000 7f14d8c00640 1 -- 172.21.15.71:0/782757438 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc035030 con 0x7f14dc0745c0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.006+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14dc1a71f0 con 0x7f14dc0745c0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.009+0000 7f14d8c00640 1 -- 172.21.15.71:0/782757438 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f14cc042070 con 0x7f14dc0745c0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.009+0000 7f14e5ad0740 1 Processor -- start 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.009+0000 7f14e5ad0740 1 -- start start 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea43b0 0x5637efea4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeaf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efeb1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5637efe0a530 con 0x5637efea4cd0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5637efdb1860 con 0x5637efea43b0 2024-09-05T12:42:39.183 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637efe064f0 con 0x5637efeaf590 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeaf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efeb1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea43b0 0x5637efea4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14db400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efeb1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59864/0 (socket says 172.21.15.71:59864) 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14dbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeaf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38208/0 (socket says 172.21.15.71:38208) 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14db400640 1 -- 172.21.15.71:0/4172389504 learned_addr learned my addr 172.21.15.71:0/4172389504 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e1000640 1 -- 172.21.15.71:0/4172389504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 msgr2=0x5637efeb1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e1000640 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efeb1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e1000640 1 -- 172.21.15.71:0/4172389504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 msgr2=0x5637efeaf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e1000640 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeaf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.010+0000 7f14e1000640 1 -- 172.21.15.71:0/4172389504 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637efb7bb90 con 0x5637efea43b0 2024-09-05T12:42:39.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14e1000640 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea43b0 0x5637efea4790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f14d4005fd0 tx=0x7f14d4011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9135d55985bed8dd server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14d8c00640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f14c00a2cf0 0x7f14c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14d1600640 1 -- 172.21.15.71:0/4172389504 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4004d20 con 0x5637efea43b0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14d1600640 1 -- 172.21.15.71:0/4172389504 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d4003970 con 0x5637efea43b0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14d1600640 1 -- 172.21.15.71:0/4172389504 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d4004480 con 0x5637efea43b0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14d8c00640 1 -- 172.21.15.71:0/782757438 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f14cc0efe00 con 0x7f14dc0745c0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.011+0000 7f14dbe00640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f14c00a2cf0 0x7f14c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 -- 172.21.15.71:0/4172389504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea43b0 msgr2=0x5637efea4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea43b0 0x5637efea4790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f14d4005fd0 tx=0x7f14d4011bf0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14dbe00640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f14c00a2cf0 0x7f14c00a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0ed1c0 tx=0x7f14cc0f38b0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 -- 172.21.15.71:0/4172389504 shutdown_connections 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efeb1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeaf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/4172389504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea43b0 0x5637efea4790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 -- 172.21.15.71:0/4172389504 >> 172.21.15.71:0/4172389504 conn(0x5637efea0940 msgr2=0x5637efea0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 -- 172.21.15.71:0/4172389504 shutdown_connections 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 -- 172.21.15.71:0/4172389504 wait complete. 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.012+0000 7f14e5ad0740 1 -- start start 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 Processor -- start 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efeacc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 0x5637efead150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5637efeb1fb0 con 0x5637efeaf590 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5637efdc4e60 con 0x5637efea4cd0 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5637efeb2130 con 0x5637efea43b0 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e5ad0740 10 client.0 _ll_register_callbacks cb 0x5637efdc9cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.013+0000 7f14e1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 0x5637efead150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 0x5637efead150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38210/0 (socket says 172.21.15.71:38210) 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 -- 172.21.15.71:0/3984246311 learned_addr learned my addr 172.21.15.71:0/3984246311 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14db400640 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efeacc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 -- 172.21.15.71:0/3984246311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 msgr2=0x5637efeac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 -- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 msgr2=0x5637efeacc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efeacc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 -- 172.21.15.71:0/3984246311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637efdd0990 con 0x5637efeaf590 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e1000640 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 0x5637efead150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f14d401b040 tx=0x7f14d4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6715125f147f9e71 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14c6000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14c6000640 1 -- 172.21.15.71:0/3984246311 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d402d060 con 0x5637efeaf590 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14c6000640 1 -- 172.21.15.71:0/3984246311 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f14d4011250 con 0x5637efeaf590 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.014+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d4003970 con 0x5637efeaf590 2024-09-05T12:42:39.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14c6000640 1 -- 172.21.15.71:0/3984246311 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f14d4034050 con 0x5637efeaf590 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14e5ad0740 10 client.17064 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5637efebdde0 con 0x5637efeaf590 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14c6000640 1 -- 172.21.15.71:0/3984246311 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d4003970 con 0x5637efeaf590 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14c6000640 1 -- 172.21.15.71:0/3984246311 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f14d40052d0 con 0x5637efeaf590 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14e5ad0740 10 client.17064 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14e5ad0740 10 client.17064 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.015+0000 7f14e5ad0740 20 client.17064 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.016+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637efebf240 0x5637efec1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.016+0000 7f14e5ad0740 4 client.17064 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.016+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637efdad310 con 0x5637efebf240 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.016+0000 7f14db400640 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637efebf240 0x5637efec1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.017+0000 7f14db400640 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637efebf240 0x5637efec1630 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd9b7068997dd4c server_cookie=ca4f62f09a970b07 in_seq=0 out_seq=0 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.017+0000 7f14c6000640 10 client.17064 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:39.188 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.017+0000 7f14c6000640 1 -- 172.21.15.71:0/3984246311 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5637efdad310 con 0x5637efebf240 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.017+0000 7f14c6000640 10 client.17064 handle_command_reply: tid=0 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.017+0000 7f14e5ad0740 1 client.17064 shutdown 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 msgr2=0x5637efead150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 0x5637efead150 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f14d401b040 tx=0x7f14d4000f40 comp rx=0 tx=0).stop 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 shutdown_connections 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637efebf240 0x5637efec1630 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efeaf590 0x5637efead150 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efeacc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3984246311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 >> 172.21.15.71:0/3984246311 conn(0x5637efea0940 msgr2=0x5637efe9fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14c6000640 0 client.17064 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 shutdown_connections 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3984246311 wait complete. 2024-09-05T12:42:39.189 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.018+0000 7f14e5ad0740 20 client.17064 trim_cache size 0 max 16384 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 Processor -- start 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 -- start start 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea43b0 0x5637efeab000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efec6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efec7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1b11b0 con 0x5637efea43b0 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1b2ed0 con 0x5637efea4cd0 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.036+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1b18e0 con 0x5637efeaf590 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14e1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea43b0 0x5637efeab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14e1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea43b0 0x5637efeab000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38222/0 (socket says 172.21.15.71:38222) 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14e1000640 1 -- 172.21.15.71:0/559873894 learned_addr learned my addr 172.21.15.71:0/559873894 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14db400640 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efec6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.190 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14e1000640 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efec7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14db400640 1 -- 172.21.15.71:0/559873894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 msgr2=0x5637efec7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14db400640 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efec7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14db400640 1 -- 172.21.15.71:0/559873894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea43b0 msgr2=0x5637efeab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14db400640 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea43b0 0x5637efeab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.037+0000 7f14db400640 1 -- 172.21.15.71:0/559873894 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637efea49d0 con 0x5637efea4cd0 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.038+0000 7f14db400640 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efec6e20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5637efec1dc0 tx=0x7f14c8005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa99a1afee4c6fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.038+0000 7f14d0c00640 1 -- 172.21.15.71:0/559873894 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8022070 con 0x5637efea4cd0 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.038+0000 7f14d0c00640 1 -- 172.21.15.71:0/559873894 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c80047a0 con 0x5637efea4cd0 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.038+0000 7f14d0c00640 1 -- 172.21.15.71:0/559873894 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c801f910 con 0x5637efea4cd0 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 -- 172.21.15.71:0/559873894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 msgr2=0x5637efec6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efec6e20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5637efec1dc0 tx=0x7f14c8005f90 comp rx=0 tx=0).stop 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 -- 172.21.15.71:0/559873894 shutdown_connections 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efeaf590 0x5637efec7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.191 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efea4cd0 0x5637efec6e20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/559873894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea43b0 0x5637efeab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 -- 172.21.15.71:0/559873894 >> 172.21.15.71:0/559873894 conn(0x5637efea0940 msgr2=0x5637efe9fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 -- 172.21.15.71:0/559873894 shutdown_connections 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 -- 172.21.15.71:0/559873894 wait complete. 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 -- start start 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.039+0000 7f14e5ad0740 1 Processor -- start 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeb73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeb7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efeaf590 0x5637efeb43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1b2800 con 0x5637efea4cd0 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1b2070 con 0x5637efeaf590 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14dc1af1b0 con 0x5637efea43b0 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e5ad0740 10 client.0 _ll_register_callbacks cb 0x7f14cc0f3780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:39.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeb73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeb73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59896/0 (socket says 172.21.15.71:59896) 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e1000640 1 -- 172.21.15.71:0/3715958567 learned_addr learned my addr 172.21.15.71:0/3715958567 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14db400640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeb7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.040+0000 7f14e1000640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efeaf590 0x5637efeb43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14e1000640 1 -- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efeaf590 msgr2=0x5637efeb43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14e1000640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efeaf590 0x5637efeb43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14e1000640 1 -- 172.21.15.71:0/3715958567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 msgr2=0x5637efeb7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14e1000640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeb7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14e1000640 1 -- 172.21.15.71:0/3715958567 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637efeb7fd0 con 0x5637efea43b0 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14db400640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14e1000640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeb73d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14d4011c30 tx=0x7f14d4005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fc5aa11752ad1dc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.041+0000 7f14c6000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:42:39.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14c6000640 1 -- 172.21.15.71:0/3715958567 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d402d040 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14c6000640 1 -- 172.21.15.71:0/3715958567 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f14d40050f0 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c80047a0 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14c6000640 1 -- 172.21.15.71:0/3715958567 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f14d401d040 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 10 client.27124 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5637eff2a3a0 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14c6000640 1 -- 172.21.15.71:0/3715958567 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c80047a0 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14c6000640 1 -- 172.21.15.71:0/3715958567 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f14d403a420 con 0x5637efea43b0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 10 client.27124 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 10 client.27124 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 20 client.27124 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.042+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637eff2aad0 0x5637eff2cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.043+0000 7f14e5ad0740 4 client.27124 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}] 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.043+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 134.5, "await_for": 45.0}) -- 0x5637efe17c60 con 0x5637eff2aad0 2024-09-05T12:42:39.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.043+0000 7f14db400640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637eff2aad0 0x5637eff2cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:39.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.043+0000 7f14db400640 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637eff2aad0 0x5637eff2cef0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff92fcd617fb501 server_cookie=ba242cfe517e0931 in_seq=0 out_seq=0 2024-09-05T12:42:39.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.043+0000 7f14c6000640 10 client.27124 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:39.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.546+0000 7f14c6000640 1 -- 172.21.15.71:0/3715958567 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5637efe17c60 con 0x5637eff2aad0 2024-09-05T12:42:39.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.546+0000 7f14c6000640 10 client.27124 handle_command_reply: tid=0 2024-09-05T12:42:39.547 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.547+0000 7f14e5ad0740 1 client.27124 shutdown 2024-09-05T12:42:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.547+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 msgr2=0x5637efeb73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.548 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.547+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeb73d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f14d4011c30 tx=0x7f14d4005fd0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.549 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 shutdown_connections 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5637eff2aad0 0x5637eff2cef0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5637efeaf590 0x5637efeb43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637efea4cd0 0x5637efeb7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 --2- 172.21.15.71:0/3715958567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5637efea43b0 0x5637efeb73d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 >> 172.21.15.71:0/3715958567 conn(0x5637efea0940 msgr2=0x5637efea5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14c6000640 0 client.27124 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 shutdown_connections 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.548+0000 7f14e5ad0740 1 -- 172.21.15.71:0/3715958567 wait complete. 2024-09-05T12:42:39.550 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.549+0000 7f14e5ad0740 20 client.27124 trim_cache size 0 max 16384 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 63, 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 46, 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "98bff790": { 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 46, 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:42:39.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 134.5, 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:42:39.552 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f14c00a2cf0 msgr2=0x7f14c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f14c00a2cf0 0x7f14c00a51b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f14cc0ed1c0 tx=0x7f14cc0f38b0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 msgr2=0x7f14dc19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 0x7f14dc19a670 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f14cc004770 tx=0x7f14cc003c10 comp rx=0 tx=0).stop 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 shutdown_connections 2024-09-05T12:42:39.553 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f14c00a2cf0 0x7f14c00a51b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f14dc10e430 0x7f14dc19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14dc074ee0 0x7f14dc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 --2- 172.21.15.71:0/782757438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14dc0745c0 0x7f14dc19a670 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.550+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 >> 172.21.15.71:0/782757438 conn(0x7f14dc0fd8b0 msgr2=0x7f14dc0fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.551+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 shutdown_connections 2024-09-05T12:42:39.554 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:39.551+0000 7f14e3000640 1 -- 172.21.15.71:0/782757438 wait complete. 2024-09-05T12:42:39.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 23.28 MB/sec warmup 93 sec latency 689.612 ms 2024-09-05T12:42:40.583 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:40 smithi071 ceph-mon[29845]: pgmap v469: 129 pgs: 129 active+clean; 472 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 679 op/s 2024-09-05T12:42:40.620 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '98bff790', quiesce took 6.9 seconds. Will release after: 7.6 2024-09-05T12:42:40.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:40 smithi123 ceph-mon[34321]: pgmap v469: 129 pgs: 129 active+clean; 472 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 679 op/s 2024-09-05T12:42:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:40 smithi174 ceph-mon[34313]: pgmap v469: 129 pgs: 129 active+clean; 472 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.6 MiB/s wr, 679 op/s 2024-09-05T12:42:40.772 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 23.03 MB/sec warmup 94 sec latency 1689.702 ms 2024-09-05T12:42:41.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 22.79 MB/sec warmup 95 sec latency 2689.806 ms 2024-09-05T12:42:42.512 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:42 smithi071 ceph-mon[29845]: pgmap v470: 129 pgs: 129 active+clean; 480 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 740 op/s 2024-09-05T12:42:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:42 smithi123 ceph-mon[34321]: pgmap v470: 129 pgs: 129 active+clean; 480 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 740 op/s 2024-09-05T12:42:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:42 smithi174 ceph-mon[34313]: pgmap v470: 129 pgs: 129 active+clean; 480 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.9 MiB/s wr, 740 op/s 2024-09-05T12:42:42.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 22.55 MB/sec warmup 96 sec latency 3689.850 ms 2024-09-05T12:42:43.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 22.32 MB/sec warmup 97 sec latency 4689.891 ms 2024-09-05T12:42:44.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:44 smithi071 ceph-mon[29845]: pgmap v471: 129 pgs: 129 active+clean; 484 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 715 op/s 2024-09-05T12:42:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:44 smithi123 ceph-mon[34321]: pgmap v471: 129 pgs: 129 active+clean; 484 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 715 op/s 2024-09-05T12:42:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:44 smithi174 ceph-mon[34313]: pgmap v471: 129 pgs: 129 active+clean; 484 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 715 op/s 2024-09-05T12:42:44.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 22.09 MB/sec warmup 98 sec latency 5689.957 ms 2024-09-05T12:42:45.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 21.87 MB/sec warmup 99 sec latency 6690.037 ms 2024-09-05T12:42:46.566 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:46 smithi071 ceph-mon[29845]: pgmap v472: 129 pgs: 129 active+clean; 484 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 715 op/s 2024-09-05T12:42:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:46 smithi123 ceph-mon[34321]: pgmap v472: 129 pgs: 129 active+clean; 484 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 715 op/s 2024-09-05T12:42:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:46 smithi174 ceph-mon[34313]: pgmap v472: 129 pgs: 129 active+clean; 484 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.4 MiB/s wr, 715 op/s 2024-09-05T12:42:46.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 21.65 MB/sec warmup 100 sec latency 7690.086 ms 2024-09-05T12:42:47.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 21.44 MB/sec warmup 101 sec latency 8690.138 ms 2024-09-05T12:42:48.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:42:48.423 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:48 smithi071 ceph-mon[29845]: pgmap v473: 129 pgs: 129 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2024-09-05T12:42:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:48 smithi123 ceph-mon[34321]: pgmap v473: 129 pgs: 129 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2024-09-05T12:42:48.701 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:42:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:48 smithi174 ceph-mon[34313]: pgmap v473: 129 pgs: 129 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.6 MiB/s wr, 707 op/s 2024-09-05T12:42:48.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 21.23 MB/sec warmup 102 sec latency 9690.192 ms 2024-09-05T12:42:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 -- 172.21.15.71:0/647679593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 msgr2=0x7f2980104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 --2- 172.21.15.71:0/647679593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f2980104330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2970009900 tx=0x7f297002a410 comp rx=0 tx=0).stop 2024-09-05T12:42:49.747 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 -- 172.21.15.71:0/647679593 shutdown_connections 2024-09-05T12:42:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 --2- 172.21.15.71:0/647679593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29801090a0 0x7f2980110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 --2- 172.21.15.71:0/647679593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2980104900 0x7f2980108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 --2- 172.21.15.71:0/647679593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f2980104330 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.745+0000 7f2985200640 1 -- 172.21.15.71:0/647679593 >> 172.21.15.71:0/647679593 conn(0x7f29800ff9e0 msgr2=0x7f2980101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.746+0000 7f2985200640 1 -- 172.21.15.71:0/647679593 shutdown_connections 2024-09-05T12:42:49.748 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.746+0000 7f2985200640 1 -- 172.21.15.71:0/647679593 wait complete. 2024-09-05T12:42:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.747+0000 7f2985200640 1 Processor -- start 2024-09-05T12:42:49.749 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.747+0000 7f2985200640 1 -- start start 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f2985200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f298019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f2985200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29801090a0 0x7f298019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f2985200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298019efd0 0x7f298019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f2985200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2980114160 con 0x7f29801090a0 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f2985200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2980113fe0 con 0x7f298019efd0 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f2985200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29801142e0 con 0x7f2980103f50 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f297d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29801090a0 0x7f298019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f297de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f298019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f297de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f298019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50098/0 (socket says 172.21.15.71:50098) 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f297de00640 1 -- 172.21.15.71:0/2497900925 learned_addr learned my addr 172.21.15.71:0/2497900925 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:49.750 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.748+0000 7f297e800640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298019efd0 0x7f298019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.749+0000 7f297de00640 1 -- 172.21.15.71:0/2497900925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298019efd0 msgr2=0x7f298019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.749+0000 7f297de00640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298019efd0 0x7f298019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.749+0000 7f297de00640 1 -- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29801090a0 msgr2=0x7f298019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.749+0000 7f297de00640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29801090a0 0x7f298019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.751 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.749+0000 7f297de00640 1 -- 172.21.15.71:0/2497900925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298019fac0 con 0x7f2980103f50 2024-09-05T12:42:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.750+0000 7f297de00640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f298019a710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2970006010 tx=0x7f29700040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.750+0000 7f2976a00640 1 -- 172.21.15.71:0/2497900925 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297003f030 con 0x7f2980103f50 2024-09-05T12:42:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.750+0000 7f2976a00640 1 -- 172.21.15.71:0/2497900925 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2970003ad0 con 0x7f2980103f50 2024-09-05T12:42:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.750+0000 7f2976a00640 1 -- 172.21.15.71:0/2497900925 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2970035070 con 0x7f2980103f50 2024-09-05T12:42:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.750+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298019fd50 con 0x7f2980103f50 2024-09-05T12:42:49.752 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.750+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29801a74c0 con 0x7f2980103f50 2024-09-05T12:42:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.753+0000 7f2987ff4740 1 Processor -- start 2024-09-05T12:42:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 -- start start 2024-09-05T12:42:49.755 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2976a00640 1 -- 172.21.15.71:0/2497900925 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2970042030 con 0x7f2980103f50 2024-09-05T12:42:49.756 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff41c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff426e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff4273a0 0x55fcff429790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fcff382100 con 0x55fcff41cb00 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcff32a1b0 con 0x55fcff41c1e0 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f2987ff4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fcff37e0c0 con 0x55fcff4273a0 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f297de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff426e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f297de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff426e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41188/0 (socket says 172.21.15.71:41188) 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.754+0000 7f297de00640 1 -- 172.21.15.71:0/492512060 learned_addr learned my addr 172.21.15.71:0/492512060 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297d400640 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff4273a0 0x55fcff429790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.757 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297de00640 1 -- 172.21.15.71:0/492512060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff4273a0 msgr2=0x55fcff429790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297de00640 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff4273a0 0x55fcff429790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297de00640 1 -- 172.21.15.71:0/492512060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 msgr2=0x55fcff41c5c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:42:49.758 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297de00640 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff41c5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297de00640 1 -- 172.21.15.71:0/492512060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcff095210 con 0x55fcff41cb00 2024-09-05T12:42:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f297de00640 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff426e60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f29700ece40 tx=0x7f29700f3dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8442b8295f888a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:49.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.755+0000 7f2976a00640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29640a2f00 0x7f29640a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.756+0000 7f2976a00640 1 -- 172.21.15.71:0/2497900925 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f29700efba0 con 0x7f2980103f50 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.756+0000 7f297e800640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29640a2f00 0x7f29640a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.756+0000 7f296b400640 1 -- 172.21.15.71:0/492512060 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700ff020 con 0x55fcff41cb00 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.756+0000 7f296b400640 1 -- 172.21.15.71:0/492512060 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29700f4040 con 0x55fcff41cb00 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.756+0000 7f296b400640 1 -- 172.21.15.71:0/492512060 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700bb810 con 0x55fcff41cb00 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f297e800640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29640a2f00 0x7f29640a53c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2978009e60 tx=0x7f2978009450 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:49.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 -- 172.21.15.71:0/492512060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 msgr2=0x55fcff426e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff426e60 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f29700ece40 tx=0x7f29700f3dd0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 -- 172.21.15.71:0/492512060 shutdown_connections 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff4273a0 0x55fcff429790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff426e60 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 --2- 172.21.15.71:0/492512060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff41c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 -- 172.21.15.71:0/492512060 >> 172.21.15.71:0/492512060 conn(0x55fcff418750 msgr2=0x55fcff418b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:49.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 -- 172.21.15.71:0/492512060 shutdown_connections 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.757+0000 7f2987ff4740 1 -- 172.21.15.71:0/492512060 wait complete. 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 -- start start 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 Processor -- start 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff4242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff424830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff4273a0 0x55fcff424d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fcff429d90 con 0x55fcff4273a0 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fcff30b090 con 0x55fcff41c1e0 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fcff429f10 con 0x55fcff41cb00 2024-09-05T12:42:49.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.758+0000 7f2987ff4740 10 client.0 _ll_register_callbacks cb 0x55fcff35ff30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff4273a0 0x55fcff424d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff4242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff4242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58646/0 (socket says 172.21.15.71:58646) 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 -- 172.21.15.71:0/4205568308 learned_addr learned my addr 172.21.15.71:0/4205568308 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff4273a0 0x55fcff424d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41194/0 (socket says 172.21.15.71:41194) 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297d400640 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff424830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 -- 172.21.15.71:0/4205568308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 msgr2=0x55fcff424830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff424830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 -- 172.21.15.71:0/4205568308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff4273a0 msgr2=0x55fcff424d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:49.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff4273a0 0x55fcff424d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 -- 172.21.15.71:0/4205568308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcff161620 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f297de00640 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff4242f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29700bb9d0 tx=0x7f2970005890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f444a157ba6cc70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f295fe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f295fe00640 1 -- 172.21.15.71:0/4205568308 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700ff070 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f295fe00640 1 -- 172.21.15.71:0/4205568308 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f29700035f0 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.759+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29700f4040 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f295fe00640 1 -- 172.21.15.71:0/4205568308 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f29700bb630 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 10 client.27080 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55fcff429d90 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f295fe00640 1 -- 172.21.15.71:0/4205568308 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29700f4040 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f295fe00640 1 -- 172.21.15.71:0/4205568308 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f29700b8340 con 0x55fcff41c1e0 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 10 client.27080 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:49.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 10 client.27080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 20 client.27080 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff436c70 0x55fcff4390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 4 client.27080 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:42:49.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.760+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcff38f830 con 0x55fcff436c70 2024-09-05T12:42:49.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 21.02 MB/sec warmup 103 sec latency 10690.245 ms 2024-09-05T12:42:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.791+0000 7f297d400640 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff436c70 0x55fcff4390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.897+0000 7f295fe00640 10 client.27080 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:49.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.897+0000 7f297d400640 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff436c70 0x55fcff4390d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ca30d0256cbc1e7 server_cookie=405791f3e901137b in_seq=0 out_seq=0 2024-09-05T12:42:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.910+0000 7f295fe00640 1 -- 172.21.15.71:0/4205568308 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcff38f830 con 0x55fcff436c70 2024-09-05T12:42:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.910+0000 7f295fe00640 10 client.27080 handle_command_reply: tid=0 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.910+0000 7f2987ff4740 1 client.27080 shutdown 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 msgr2=0x55fcff4242f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff4242f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f29700bb9d0 tx=0x7f2970005890 comp rx=0 tx=0).stop 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 shutdown_connections 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff436c70 0x55fcff4390d0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff4273a0 0x55fcff424d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff424830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 --2- 172.21.15.71:0/4205568308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff41c1e0 0x55fcff4242f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 >> 172.21.15.71:0/4205568308 conn(0x55fcff418750 msgr2=0x55fcff41e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f295fe00640 0 client.27080 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 shutdown_connections 2024-09-05T12:42:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 1 -- 172.21.15.71:0/4205568308 wait complete. 2024-09-05T12:42:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.911+0000 7f2987ff4740 20 client.27080 trim_cache size 0 max 16384 2024-09-05T12:42:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.937+0000 7f2987ff4740 1 Processor -- start 2024-09-05T12:42:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.937+0000 7f2987ff4740 1 -- start start 2024-09-05T12:42:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.937+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 0x55fcff426a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff43e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 0x55fcff43eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f2987ff4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b11a0 con 0x55fcff41cb00 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f2987ff4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b2ec0 con 0x55fcff4273a0 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f2987ff4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b18d0 con 0x55fcff41c1e0 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 0x55fcff426a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 0x55fcff426a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50132/0 (socket says 172.21.15.71:50132) 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 -- 172.21.15.71:0/3213011361 learned_addr learned my addr 172.21.15.71:0/3213011361 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297d400640 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff43e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 -- 172.21.15.71:0/3213011361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 msgr2=0x55fcff43eee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 0x55fcff43eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 -- 172.21.15.71:0/3213011361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 msgr2=0x55fcff43e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.938+0000 7f297de00640 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff43e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.939+0000 7f297de00640 1 -- 172.21.15.71:0/3213011361 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcff413d60 con 0x55fcff41c1e0 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.939+0000 7f297de00640 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 0x55fcff426a50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f29700c4410 tx=0x7f29700ba1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6042e0a60c4b4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.939+0000 7f296aa00640 1 -- 172.21.15.71:0/3213011361 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700ff070 con 0x55fcff41c1e0 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.939+0000 7f296aa00640 1 -- 172.21.15.71:0/3213011361 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29700f3420 con 0x55fcff41c1e0 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.939+0000 7f296aa00640 1 -- 172.21.15.71:0/3213011361 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700b8340 con 0x55fcff41c1e0 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.940+0000 7f2987ff4740 1 -- 172.21.15.71:0/3213011361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 msgr2=0x55fcff426a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.940+0000 7f2987ff4740 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 0x55fcff426a50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f29700c4410 tx=0x7f29700ba1e0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 -- 172.21.15.71:0/3213011361 shutdown_connections 2024-09-05T12:42:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 0x55fcff43eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41cb00 0x55fcff43e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 --2- 172.21.15.71:0/3213011361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41c1e0 0x55fcff426a50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 -- 172.21.15.71:0/3213011361 >> 172.21.15.71:0/3213011361 conn(0x55fcff418750 msgr2=0x55fcff4233c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 -- 172.21.15.71:0/3213011361 shutdown_connections 2024-09-05T12:42:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.941+0000 7f2987ff4740 1 -- 172.21.15.71:0/3213011361 wait complete. 2024-09-05T12:42:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.942+0000 7f2987ff4740 1 -- start start 2024-09-05T12:42:49.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.942+0000 7f2987ff4740 1 Processor -- start 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 0x55fcff42f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff42f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 0x55fcff42a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b27f0 con 0x55fcff41c1e0 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29801b2060 con 0x55fcff4273a0 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29801af1a0 con 0x55fcff41cb00 2024-09-05T12:42:49.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f2987ff4740 10 client.0 _ll_register_callbacks cb 0x7f296c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:42:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.943+0000 7f297de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 0x55fcff42f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 0x55fcff42f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41212/0 (socket says 172.21.15.71:41212) 2024-09-05T12:42:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 -- 172.21.15.71:0/1288199513 learned_addr learned my addr 172.21.15.71:0/1288199513 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 -- 172.21.15.71:0/1288199513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 msgr2=0x55fcff42f8f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:42:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff42f8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 -- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 msgr2=0x55fcff42a780 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:42:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 0x55fcff42a780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 -- 172.21.15.71:0/1288199513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcff42afb0 con 0x55fcff41c1e0 2024-09-05T12:42:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.944+0000 7f297de00640 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 0x55fcff42f3b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f29700f3f00 tx=0x7f29700c4ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0925b91cf6aad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f295fe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:42:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f295fe00640 1 -- 172.21.15.71:0/1288199513 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29700c9060 con 0x55fcff41c1e0 2024-09-05T12:42:49.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f295fe00640 1 -- 172.21.15.71:0/1288199513 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f2970003c80 con 0x55fcff41c1e0 2024-09-05T12:42:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29700f3420 con 0x55fcff41c1e0 2024-09-05T12:42:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f295fe00640 1 -- 172.21.15.71:0/1288199513 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f29700ff070 con 0x55fcff41c1e0 2024-09-05T12:42:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f2987ff4740 10 client.17088 fetch_fsmap learned FSMap version 63 2024-09-05T12:42:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.945+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55fcff42afb0 con 0x55fcff41c1e0 2024-09-05T12:42:49.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f295fe00640 1 -- 172.21.15.71:0/1288199513 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29700f3420 con 0x55fcff41c1e0 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f295fe00640 1 -- 172.21.15.71:0/1288199513 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f29700ed180 con 0x55fcff41c1e0 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f2987ff4740 10 client.17088 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f2987ff4740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f2987ff4740 20 client.17088 populate_metadata read hostname 'smithi071' 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f2987ff4740 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff4aa000 0x55fcff4ac420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f2987ff4740 4 client.17088 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fcff389b80 con 0x55fcff4aa000 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.946+0000 7f297d400640 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff4aa000 0x55fcff4ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.947+0000 7f295fe00640 10 client.17088 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:49.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:49.947+0000 7f297d400640 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff4aa000 0x55fcff4ac420 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff011e6748e59820 server_cookie=77682ca3d2f69622 in_seq=0 out_seq=0 2024-09-05T12:42:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.041+0000 7f295fe00640 1 -- 172.21.15.71:0/1288199513 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fcff389b80 con 0x55fcff4aa000 2024-09-05T12:42:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.041+0000 7f295fe00640 10 client.17088 handle_command_reply: tid=0 2024-09-05T12:42:50.044 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.041+0000 7f2987ff4740 1 client.17088 shutdown 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 msgr2=0x55fcff42f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 0x55fcff42f3b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f29700f3f00 tx=0x7f29700c4ac0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 shutdown_connections 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55fcff4aa000 0x55fcff4ac420 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fcff4273a0 0x55fcff42a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fcff41cb00 0x55fcff42f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f295fe00640 0 client.17088 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 --2- 172.21.15.71:0/1288199513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fcff41c1e0 0x55fcff42f3b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 >> 172.21.15.71:0/1288199513 conn(0x55fcff418750 msgr2=0x55fcff41dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 shutdown_connections 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.042+0000 7f2987ff4740 1 -- 172.21.15.71:0/1288199513 wait complete. 2024-09-05T12:42:50.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.043+0000 7f2987ff4740 20 client.17088 trim_cache size 0 max 16384 2024-09-05T12:42:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:42:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:42:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:42:50.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:42:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29640a2f00 msgr2=0x7f29640a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29640a2f00 0x7f29640a53c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2978009e60 tx=0x7f2978009450 comp rx=0 tx=0).stop 2024-09-05T12:42:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 msgr2=0x7f298019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f298019a710 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2970006010 tx=0x7f29700040c0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 shutdown_connections 2024-09-05T12:42:50.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f298019efd0 0x7f298019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29640a2f00 0x7f29640a53c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29801090a0 0x7f298019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 --2- 172.21.15.71:0/2497900925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2980103f50 0x7f298019a710 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 >> 172.21.15.71:0/2497900925 conn(0x7f29800ff9e0 msgr2=0x7f2980100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.044+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 shutdown_connections 2024-09-05T12:42:50.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:42:50.045+0000 7f2985200640 1 -- 172.21.15.71:0/2497900925 wait complete. 2024-09-05T12:42:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:50 smithi071 ceph-mon[29845]: pgmap v474: 129 pgs: 129 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 379 op/s 2024-09-05T12:42:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:50 smithi123 ceph-mon[34321]: pgmap v474: 129 pgs: 129 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 379 op/s 2024-09-05T12:42:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:50 smithi174 ceph-mon[34313]: pgmap v474: 129 pgs: 129 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.0 MiB/s wr, 379 op/s 2024-09-05T12:42:50.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 20.82 MB/sec warmup 104 sec latency 11690.337 ms 2024-09-05T12:42:51.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 20.62 MB/sec warmup 105 sec latency 12690.385 ms 2024-09-05T12:42:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:52 smithi071 ceph-mon[29845]: pgmap v475: 129 pgs: 129 active+clean; 518 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.7 MiB/s wr, 517 op/s 2024-09-05T12:42:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:52 smithi123 ceph-mon[34321]: pgmap v475: 129 pgs: 129 active+clean; 518 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.7 MiB/s wr, 517 op/s 2024-09-05T12:42:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:52 smithi174 ceph-mon[34313]: pgmap v475: 129 pgs: 129 active+clean; 518 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.7 MiB/s wr, 517 op/s 2024-09-05T12:42:52.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 20.42 MB/sec warmup 106 sec latency 13690.453 ms 2024-09-05T12:42:53.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 20.23 MB/sec warmup 107 sec latency 14690.494 ms 2024-09-05T12:42:54.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:54 smithi071 ceph-mon[29845]: pgmap v476: 129 pgs: 129 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.7 MiB/s wr, 334 op/s 2024-09-05T12:42:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:54 smithi123 ceph-mon[34321]: pgmap v476: 129 pgs: 129 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.7 MiB/s wr, 334 op/s 2024-09-05T12:42:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:54 smithi174 ceph-mon[34313]: pgmap v476: 129 pgs: 129 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.7 MiB/s wr, 334 op/s 2024-09-05T12:42:54.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 20.05 MB/sec warmup 108 sec latency 15690.536 ms 2024-09-05T12:42:55.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 19.86 MB/sec warmup 109 sec latency 16690.577 ms 2024-09-05T12:42:56.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:56 smithi071 ceph-mon[29845]: pgmap v477: 129 pgs: 129 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 8.3 MiB/s wr, 264 op/s 2024-09-05T12:42:56.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:56 smithi123 ceph-mon[34321]: pgmap v477: 129 pgs: 129 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 8.3 MiB/s wr, 264 op/s 2024-09-05T12:42:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:56 smithi174 ceph-mon[34313]: pgmap v477: 129 pgs: 129 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 8.3 MiB/s wr, 264 op/s 2024-09-05T12:42:56.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 19.68 MB/sec warmup 110 sec latency 17690.620 ms 2024-09-05T12:42:57.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 19.50 MB/sec warmup 111 sec latency 18690.668 ms 2024-09-05T12:42:58.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:58 smithi071 ceph-mon[29845]: pgmap v478: 129 pgs: 129 active+clean; 492 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 8.5 MiB/s wr, 323 op/s 2024-09-05T12:42:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:58 smithi123 ceph-mon[34321]: pgmap v478: 129 pgs: 129 active+clean; 492 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 8.5 MiB/s wr, 323 op/s 2024-09-05T12:42:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:58 smithi174 ceph-mon[34313]: pgmap v478: 129 pgs: 129 active+clean; 492 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 8.5 MiB/s wr, 323 op/s 2024-09-05T12:42:58.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 19.33 MB/sec warmup 112 sec latency 19690.721 ms 2024-09-05T12:42:58.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.907+0000 7f41b1400640 1 -- 172.21.15.71:0/469594781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac1079c0 msgr2=0x7f41ac107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:58.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.907+0000 7f41b1400640 1 --2- 172.21.15.71:0/469594781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac1079c0 0x7f41ac107da0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f419c009930 tx=0x7f419c02a420 comp rx=0 tx=0).stop 2024-09-05T12:42:58.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.907+0000 7f41b1400640 1 -- 172.21.15.71:0/469594781 shutdown_connections 2024-09-05T12:42:58.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.907+0000 7f41b1400640 1 --2- 172.21.15.71:0/469594781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac102360 0x7f41ac10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:58.910 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.907+0000 7f41b1400640 1 --2- 172.21.15.71:0/469594781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac1019c0 0x7f41ac101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:58.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.908+0000 7f41b1400640 1 --2- 172.21.15.71:0/469594781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac1079c0 0x7f41ac107da0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:58.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.908+0000 7f41b1400640 1 -- 172.21.15.71:0/469594781 >> 172.21.15.71:0/469594781 conn(0x7f41ac0fd740 msgr2=0x7f41ac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:58.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.908+0000 7f41b1400640 1 -- 172.21.15.71:0/469594781 shutdown_connections 2024-09-05T12:42:58.911 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.908+0000 7f41b1400640 1 -- 172.21.15.71:0/469594781 wait complete. 2024-09-05T12:42:58.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.908+0000 7f41b1400640 1 Processor -- start 2024-09-05T12:42:58.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 -- start start 2024-09-05T12:42:58.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 0x7f41ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac102360 0x7f41ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac1079c0 0x7f41ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111dc0 con 0x7f41ac102360 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111c40 con 0x7f41ac1019c0 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41b1400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41ac111f40 con 0x7f41ac1079c0 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 0x7f41ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 0x7f41ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60384/0 (socket says 172.21.15.71:60384) 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41abe00640 1 -- 172.21.15.71:0/1670794899 learned_addr learned my addr 172.21.15.71:0/1670794899 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:42:58.913 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.909+0000 7f41ab400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac102360 0x7f41ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41abe00640 1 -- 172.21.15.71:0/1670794899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac1079c0 msgr2=0x7f41ac074f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:42:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41abe00640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac1079c0 0x7f41ac074f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41abe00640 1 -- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac102360 msgr2=0x7f41ac074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:58.914 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41abe00640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac102360 0x7f41ac074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41abe00640 1 -- 172.21.15.71:0/1670794899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41ac1a67e0 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41abe00640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 0x7f41ac0744f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f419c009230 tx=0x7f419c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c03f030 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f419c003ad0 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.910+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41ac1a6a70 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.911+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41ac1a6fa0 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.911+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419c035070 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.911+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41ac101e20 con 0x7f41ac1019c0 2024-09-05T12:42:58.915 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.913+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f419c042030 con 0x7f41ac1019c0 2024-09-05T12:42:58.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.915+0000 7f41a8c00640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f41900a2d70 0x7f41900a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:42:58.916 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.915+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f419c0efc90 con 0x7f41ac1019c0 2024-09-05T12:42:58.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.915+0000 7f41ab400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f41900a2d70 0x7f41900a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:42:58.917 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.916+0000 7f41ab400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f41900a2d70 0x7f41900a5230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41ac075f80 tx=0x7f419800b390 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:42:58.918 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:58.917+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f419c0bc4c0 con 0x7f41ac1019c0 2024-09-05T12:42:59.136 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.134+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41ac02d7c0 con 0x7f41ac1019c0 2024-09-05T12:42:59.137 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.135+0000 7f41a8c00640 1 -- 172.21.15.71:0/1670794899 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+11998 (secure 0 0 0) 0x7f419c0bbdc0 con 0x7f41ac1019c0 2024-09-05T12:42:59.137 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:42:59.138 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":63,"btime":"2024-09-05T12:41:51:592773+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:41:51.380979+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:42:59.138 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 63 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f41900a2d70 msgr2=0x7f41900a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f41900a2d70 0x7f41900a5230 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f41ac075f80 tx=0x7f419800b390 comp rx=0 tx=0).stop 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 msgr2=0x7f41ac0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 0x7f41ac0744f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f419c009230 tx=0x7f419c004270 comp rx=0 tx=0).stop 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 shutdown_connections 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41ac1079c0 0x7f41ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:59.140 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f41900a2d70 0x7f41900a5230 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:59.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41ac102360 0x7f41ac074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:59.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 --2- 172.21.15.71:0/1670794899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41ac1019c0 0x7f41ac0744f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:42:59.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 >> 172.21.15.71:0/1670794899 conn(0x7f41ac0fd740 msgr2=0x7f41ac0fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:42:59.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 shutdown_connections 2024-09-05T12:42:59.141 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:42:59.138+0000 7f41b1400640 1 -- 172.21.15.71:0/1670794899 wait complete. 2024-09-05T12:42:59.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:42:59 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1670794899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:42:59.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:42:59 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1670794899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:42:59.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:42:59 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1670794899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:42:59.773 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 19.16 MB/sec warmup 113 sec latency 20690.779 ms 2024-09-05T12:43:00.208 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 98bff790 --release --await' 2024-09-05T12:43:00.208 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 98bff790 --release --await 2024-09-05T12:43:00.623 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:43:00.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:00 smithi071 ceph-mon[29845]: pgmap v479: 129 pgs: 129 active+clean; 492 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.6 MiB/s wr, 241 op/s 2024-09-05T12:43:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:00 smithi123 ceph-mon[34321]: pgmap v479: 129 pgs: 129 active+clean; 492 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.6 MiB/s wr, 241 op/s 2024-09-05T12:43:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:00 smithi174 ceph-mon[34313]: pgmap v479: 129 pgs: 129 active+clean; 492 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 5.6 MiB/s wr, 241 op/s 2024-09-05T12:43:00.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 36946 18.99 MB/sec warmup 114 sec latency 21690.921 ms 2024-09-05T12:43:01.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 -- 172.21.15.71:0/1221559411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36041049c0 msgr2=0x7f3604108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 --2- 172.21.15.71:0/1221559411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36041049c0 0x7f3604108a10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009900 tx=0x7f35f002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 -- 172.21.15.71:0/1221559411 shutdown_connections 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 --2- 172.21.15.71:0/1221559411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604109140 0x7f3604110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 --2- 172.21.15.71:0/1221559411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f36041049c0 0x7f3604108a10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 --2- 172.21.15.71:0/1221559411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3604104010 0x7f36041043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 -- 172.21.15.71:0/1221559411 >> 172.21.15.71:0/1221559411 conn(0x7f36040ffa50 msgr2=0x7f3604101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 -- 172.21.15.71:0/1221559411 shutdown_connections 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.254+0000 7f3609600640 1 -- 172.21.15.71:0/1221559411 wait complete. 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 Processor -- start 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 -- start start 2024-09-05T12:43:01.280 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604104010 0x7f360419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36041049c0 0x7f360419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 0x7f360419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3604114190 con 0x7f3604109140 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3604114010 con 0x7f36041049c0 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.255+0000 7f3609600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3604114310 con 0x7f3604104010 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 0x7f360419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 0x7f360419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40978/0 (socket says 172.21.15.71:40978) 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 -- 172.21.15.71:0/1152561289 learned_addr learned my addr 172.21.15.71:0/1152561289 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3602000640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36041049c0 0x7f360419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3602a00640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604104010 0x7f360419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 -- 172.21.15.71:0/1152561289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604104010 msgr2=0x7f360419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604104010 0x7f360419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 -- 172.21.15.71:0/1152561289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36041049c0 msgr2=0x7f360419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36041049c0 0x7f360419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 -- 172.21.15.71:0/1152561289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360419f600 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3602a00640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604104010 0x7f360419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.256+0000 7f3603400640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 0x7f360419ef20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00eaa0 tx=0x7f35fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.257+0000 7f35fb400640 1 -- 172.21.15.71:0/1152561289 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc00cdb0 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.257+0000 7f35fb400640 1 -- 172.21.15.71:0/1152561289 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35fc005020 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.257+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360419f8f0 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.257+0000 7f35fb400640 1 -- 172.21.15.71:0/1152561289 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc0116b0 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.257+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36041a71d0 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 Processor -- start 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f35fb400640 1 -- 172.21.15.71:0/1152561289 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f35fc005560 con 0x7f3604109140 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 -- start start 2024-09-05T12:43:01.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a9496f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a9568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563a8a8af650 con 0x563a8a949310 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a8a856f10 con 0x563a8a9544f0 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.261+0000 7f360c064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a8a8ab610 con 0x563a8a949c30 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a9496f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a9496f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40980/0 (socket says 172.21.15.71:40980) 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 -- 172.21.15.71:0/3084249406 learned_addr learned my addr 172.21.15.71:0/3084249406 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602a00640 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a9568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 -- 172.21.15.71:0/3084249406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 msgr2=0x563a8a953fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.283 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 -- 172.21.15.71:0/3084249406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 msgr2=0x563a8a9568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a9568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 -- 172.21.15.71:0/3084249406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8a5be580 con 0x563a8a949310 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.262+0000 7f3602000640 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a9496f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009230 tx=0x7f35f0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2720d8f71ec24ce server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f35fb400640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35e80a2d70 0x7f35e80a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f35ef400640 1 -- 172.21.15.71:0/3084249406 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0044020 con 0x563a8a949310 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f35ef400640 1 -- 172.21.15.71:0/3084249406 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f0004700 con 0x563a8a949310 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f35fb400640 1 -- 172.21.15.71:0/1152561289 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f35fc0cf9a0 con 0x7f3604109140 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f3602a00640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35e80a2d70 0x7f35e80a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f35ef400640 1 -- 172.21.15.71:0/3084249406 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0035040 con 0x563a8a949310 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f360c064740 1 -- 172.21.15.71:0/3084249406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 msgr2=0x563a8a9496f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.263+0000 7f360c064740 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a9496f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009230 tx=0x7f35f0005bc0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f3602a00640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35e80a2d70 0x7f35e80a5230 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009db0 tx=0x7f35f4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 -- 172.21.15.71:0/3084249406 shutdown_connections 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a9568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 --2- 172.21.15.71:0/3084249406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a9496f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 -- 172.21.15.71:0/3084249406 >> 172.21.15.71:0/3084249406 conn(0x563a8a9458a0 msgr2=0x563a8a945c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 -- 172.21.15.71:0/3084249406 shutdown_connections 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 -- 172.21.15.71:0/3084249406 wait complete. 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.264+0000 7f360c064740 1 -- start start 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 Processor -- start 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a951330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a951870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a951db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563a8a9570b0 con 0x563a8a949310 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a8a86a350 con 0x563a8a9544f0 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a8a957230 con 0x563a8a949c30 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f360c064740 10 client.0 _ll_register_callbacks cb 0x563a8a88d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f3602a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a951330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f3603400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a951db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f3603400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a951db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:60434/0 (socket says 172.21.15.71:60434) 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f3603400640 1 -- 172.21.15.71:0/1228350642 learned_addr learned my addr 172.21.15.71:0/1228350642 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.265+0000 7f3602a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a951330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40990/0 (socket says 172.21.15.71:40990) 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3602000640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a951870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3603400640 1 -- 172.21.15.71:0/1228350642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 msgr2=0x563a8a951870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3603400640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a951870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3603400640 1 -- 172.21.15.71:0/1228350642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 msgr2=0x563a8a951330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3603400640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a951330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3603400640 1 -- 172.21.15.71:0/1228350642 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8a688960 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3602000640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a951870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f3603400640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a951db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0984a0 tx=0x7f35fc09c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13167920047ab87b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f35e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f35e3e00640 1 -- 172.21.15.71:0/1228350642 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01f070 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f0004700 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.266+0000 7f35e3e00640 1 -- 172.21.15.71:0/1228350642 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f35fc098140 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f35e3e00640 1 -- 172.21.15.71:0/1228350642 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f35fc0b4020 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f360c064740 10 client.27098 fetch_fsmap learned FSMap version 63 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563a8a962d20 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f35e3e00640 1 -- 172.21.15.71:0/1228350642 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f0004700 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f35e3e00640 1 -- 172.21.15.71:0/1228350642 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f35fc00f040 con 0x563a8a9544f0 2024-09-05T12:43:01.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f360c064740 10 client.27098 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f360c064740 10 client.27098 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f360c064740 20 client.27098 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.267+0000 7f360c064740 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9641e0 0x563a8a966660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.268+0000 7f360c064740 4 client.27098 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.268+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a8a8bcbc0 con 0x563a8a9641e0 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.268+0000 7f3602a00640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9641e0 0x563a8a966660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.268+0000 7f3602a00640 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9641e0 0x563a8a966660 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4717cb0bbb7c11 server_cookie=6a8cb0cb4b24b137 in_seq=0 out_seq=0 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.268+0000 7f35e3e00640 10 client.27098 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f35e3e00640 1 -- 172.21.15.71:0/1228350642 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a8a8bcbc0 con 0x563a8a9641e0 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f35e3e00640 10 client.27098 handle_command_reply: tid=0 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 client.27098 shutdown 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 msgr2=0x563a8a951db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a951db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f35fc0984a0 tx=0x7f35fc09c300 comp rx=0 tx=0).stop 2024-09-05T12:43:01.288 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 shutdown_connections 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9641e0 0x563a8a966660 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a951db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a951870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 --2- 172.21.15.71:0/1228350642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a951330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.269+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 >> 172.21.15.71:0/1228350642 conn(0x563a8a9458a0 msgr2=0x563a8a94b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.270+0000 7f35e3e00640 0 client.27098 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.270+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 shutdown_connections 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.270+0000 7f360c064740 1 -- 172.21.15.71:0/1228350642 wait complete. 2024-09-05T12:43:01.289 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.270+0000 7f360c064740 20 client.27098 trim_cache size 0 max 16384 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.287+0000 7f360c064740 1 Processor -- start 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.287+0000 7f360c064740 1 -- start start 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f360c064740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a94bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f360c064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f360c064740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a96bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f360c064740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36041b1080 con 0x563a8a949310 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f360c064740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36041b2da0 con 0x563a8a9544f0 2024-09-05T12:43:01.291 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f360c064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36041b17b0 con 0x563a8a949c30 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a94bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a94bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41006/0 (socket says 172.21.15.71:41006) 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 -- 172.21.15.71:0/655470791 learned_addr learned my addr 172.21.15.71:0/655470791 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602000640 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3603400640 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a96bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 -- 172.21.15.71:0/655470791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 msgr2=0x563a8a953190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 -- 172.21.15.71:0/655470791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 msgr2=0x563a8a96bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a96bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.292 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3602a00640 1 -- 172.21.15.71:0/655470791 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a8a941260 con 0x563a8a949310 2024-09-05T12:43:01.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.288+0000 7f3603400640 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a96bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:01.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.289+0000 7f3602a00640 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a94bd70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x563a8a966e50 tx=0x7f35f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87be393f944a61a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:01.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.289+0000 7f35eea00640 1 -- 172.21.15.71:0/655470791 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4022d30 con 0x563a8a949310 2024-09-05T12:43:01.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.289+0000 7f35eea00640 1 -- 172.21.15.71:0/655470791 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f40220d0 con 0x563a8a949310 2024-09-05T12:43:01.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.289+0000 7f35eea00640 1 -- 172.21.15.71:0/655470791 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4034420 con 0x563a8a949310 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 -- 172.21.15.71:0/655470791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 msgr2=0x563a8a94bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a94bd70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x563a8a966e50 tx=0x7f35f4017000 comp rx=0 tx=0).stop 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 -- 172.21.15.71:0/655470791 shutdown_connections 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a9544f0 0x563a8a96bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a953190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 --2- 172.21.15.71:0/655470791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a949310 0x563a8a94bd70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 -- 172.21.15.71:0/655470791 >> 172.21.15.71:0/655470791 conn(0x563a8a9458a0 msgr2=0x563a8a952a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 -- 172.21.15.71:0/655470791 shutdown_connections 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 -- 172.21.15.71:0/655470791 wait complete. 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 -- start start 2024-09-05T12:43:01.296 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.290+0000 7f360c064740 1 Processor -- start 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a949310 0x563a8a95c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a95cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 0x563a8a958c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f36041b26d0 con 0x563a8a9544f0 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36041b1f40 con 0x563a8a949310 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36041af080 con 0x563a8a949c30 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f360c064740 10 client.0 _ll_register_callbacks cb 0x7f35fc097d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f3603400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 0x563a8a958c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f3603400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 0x563a8a958c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41010/0 (socket says 172.21.15.71:41010) 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f3602000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a95cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f3603400640 1 -- 172.21.15.71:0/1480381496 learned_addr learned my addr 172.21.15.71:0/1480381496 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.291+0000 7f3602a00640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a949310 0x563a8a95c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3603400640 1 -- 172.21.15.71:0/1480381496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 msgr2=0x563a8a95cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3603400640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a95cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3603400640 1 -- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a949310 msgr2=0x563a8a95c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3603400640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a949310 0x563a8a95c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3603400640 1 -- 172.21.15.71:0/1480381496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a8a959460 con 0x563a8a9544f0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3602a00640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a949310 0x563a8a95c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f3603400640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 0x563a8a958c30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f35fc09b7d0 tx=0x7f35fc09b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a53e8b90c83e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f35e3e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f35e3e00640 1 -- 172.21.15.71:0/1480381496 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35fc01f040 con 0x563a8a9544f0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f35e3e00640 1 -- 172.21.15.71:0/1480381496 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9438+0+0 (secure 0 0 0) 0x7f35fc097820 con 0x563a8a9544f0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.292+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f40220d0 con 0x563a8a9544f0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f35e3e00640 1 -- 172.21.15.71:0/1480381496 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f35fc09b430 con 0x563a8a9544f0 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 10 client.17112 fetch_fsmap learned FSMap version 63 2024-09-05T12:43:01.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563a8a9cfb00 con 0x563a8a9544f0 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f35e3e00640 1 -- 172.21.15.71:0/1480381496 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35f40220d0 con 0x563a8a9544f0 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f35e3e00640 1 -- 172.21.15.71:0/1480381496 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f35fc0a1010 con 0x563a8a9544f0 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 10 client.17112 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 10 client.17112 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 20 client.17112 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9d0310 0x563a8a9d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 4 client.17112 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "98bff790", "release": true, "await": true}] 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.293+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "98bff790", "release": true, "await": true}) -- 0x563a8a8b70d0 con 0x563a8a9d0310 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.294+0000 7f3602a00640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9d0310 0x563a8a9d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.294+0000 7f3602a00640 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9d0310 0x563a8a9d2730 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35d5dedd80de7f0c server_cookie=9b74d9cdbb3dca8d in_seq=0 out_seq=0 2024-09-05T12:43:01.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.294+0000 7f35e3e00640 10 client.17112 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:01.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.322+0000 7f35e3e00640 1 -- 172.21.15.71:0/1480381496 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563a8a8b70d0 con 0x563a8a9d0310 2024-09-05T12:43:01.324 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.322+0000 7f35e3e00640 10 client.17112 handle_command_reply: tid=0 2024-09-05T12:43:01.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.323+0000 7f360c064740 1 client.17112 shutdown 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.325+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 msgr2=0x563a8a958c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.325+0000 7f360c064740 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 0x563a8a958c30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f35fc09b7d0 tx=0x7f35fc09b740 comp rx=0 tx=0).stop 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 shutdown_connections 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563a8a9d0310 0x563a8a9d2730 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a8a9544f0 0x563a8a958c30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a8a949c30 0x563a8a95cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 --2- 172.21.15.71:0/1480381496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a8a949310 0x563a8a95c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f35e3e00640 0 client.17112 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 >> 172.21.15.71:0/1480381496 conn(0x563a8a9458a0 msgr2=0x563a8a951b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:01.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 shutdown_connections 2024-09-05T12:43:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 1 -- 172.21.15.71:0/1480381496 wait complete. 2024-09-05T12:43:01.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.326+0000 7f360c064740 20 client.17112 trim_cache size 0 max 16384 2024-09-05T12:43:01.329 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:43:01.329 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 63, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 48, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "98bff790": { 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 48, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 134.5, 2024-09-05T12:43:01.330 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:01.331 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:43:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35e80a2d70 msgr2=0x7f35e80a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35e80a2d70 0x7f35e80a5230 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f35f4009db0 tx=0x7f35f4005f90 comp rx=0 tx=0).stop 2024-09-05T12:43:01.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 msgr2=0x7f360419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 0x7f360419ef20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f35fc00eaa0 tx=0x7f35fc00ef60 comp rx=0 tx=0).stop 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 shutdown_connections 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f35e80a2d70 0x7f35e80a5230 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3604109140 0x7f360419ef20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36041049c0 0x7f360419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 --2- 172.21.15.71:0/1152561289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3604104010 0x7f360419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.328+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 >> 172.21.15.71:0/1152561289 conn(0x7f36040ffa50 msgr2=0x7f3604100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.329+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 shutdown_connections 2024-09-05T12:43:01.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:01.329+0000 7f3609600640 1 -- 172.21.15.71:0/1152561289 wait complete. 2024-09-05T12:43:01.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 37329 19.00 MB/sec warmup 115 sec latency 22230.005 ms 2024-09-05T12:43:02.357 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '98bff790', total seconds elapsed: 28.7 2024-09-05T12:43:02.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:02 smithi071 ceph-mon[29845]: pgmap v480: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 5.6 MiB/s wr, 248 op/s 2024-09-05T12:43:02.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:02 smithi123 ceph-mon[34321]: pgmap v480: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 5.6 MiB/s wr, 248 op/s 2024-09-05T12:43:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:02 smithi174 ceph-mon[34313]: pgmap v480: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 5.6 MiB/s wr, 248 op/s 2024-09-05T12:43:02.775 INFO:tasks.workunit.client.0.smithi071.stdout: 10 38072 19.16 MB/sec warmup 116 sec latency 99.396 ms 2024-09-05T12:43:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.256+0000 7f923f800640 1 -- 172.21.15.71:0/3249274337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381095d0 msgr2=0x7f9238109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.258 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.256+0000 7f923f800640 1 --2- 172.21.15.71:0/3249274337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381095d0 0x7f9238109a10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9224009930 tx=0x7f922402a420 comp rx=0 tx=0).stop 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 -- 172.21.15.71:0/3249274337 shutdown_connections 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 --2- 172.21.15.71:0/3249274337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f923811b140 0x7f923811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 --2- 172.21.15.71:0/3249274337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92381095d0 0x7f9238109a10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 --2- 172.21.15.71:0/3249274337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 0x7f9238110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 -- 172.21.15.71:0/3249274337 >> 172.21.15.71:0/3249274337 conn(0x7f923806ce00 msgr2=0x7f923806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 -- 172.21.15.71:0/3249274337 shutdown_connections 2024-09-05T12:43:03.259 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.257+0000 7f923f800640 1 -- 172.21.15.71:0/3249274337 wait complete. 2024-09-05T12:43:03.260 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.258+0000 7f923f800640 1 Processor -- start 2024-09-05T12:43:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.259+0000 7f923f800640 1 -- start start 2024-09-05T12:43:03.261 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923f800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 0x7f92381a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923f800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92381095d0 0x7f92381a76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923f800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923811b140 0x7f92381aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923f800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9238120d30 con 0x7f92381091f0 2024-09-05T12:43:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923f800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9238120bb0 con 0x7f923811b140 2024-09-05T12:43:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923f800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9238120eb0 con 0x7f92381095d0 2024-09-05T12:43:03.262 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923811b140 0x7f92381aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923811b140 0x7f92381aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58230/0 (socket says 172.21.15.71:58230) 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.260+0000 7f923d800640 1 -- 172.21.15.71:0/2189790629 learned_addr learned my addr 172.21.15.71:0/2189790629 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.261+0000 7f923ce00640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 0x7f92381a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.261+0000 7f923ce00640 1 -- 172.21.15.71:0/2189790629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92381095d0 msgr2=0x7f92381a76d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.261+0000 7f923ce00640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92381095d0 0x7f92381a76d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.261+0000 7f923ce00640 1 -- 172.21.15.71:0/2189790629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923811b140 msgr2=0x7f92381aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.261+0000 7f923ce00640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923811b140 0x7f92381aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.263 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.261+0000 7f923ce00640 1 -- 172.21.15.71:0/2189790629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92381ac180 con 0x7f92381091f0 2024-09-05T12:43:03.264 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.262+0000 7f923ce00640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 0x7f92381a7190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f923000dab0 tx=0x7f923000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.263+0000 7f9235600640 1 -- 172.21.15.71:0/2189790629 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923000be30 con 0x7f92381091f0 2024-09-05T12:43:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.263+0000 7f9235600640 1 -- 172.21.15.71:0/2189790629 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230003e50 con 0x7f92381091f0 2024-09-05T12:43:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.263+0000 7f9235600640 1 -- 172.21.15.71:0/2189790629 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9230010770 con 0x7f92381091f0 2024-09-05T12:43:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.263+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92381ac470 con 0x7f92381091f0 2024-09-05T12:43:03.265 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.263+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92381b3d00 con 0x7f92381091f0 2024-09-05T12:43:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.269+0000 7f92425db740 1 Processor -- start 2024-09-05T12:43:03.271 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.269+0000 7f92425db740 1 -- start start 2024-09-05T12:43:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.270+0000 7f92425db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b61f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.270+0000 7f92425db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b629e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.270+0000 7f92425db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b62c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.270+0000 7f92425db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56074b585100 con 0x56074b62a3a0 2024-09-05T12:43:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.270+0000 7f92425db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56074b52d1b0 con 0x56074b61f1e0 2024-09-05T12:43:03.272 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.270+0000 7f92425db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56074b5810c0 con 0x56074b61fb00 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f9237e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b62c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f9237e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b62c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49078/0 (socket says 172.21.15.71:49078) 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f9237e00640 1 -- 172.21.15.71:0/669793335 learned_addr learned my addr 172.21.15.71:0/669793335 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f923d800640 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b61f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f923d800640 1 -- 172.21.15.71:0/669793335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 msgr2=0x56074b629e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f923d800640 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b629e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.273 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f923d800640 1 -- 172.21.15.71:0/669793335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 msgr2=0x56074b62c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.271+0000 7f923d800640 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b62c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.272+0000 7f923d800640 1 -- 172.21.15.71:0/669793335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074b298210 con 0x56074b61f1e0 2024-09-05T12:43:03.274 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.273+0000 7f9235600640 1 -- 172.21.15.71:0/2189790629 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9230004000 con 0x7f92381091f0 2024-09-05T12:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.273+0000 7f923d800640 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b61f5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f922800b7a0 tx=0x7f922800bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9473aa2919848eab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.273+0000 7f922e000640 1 -- 172.21.15.71:0/669793335 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922800d950 con 0x56074b61f1e0 2024-09-05T12:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.274+0000 7f922e000640 1 -- 172.21.15.71:0/669793335 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9228012420 con 0x56074b61f1e0 2024-09-05T12:43:03.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.274+0000 7f922e000640 1 -- 172.21.15.71:0/669793335 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228014650 con 0x56074b61f1e0 2024-09-05T12:43:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.275+0000 7f92425db740 1 -- 172.21.15.71:0/669793335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 msgr2=0x56074b61f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.275+0000 7f92425db740 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b61f5c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f922800b7a0 tx=0x7f922800bc60 comp rx=0 tx=0).stop 2024-09-05T12:43:03.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.275+0000 7f9235600640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f921c0a2f80 0x7f921c0a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.275+0000 7f9235600640 1 -- 172.21.15.71:0/2189790629 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f92300cfcd0 con 0x7f92381091f0 2024-09-05T12:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.276+0000 7f92425db740 1 -- 172.21.15.71:0/669793335 shutdown_connections 2024-09-05T12:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.276+0000 7f92425db740 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b62c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.276+0000 7f92425db740 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b629e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.276+0000 7f92425db740 1 --2- 172.21.15.71:0/669793335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b61f5c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.276+0000 7f92425db740 1 -- 172.21.15.71:0/669793335 >> 172.21.15.71:0/669793335 conn(0x56074b61b750 msgr2=0x56074b61bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.277+0000 7f923d800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f921c0a2f80 0x7f921c0a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.278+0000 7f92425db740 1 -- 172.21.15.71:0/669793335 shutdown_connections 2024-09-05T12:43:03.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.279+0000 7f923d800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f921c0a2f80 0x7f921c0a5440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9228013040 tx=0x7f92280046c0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.279+0000 7f92425db740 1 -- 172.21.15.71:0/669793335 wait complete. 2024-09-05T12:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.279+0000 7f92425db740 1 -- start start 2024-09-05T12:43:03.281 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.279+0000 7f92425db740 1 Processor -- start 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6273b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b6278f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b627e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56074b62cfd0 con 0x56074b61f1e0 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56074b50e090 con 0x56074b61fb00 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56074b62d150 con 0x56074b62a3a0 2024-09-05T12:43:03.282 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.280+0000 7f92425db740 10 client.0 _ll_register_callbacks cb 0x56074b562f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:03.283 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.281+0000 7f923d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b627e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.281+0000 7f923d800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b627e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59184/0 (socket says 172.21.15.71:59184) 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.281+0000 7f923d800640 1 -- 172.21.15.71:0/4168599715 learned_addr learned my addr 172.21.15.71:0/4168599715 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.281+0000 7f9237e00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b6278f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f923ce00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6273b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f9237e00640 1 -- 172.21.15.71:0/4168599715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 msgr2=0x56074b627e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f9237e00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b627e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f9237e00640 1 -- 172.21.15.71:0/4168599715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 msgr2=0x56074b6273b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.284 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f9237e00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6273b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f9237e00640 1 -- 172.21.15.71:0/4168599715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074b364620 con 0x56074b61fb00 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f923ce00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6273b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.282+0000 7f9237e00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b6278f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9224006010 tx=0x7f9224005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b9fd8f9c8457ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.283+0000 7f9222a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.283+0000 7f9222a00640 1 -- 172.21.15.71:0/4168599715 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224040070 con 0x56074b61fb00 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.283+0000 7f9222a00640 1 -- 172.21.15.71:0/4168599715 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f92240046c0 con 0x56074b61fb00 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.284+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9228012420 con 0x56074b61fb00 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.284+0000 7f9222a00640 1 -- 172.21.15.71:0/4168599715 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f922404a050 con 0x56074b61fb00 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.284+0000 7f92425db740 10 client.27116 fetch_fsmap learned FSMap version 63 2024-09-05T12:43:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.284+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x56074b638bf0 con 0x56074b61fb00 2024-09-05T12:43:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.285+0000 7f9222a00640 1 -- 172.21.15.71:0/4168599715 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9228012420 con 0x56074b61fb00 2024-09-05T12:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.287+0000 7f9222a00640 1 -- 172.21.15.71:0/4168599715 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f922403a040 con 0x56074b61fb00 2024-09-05T12:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f92425db740 10 client.27116 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:43:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f92425db740 10 client.27116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f92425db740 20 client.27116 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f92425db740 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b63a150 0x56074b63c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f92425db740 4 client.27116 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56074b592830 con 0x56074b63a150 2024-09-05T12:43:03.290 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.288+0000 7f923ce00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b63a150 0x56074b63c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.290+0000 7f923ce00640 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b63a150 0x56074b63c5a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fed25bc94de08a1 server_cookie=429e9aed6b215a70 in_seq=0 out_seq=0 2024-09-05T12:43:03.292 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.290+0000 7f9222a00640 10 client.27116 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.292+0000 7f9222a00640 1 -- 172.21.15.71:0/4168599715 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56074b592830 con 0x56074b63a150 2024-09-05T12:43:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.292+0000 7f9222a00640 10 client.27116 handle_command_reply: tid=0 2024-09-05T12:43:03.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.292+0000 7f92425db740 1 client.27116 shutdown 2024-09-05T12:43:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.292+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 msgr2=0x56074b6278f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.292+0000 7f92425db740 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b6278f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9224006010 tx=0x7f9224005270 comp rx=0 tx=0).stop 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.292+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 shutdown_connections 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b63a150 0x56074b63c5a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b627e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b6278f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 --2- 172.21.15.71:0/4168599715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6273b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 >> 172.21.15.71:0/4168599715 conn(0x56074b61b750 msgr2=0x56074b621460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f9222a00640 0 client.27116 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 shutdown_connections 2024-09-05T12:43:03.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 1 -- 172.21.15.71:0/4168599715 wait complete. 2024-09-05T12:43:03.296 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.293+0000 7f92425db740 20 client.27116 trim_cache size 0 max 16384 2024-09-05T12:43:03.330 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.329+0000 7f92425db740 1 Processor -- start 2024-09-05T12:43:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.329+0000 7f92425db740 1 -- start start 2024-09-05T12:43:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.329+0000 7f92425db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b629210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f92425db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b641e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f92425db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b6423b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.331 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f92425db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f92381bdf10 con 0x56074b62a3a0 2024-09-05T12:43:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f92425db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92381bfc30 con 0x56074b61f1e0 2024-09-05T12:43:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f92425db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92381be640 con 0x56074b61fb00 2024-09-05T12:43:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f9237e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b641e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f9237e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b641e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:59198/0 (socket says 172.21.15.71:59198) 2024-09-05T12:43:03.332 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f9237e00640 1 -- 172.21.15.71:0/903272116 learned_addr learned my addr 172.21.15.71:0/903272116 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f923ce00640 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b629210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.330+0000 7f923d800640 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b6423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.331+0000 7f923ce00640 1 -- 172.21.15.71:0/903272116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 msgr2=0x56074b641e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.331+0000 7f923ce00640 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b641e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.331+0000 7f923ce00640 1 -- 172.21.15.71:0/903272116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 msgr2=0x56074b6423b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.331+0000 7f923ce00640 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b6423b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.333 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.331+0000 7f923ce00640 1 -- 172.21.15.71:0/903272116 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56074b636160 con 0x56074b61f1e0 2024-09-05T12:43:03.334 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.332+0000 7f923ce00640 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b629210 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f92300ccfd0 tx=0x7f92300042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee52687f0f78c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.333+0000 7f922d600640 1 -- 172.21.15.71:0/903272116 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f923009f050 con 0x56074b61f1e0 2024-09-05T12:43:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.333+0000 7f922d600640 1 -- 172.21.15.71:0/903272116 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230014040 con 0x56074b61f1e0 2024-09-05T12:43:03.335 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.333+0000 7f922d600640 1 -- 172.21.15.71:0/903272116 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92300cd250 con 0x56074b61f1e0 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.334+0000 7f92425db740 1 -- 172.21.15.71:0/903272116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 msgr2=0x56074b629210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.335+0000 7f92425db740 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b629210 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f92300ccfd0 tx=0x7f92300042f0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.335+0000 7f92425db740 1 -- 172.21.15.71:0/903272116 shutdown_connections 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.335+0000 7f92425db740 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b62a3a0 0x56074b6423b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.335+0000 7f92425db740 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b61fb00 0x56074b641e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.335+0000 7f92425db740 1 --2- 172.21.15.71:0/903272116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61f1e0 0x56074b629210 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.337 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.335+0000 7f92425db740 1 -- 172.21.15.71:0/903272116 >> 172.21.15.71:0/903272116 conn(0x56074b61b750 msgr2=0x56074b628b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.336+0000 7f92425db740 1 -- 172.21.15.71:0/903272116 shutdown_connections 2024-09-05T12:43:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.336+0000 7f92425db740 1 -- 172.21.15.71:0/903272116 wait complete. 2024-09-05T12:43:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.337+0000 7f92425db740 1 -- start start 2024-09-05T12:43:03.340 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.337+0000 7f92425db740 1 Processor -- start 2024-09-05T12:43:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6328b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b62dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b62e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f92381bf560 con 0x56074b61f1e0 2024-09-05T12:43:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92381bedd0 con 0x56074b61fb00 2024-09-05T12:43:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92381bbed0 con 0x56074b62a3a0 2024-09-05T12:43:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f92425db740 10 client.0 _ll_register_callbacks cb 0x7f9224009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f9237e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b62dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f9237e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b62dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58286/0 (socket says 172.21.15.71:58286) 2024-09-05T12:43:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.339+0000 7f9237e00640 1 -- 172.21.15.71:0/167201629 learned_addr learned my addr 172.21.15.71:0/167201629 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.341+0000 7f9237e00640 1 -- 172.21.15.71:0/167201629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 msgr2=0x56074b62e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.340+0000 7f923ce00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.340+0000 7f923d800640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b62e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.341+0000 7f9237e00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b62e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.341+0000 7f9237e00640 1 -- 172.21.15.71:0/167201629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 msgr2=0x56074b6328b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.341+0000 7f9237e00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6328b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.342+0000 7f9237e00640 1 -- 172.21.15.71:0/167201629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56074b62e7b0 con 0x56074b61fb00 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.342+0000 7f923ce00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.342+0000 7f923d800640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b62e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:43:03.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.342+0000 7f9237e00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b62dbd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9224009850 tx=0x7f9224004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90567567f74919a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.342+0000 7f9222a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.342+0000 7f9222a00640 1 -- 172.21.15.71:0/167201629 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922404a050 con 0x56074b61fb00 2024-09-05T12:43:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.343+0000 7f9222a00640 1 -- 172.21.15.71:0/167201629 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9526+0+0 (secure 0 0 0) 0x7f92240055f0 con 0x56074b61fb00 2024-09-05T12:43:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.344+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9230014040 con 0x56074b61fb00 2024-09-05T12:43:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.344+0000 7f9222a00640 1 -- 172.21.15.71:0/167201629 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f9224035040 con 0x56074b61fb00 2024-09-05T12:43:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.344+0000 7f92425db740 10 client.27122 fetch_fsmap learned FSMap version 63 2024-09-05T12:43:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.344+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x56074b6a4d10 con 0x56074b61fb00 2024-09-05T12:43:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.345+0000 7f9222a00640 1 -- 172.21.15.71:0/167201629 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9230014040 con 0x56074b61fb00 2024-09-05T12:43:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.345+0000 7f9222a00640 1 -- 172.21.15.71:0/167201629 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 63) ==== 8434+0+0 (secure 0 0 0) 0x7f9224031be0 con 0x56074b61fb00 2024-09-05T12:43:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.345+0000 7f92425db740 10 client.27122 fetch_fsmap finished waiting for FSMap version 63 2024-09-05T12:43:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.345+0000 7f92425db740 10 client.27122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.345+0000 7f92425db740 20 client.27122 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.345+0000 7f92425db740 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b6a5530 0x56074b6a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.346+0000 7f92425db740 4 client.27122 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:43:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.346+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56074b58cb80 con 0x56074b6a5530 2024-09-05T12:43:03.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.346+0000 7f923ce00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b6a5530 0x56074b6a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.347+0000 7f923ce00640 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b6a5530 0x56074b6a7950 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47586f3ae0b595d server_cookie=9bb9757d6abd9a0e in_seq=0 out_seq=0 2024-09-05T12:43:03.349 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.347+0000 7f9222a00640 10 client.27122 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.640+0000 7f9222a00640 1 -- 172.21.15.71:0/167201629 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56074b58cb80 con 0x56074b6a5530 2024-09-05T12:43:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.640+0000 7f9222a00640 10 client.27122 handle_command_reply: tid=0 2024-09-05T12:43:03.643 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.641+0000 7f92425db740 1 client.27122 shutdown 2024-09-05T12:43:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.642+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 msgr2=0x56074b62dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.642+0000 7f92425db740 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b62dbd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f9224009850 tx=0x7f9224004610 comp rx=0 tx=0).stop 2024-09-05T12:43:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.642+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 shutdown_connections 2024-09-05T12:43:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.642+0000 7f92425db740 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56074b6a5530 0x56074b6a7950 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f92425db740 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56074b62a3a0 0x56074b62e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f9222a00640 0 client.27122 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f92425db740 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56074b61fb00 0x56074b62dbd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f92425db740 1 --2- 172.21.15.71:0/167201629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56074b61f1e0 0x56074b6328b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 >> 172.21.15.71:0/167201629 conn(0x56074b61b750 msgr2=0x56074b635970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 shutdown_connections 2024-09-05T12:43:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.643+0000 7f92425db740 1 -- 172.21.15.71:0/167201629 wait complete. 2024-09-05T12:43:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.644+0000 7f92425db740 20 client.27122 trim_cache size 0 max 16384 2024-09-05T12:43:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:43:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:43:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:43:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:43:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f921c0a2f80 msgr2=0x7f921c0a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f921c0a2f80 0x7f921c0a5440 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9228013040 tx=0x7f92280046c0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 msgr2=0x7f92381a7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 0x7f92381a7190 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f923000dab0 tx=0x7f923000df70 comp rx=0 tx=0).stop 2024-09-05T12:43:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 shutdown_connections 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f923811b140 0x7f92381aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92381095d0 0x7f92381a76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f921c0a2f80 0x7f921c0a5440 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 --2- 172.21.15.71:0/2189790629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92381091f0 0x7f92381a7190 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.645+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 >> 172.21.15.71:0/2189790629 conn(0x7f923806ce00 msgr2=0x7f923811b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.646+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 shutdown_connections 2024-09-05T12:43:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:03.646+0000 7f923f800640 1 -- 172.21.15.71:0/2189790629 wait complete. 2024-09-05T12:43:03.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:03 smithi071 ceph-mon[29845]: osdmap e115: 12 total, 12 up, 12 in 2024-09-05T12:43:03.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:03 smithi123 ceph-mon[34321]: osdmap e115: 12 total, 12 up, 12 in 2024-09-05T12:43:03.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:03 smithi174 ceph-mon[34313]: osdmap e115: 12 total, 12 up, 12 in 2024-09-05T12:43:03.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 38545 19.22 MB/sec warmup 117 sec latency 199.454 ms 2024-09-05T12:43:04.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:04 smithi123 ceph-mon[34321]: pgmap v482: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 402 KiB/s wr, 104 op/s 2024-09-05T12:43:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:04 smithi174 ceph-mon[34313]: pgmap v482: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 402 KiB/s wr, 104 op/s 2024-09-05T12:43:04.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 39246 19.38 MB/sec warmup 118 sec latency 91.179 ms 2024-09-05T12:43:04.875 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:04 smithi071 ceph-mon[29845]: pgmap v482: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 402 KiB/s wr, 104 op/s 2024-09-05T12:43:05.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:05 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:05.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:05 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:05.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 40224 19.68 MB/sec warmup 119 sec latency 55.969 ms 2024-09-05T12:43:05.865 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:05 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:06.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:06 smithi071 ceph-mon[29845]: pgmap v483: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 402 KiB/s wr, 104 op/s 2024-09-05T12:43:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:06 smithi174 ceph-mon[34313]: pgmap v483: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 402 KiB/s wr, 104 op/s 2024-09-05T12:43:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:06 smithi123 ceph-mon[34321]: pgmap v483: 129 pgs: 129 active+clean; 491 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 402 KiB/s wr, 104 op/s 2024-09-05T12:43:07.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 42179 55.34 MB/sec execute 1 sec latency 63.484 ms 2024-09-05T12:43:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:08 smithi174 ceph-mon[34313]: pgmap v484: 129 pgs: 129 active+clean; 515 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-05T12:43:08.775 INFO:tasks.workunit.client.0.smithi071.stdout: 10 43227 56.88 MB/sec execute 2 sec latency 62.226 ms 2024-09-05T12:43:08.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:08 smithi071 ceph-mon[29845]: pgmap v484: 129 pgs: 129 active+clean; 515 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-05T12:43:08.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:08 smithi123 ceph-mon[34321]: pgmap v484: 129 pgs: 129 active+clean; 515 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-05T12:43:09.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 44208 56.09 MB/sec execute 3 sec latency 87.310 ms 2024-09-05T12:43:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:10 smithi071 ceph-mon[29845]: pgmap v485: 129 pgs: 129 active+clean; 515 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-05T12:43:10.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:10 smithi071 ceph-mon[29845]: osdmap e116: 12 total, 12 up, 12 in 2024-09-05T12:43:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:10 smithi174 ceph-mon[34313]: pgmap v485: 129 pgs: 129 active+clean; 515 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-05T12:43:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:10 smithi174 ceph-mon[34313]: osdmap e116: 12 total, 12 up, 12 in 2024-09-05T12:43:10.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 44946 51.02 MB/sec execute 4 sec latency 140.674 ms 2024-09-05T12:43:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:10 smithi123 ceph-mon[34321]: pgmap v485: 129 pgs: 129 active+clean; 515 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-05T12:43:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:10 smithi123 ceph-mon[34321]: osdmap e116: 12 total, 12 up, 12 in 2024-09-05T12:43:11.775 INFO:tasks.workunit.client.0.smithi071.stdout: 10 45980 50.79 MB/sec execute 5 sec latency 34.249 ms 2024-09-05T12:43:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:12 smithi174 ceph-mon[34313]: pgmap v487: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.9 MiB/s wr, 704 op/s 2024-09-05T12:43:12.774 INFO:tasks.workunit.client.0.smithi071.stdout: 10 47005 53.00 MB/sec execute 6 sec latency 65.216 ms 2024-09-05T12:43:12.869 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:12 smithi071 ceph-mon[29845]: pgmap v487: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.9 MiB/s wr, 704 op/s 2024-09-05T12:43:12.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:12 smithi123 ceph-mon[34321]: pgmap v487: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.9 MiB/s wr, 704 op/s 2024-09-05T12:43:13.775 INFO:tasks.workunit.client.0.smithi071.stdout: 10 47800 50.97 MB/sec execute 7 sec latency 68.504 ms 2024-09-05T12:43:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:14 smithi174 ceph-mon[34313]: pgmap v488: 129 pgs: 129 active+clean; 551 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 693 op/s 2024-09-05T12:43:14.775 INFO:tasks.workunit.client.0.smithi071.stdout: 10 48638 49.96 MB/sec execute 8 sec latency 49.714 ms 2024-09-05T12:43:14.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:14 smithi071 ceph-mon[29845]: pgmap v488: 129 pgs: 129 active+clean; 551 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 693 op/s 2024-09-05T12:43:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:14 smithi123 ceph-mon[34321]: pgmap v488: 129 pgs: 129 active+clean; 551 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 693 op/s 2024-09-05T12:43:15.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:15 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:15.775 INFO:tasks.workunit.client.0.smithi071.stdout: 10 49674 50.71 MB/sec execute 9 sec latency 65.956 ms 2024-09-05T12:43:15.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:15 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:15.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:15 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:16 smithi174 ceph-mon[34313]: pgmap v489: 129 pgs: 129 active+clean; 551 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 693 op/s 2024-09-05T12:43:16.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 50526 51.23 MB/sec execute 10 sec latency 74.214 ms 2024-09-05T12:43:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.853+0000 7f8d84000640 1 -- 172.21.15.71:0/3518237056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1091f0 msgr2=0x7f8d7c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.854+0000 7f8d84000640 1 --2- 172.21.15.71:0/3518237056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c110250 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009cd0 tx=0x7f8d7402a910 comp rx=0 tx=0).stop 2024-09-05T12:43:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.854+0000 7f8d84000640 1 -- 172.21.15.71:0/3518237056 shutdown_connections 2024-09-05T12:43:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.854+0000 7f8d84000640 1 --2- 172.21.15.71:0/3518237056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c11b140 0x7f8d7c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.854+0000 7f8d84000640 1 --2- 172.21.15.71:0/3518237056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d7c1095d0 0x7f8d7c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.854+0000 7f8d84000640 1 --2- 172.21.15.71:0/3518237056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c110250 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.854+0000 7f8d84000640 1 -- 172.21.15.71:0/3518237056 >> 172.21.15.71:0/3518237056 conn(0x7f8d7c06ce00 msgr2=0x7f8d7c06d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.855+0000 7f8d84000640 1 -- 172.21.15.71:0/3518237056 shutdown_connections 2024-09-05T12:43:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.855+0000 7f8d84000640 1 -- 172.21.15.71:0/3518237056 wait complete. 2024-09-05T12:43:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.857+0000 7f8d84000640 1 Processor -- start 2024-09-05T12:43:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.858+0000 7f8d84000640 1 -- start start 2024-09-05T12:43:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.859+0000 7f8d84000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c1a72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.859+0000 7f8d84000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1095d0 0x7f8d7c1a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.859+0000 7f8d84000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d7c11b140 0x7f8d7c1abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.859+0000 7f8d81600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c1a72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.860+0000 7f8d81600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c1a72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33340/0 (socket says 172.21.15.71:33340) 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.861+0000 7f8d81600640 1 -- 172.21.15.71:0/1982783099 learned_addr learned my addr 172.21.15.71:0/1982783099 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.860+0000 7f8d80c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1095d0 0x7f8d7c1a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.861+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c120dd0 con 0x7f8d7c11b140 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.861+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c120c50 con 0x7f8d7c1095d0 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.860+0000 7f8d82000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d7c11b140 0x7f8d7c1abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.861+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c120f50 con 0x7f8d7c1091f0 2024-09-05T12:43:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.863+0000 7f8d81600640 1 -- 172.21.15.71:0/1982783099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1095d0 msgr2=0x7f8d7c1a77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:16.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.863+0000 7f8d81600640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1095d0 0x7f8d7c1a77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.863+0000 7f8d81600640 1 -- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d7c11b140 msgr2=0x7f8d7c1abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.864+0000 7f8d81600640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d7c11b140 0x7f8d7c1abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.864+0000 7f8d81600640 1 -- 172.21.15.71:0/1982783099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c1ac290 con 0x7f8d7c1091f0 2024-09-05T12:43:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.864+0000 7f8d80c00640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1095d0 0x7f8d7c1a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.864+0000 7f8d81600640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c1a72a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009430 tx=0x7f8d74003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:16.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.865+0000 7f8d7a000640 1 -- 172.21.15.71:0/1982783099 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740317e0 con 0x7f8d7c1091f0 2024-09-05T12:43:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.865+0000 7f8d7a000640 1 -- 172.21.15.71:0/1982783099 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74031980 con 0x7f8d7c1091f0 2024-09-05T12:43:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.865+0000 7f8d7a000640 1 -- 172.21.15.71:0/1982783099 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d7402fcc0 con 0x7f8d7c1091f0 2024-09-05T12:43:16.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.865+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c1ac520 con 0x7f8d7c1091f0 2024-09-05T12:43:16.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.867+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1b3e10 con 0x7f8d7c1091f0 2024-09-05T12:43:16.871 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.871+0000 7f8d7a000640 1 -- 172.21.15.71:0/1982783099 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8d74004040 con 0x7f8d7c1091f0 2024-09-05T12:43:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.873+0000 7f8d7a000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8d640a2e20 0x7f8d640a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.874+0000 7f8d87031740 1 Processor -- start 2024-09-05T12:43:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.874+0000 7f8d80c00640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8d640a2e20 0x7f8d640a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.874+0000 7f8d7a000640 1 -- 172.21.15.71:0/1982783099 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f8d740f4060 con 0x7f8d7c1091f0 2024-09-05T12:43:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.874+0000 7f8d87031740 1 -- start start 2024-09-05T12:43:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.875+0000 7f8d87031740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.875+0000 7f8d87031740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.875+0000 7f8d87031740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.875+0000 7f8d87031740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55598871f100 con 0x5559887b91e0 2024-09-05T12:43:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.875+0000 7f8d87031740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559886c71b0 con 0x5559887b9b00 2024-09-05T12:43:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.875+0000 7f8d87031740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55598871b0c0 con 0x5559887c43a0 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d81600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33346/0 (socket says 172.21.15.71:33346) 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 -- 172.21.15.71:0/2429003438 learned_addr learned my addr 172.21.15.71:0/2429003438 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 -- 172.21.15.71:0/2429003438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 msgr2=0x5559887c3e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c3e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 -- 172.21.15.71:0/2429003438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 msgr2=0x5559887b95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.876+0000 7f8d82000640 1 -- 172.21.15.71:0/2429003438 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555988432210 con 0x5559887c43a0 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.877+0000 7f8d82000640 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c6790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d700099c0 tx=0x7f8d70009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc3e49ae49b96cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.877+0000 7f8d6a600640 1 -- 172.21.15.71:0/2429003438 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017070 con 0x5559887c43a0 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.877+0000 7f8d6a600640 1 -- 172.21.15.71:0/2429003438 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d70013470 con 0x5559887c43a0 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.877+0000 7f8d6a600640 1 -- 172.21.15.71:0/2429003438 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70012630 con 0x5559887c43a0 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.878+0000 7f8d87031740 1 -- 172.21.15.71:0/2429003438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 msgr2=0x5559887c6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.878+0000 7f8d87031740 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c6790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8d700099c0 tx=0x7f8d70009e80 comp rx=0 tx=0).stop 2024-09-05T12:43:16.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.878+0000 7f8d80c00640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8d640a2e20 0x7f8d640a52e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c11d500 tx=0x7f8d6c009110 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.879+0000 7f8d87031740 1 -- 172.21.15.71:0/2429003438 shutdown_connections 2024-09-05T12:43:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.879+0000 7f8d87031740 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c6790 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.879+0000 7f8d87031740 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.879+0000 7f8d87031740 1 --2- 172.21.15.71:0/2429003438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887b95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.879+0000 7f8d87031740 1 -- 172.21.15.71:0/2429003438 >> 172.21.15.71:0/2429003438 conn(0x5559887b5750 msgr2=0x5559887b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:16.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.879+0000 7f8d87031740 1 -- 172.21.15.71:0/2429003438 shutdown_connections 2024-09-05T12:43:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.880+0000 7f8d87031740 1 -- 172.21.15.71:0/2429003438 wait complete. 2024-09-05T12:43:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.880+0000 7f8d87031740 1 -- start start 2024-09-05T12:43:16.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.880+0000 7f8d87031740 1 Processor -- start 2024-09-05T12:43:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.882+0000 7f8d87031740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.882+0000 7f8d87031740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.882+0000 7f8d87031740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.882+0000 7f8d80c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.882+0000 7f8d80c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:39692/0 (socket says 172.21.15.71:39692) 2024-09-05T12:43:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d80c00640 1 -- 172.21.15.71:0/282002906 learned_addr learned my addr 172.21.15.71:0/282002906 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d82000640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d81600640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5559887c6fd0 con 0x5559887b91e0 2024-09-05T12:43:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559886a8090 con 0x5559887b9b00 2024-09-05T12:43:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5559887c7150 con 0x5559887c43a0 2024-09-05T12:43:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.883+0000 7f8d87031740 10 client.0 _ll_register_callbacks cb 0x5559886fcf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:16.885 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.884+0000 7f8d82000640 1 -- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 msgr2=0x5559887c1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.884+0000 7f8d82000640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.884+0000 7f8d82000640 1 -- 172.21.15.71:0/282002906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 msgr2=0x5559887c1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.884+0000 7f8d82000640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887c1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.884+0000 7f8d82000640 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559884fe620 con 0x5559887c43a0 2024-09-05T12:43:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.885+0000 7f8d82000640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c1dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d70013e70 tx=0x7f8d7000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cdffecbb823d34e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:16.886 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.885+0000 7f8d5ea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:43:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.885+0000 7f8d5ea00640 1 -- 172.21.15.71:0/282002906 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017040 con 0x5559887c43a0 2024-09-05T12:43:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.885+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d70013470 con 0x5559887c43a0 2024-09-05T12:43:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.885+0000 7f8d5ea00640 1 -- 172.21.15.71:0/282002906 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f8d7001c450 con 0x5559887c43a0 2024-09-05T12:43:16.887 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.887+0000 7f8d5ea00640 1 -- 172.21.15.71:0/282002906 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8d7001c650 con 0x5559887c43a0 2024-09-05T12:43:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.887+0000 7f8d87031740 10 client.27175 fetch_fsmap learned FSMap version 65 2024-09-05T12:43:16.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.887+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x5559887d2c30 con 0x5559887c43a0 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.888+0000 7f8d5ea00640 1 -- 172.21.15.71:0/282002906 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d70013470 con 0x5559887c43a0 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d5ea00640 1 -- 172.21.15.71:0/282002906 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 65) ==== 8426+0+0 (secure 0 0 0) 0x7f8d70023070 con 0x5559887c43a0 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d87031740 10 client.27175 fetch_fsmap finished waiting for FSMap version 65 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d87031740 10 client.27175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d87031740 20 client.27175 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d87031740 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5559887d3f80 0x5559887d6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d87031740 4 client.27175 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:16.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55598872c830 con 0x5559887d3f80 2024-09-05T12:43:16.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.890+0000 7f8d81600640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5559887d3f80 0x5559887d6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.892+0000 7f8d81600640 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5559887d3f80 0x5559887d6370 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c855c59dcf9e1d server_cookie=7bd4e52bbcd189fc in_seq=0 out_seq=0 2024-09-05T12:43:16.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.892+0000 7f8d5ea00640 10 client.27175 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.893+0000 7f8d5ea00640 1 -- 172.21.15.71:0/282002906 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55598872c830 con 0x5559887d3f80 2024-09-05T12:43:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.893+0000 7f8d5ea00640 10 client.27175 handle_command_reply: tid=0 2024-09-05T12:43:16.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.893+0000 7f8d87031740 1 client.27175 shutdown 2024-09-05T12:43:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.894+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 msgr2=0x5559887c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:16.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.894+0000 7f8d87031740 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c1dc0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8d70013e70 tx=0x7f8d7000ee50 comp rx=0 tx=0).stop 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 shutdown_connections 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d87031740 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5559887d3f80 0x5559887d6370 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d87031740 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887c1dc0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d87031740 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b9b00 0x5559887c1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d87031740 1 --2- 172.21.15.71:0/282002906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b91e0 0x5559887c1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 >> 172.21.15.71:0/282002906 conn(0x5559887b5750 msgr2=0x5559887bb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.895+0000 7f8d5ea00640 0 client.27175 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:16.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.896+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 shutdown_connections 2024-09-05T12:43:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.896+0000 7f8d87031740 1 -- 172.21.15.71:0/282002906 wait complete. 2024-09-05T12:43:16.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.897+0000 7f8d87031740 20 client.27175 trim_cache size 0 max 16384 2024-09-05T12:43:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:16 smithi071 ceph-mon[29845]: pgmap v489: 129 pgs: 129 active+clean; 551 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 693 op/s 2024-09-05T12:43:16.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:16 smithi123 ceph-mon[34321]: pgmap v489: 129 pgs: 129 active+clean; 551 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.0 MiB/s wr, 693 op/s 2024-09-05T12:43:16.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.930+0000 7f8d87031740 1 Processor -- start 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.930+0000 7f8d87031740 1 -- start start 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.930+0000 7f8d87031740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 0x5559887bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.930+0000 7f8d87031740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.930+0000 7f8d87031740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.931+0000 7f8d87031740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1bdeb0 con 0x5559887b9b00 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.931+0000 7f8d87031740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1bfbd0 con 0x5559887b91e0 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.931+0000 7f8d87031740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1be5e0 con 0x5559887c43a0 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.931+0000 7f8d81600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 0x5559887bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.931+0000 7f8d81600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 0x5559887bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:39708/0 (socket says 172.21.15.71:39708) 2024-09-05T12:43:16.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.931+0000 7f8d81600640 1 -- 172.21.15.71:0/293893929 learned_addr learned my addr 172.21.15.71:0/293893929 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.932+0000 7f8d82000640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.932+0000 7f8d80c00640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.933+0000 7f8d81600640 1 -- 172.21.15.71:0/293893929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 msgr2=0x5559887dbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.933+0000 7f8d81600640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887dbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.933+0000 7f8d81600640 1 -- 172.21.15.71:0/293893929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 msgr2=0x5559887c0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.933+0000 7f8d81600640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.933+0000 7f8d81600640 1 -- 172.21.15.71:0/293893929 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559887bcf40 con 0x5559887b91e0 2024-09-05T12:43:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.933+0000 7f8d82000640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.934+0000 7f8d80c00640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.934+0000 7f8d81600640 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 0x5559887bfbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8d740b90a0 tx=0x7f8d7403e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926bb9f7f3500b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.934+0000 7f8d69c00640 1 -- 172.21.15.71:0/293893929 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740d4030 con 0x5559887b91e0 2024-09-05T12:43:16.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.934+0000 7f8d69c00640 1 -- 172.21.15.71:0/293893929 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d74036040 con 0x5559887b91e0 2024-09-05T12:43:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.934+0000 7f8d69c00640 1 -- 172.21.15.71:0/293893929 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d740ba770 con 0x5559887b91e0 2024-09-05T12:43:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 -- 172.21.15.71:0/293893929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 msgr2=0x5559887bfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:16.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 0x5559887bfbf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8d740b90a0 tx=0x7f8d7403e040 comp rx=0 tx=0).stop 2024-09-05T12:43:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 -- 172.21.15.71:0/293893929 shutdown_connections 2024-09-05T12:43:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887c43a0 0x5559887dbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 --2- 172.21.15.71:0/293893929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887b91e0 0x5559887bfbf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.936+0000 7f8d87031740 1 -- 172.21.15.71:0/293893929 >> 172.21.15.71:0/293893929 conn(0x5559887b5750 msgr2=0x5559887d5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.937+0000 7f8d87031740 1 -- 172.21.15.71:0/293893929 shutdown_connections 2024-09-05T12:43:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.937+0000 7f8d87031740 1 -- 172.21.15.71:0/293893929 wait complete. 2024-09-05T12:43:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.938+0000 7f8d87031740 1 -- start start 2024-09-05T12:43:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.938+0000 7f8d87031740 1 Processor -- start 2024-09-05T12:43:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.939+0000 7f8d87031740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887b91e0 0x5559887c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.939+0000 7f8d87031740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.939+0000 7f8d87031740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 0x555988836ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d82000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 0x555988836ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d82000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 0x555988836ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:39716/0 (socket says 172.21.15.71:39716) 2024-09-05T12:43:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d82000640 1 -- 172.21.15.71:0/863285375 learned_addr learned my addr 172.21.15.71:0/863285375 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d81600640 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887b91e0 0x5559887c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1bf500 con 0x5559887b9b00 2024-09-05T12:43:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1bed70 con 0x5559887c43a0 2024-09-05T12:43:16.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.940+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1bbeb0 con 0x5559887b91e0 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d87031740 10 client.0 _ll_register_callbacks cb 0x7f8d70009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d82000640 1 -- 172.21.15.71:0/863285375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887b91e0 msgr2=0x5559887c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d82000640 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887b91e0 0x5559887c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d82000640 1 -- 172.21.15.71:0/863285375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 msgr2=0x5559887c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d82000640 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d82000640 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559888370a0 con 0x5559887c43a0 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d74036040 con 0x5559887c43a0 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d82000640 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 0x555988836ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009470 tx=0x7f8d7000e6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03ed8117abee011 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:16.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.941+0000 7f8d5ea00640 10 client.27137 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.942+0000 7f8d5ea00640 1 -- 172.21.15.71:0/863285375 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d70017040 con 0x5559887c43a0 2024-09-05T12:43:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.942+0000 7f8d5ea00640 1 -- 172.21.15.71:0/863285375 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f8d70013830 con 0x5559887c43a0 2024-09-05T12:43:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.942+0000 7f8d5ea00640 1 -- 172.21.15.71:0/863285375 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8d70013470 con 0x5559887c43a0 2024-09-05T12:43:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.942+0000 7f8d87031740 10 client.27137 fetch_fsmap learned FSMap version 65 2024-09-05T12:43:16.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.942+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=65}) -- 0x55598883e350 con 0x5559887c43a0 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d5ea00640 1 -- 172.21.15.71:0/863285375 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 65) ==== 8426+0+0 (secure 0 0 0) 0x7f8d70023050 con 0x5559887c43a0 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d87031740 10 client.27137 fetch_fsmap finished waiting for FSMap version 65 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d87031740 10 client.27137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d87031740 20 client.27137 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d87031740 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55598883e940 0x555988840da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d87031740 4 client.27137 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5559887b0d80 con 0x55598883e940 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.943+0000 7f8d81600640 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55598883e940 0x555988840da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:16.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.944+0000 7f8d81600640 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55598883e940 0x555988840da0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e7ddf7939bf2d25 server_cookie=a3405fe0a1ae4d0a in_seq=0 out_seq=0 2024-09-05T12:43:16.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:16.944+0000 7f8d5ea00640 10 client.27137 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.298+0000 7f8d5ea00640 1 -- 172.21.15.71:0/863285375 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5559887b0d80 con 0x55598883e940 2024-09-05T12:43:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.298+0000 7f8d5ea00640 10 client.27137 handle_command_reply: tid=0 2024-09-05T12:43:17.300 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.298+0000 7f8d87031740 1 client.27137 shutdown 2024-09-05T12:43:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 msgr2=0x555988836ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:17.301 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 0x555988836ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009470 tx=0x7f8d7000e6d0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 shutdown_connections 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55598883e940 0x555988840da0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5559887c43a0 0x555988836ab0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5559887b9b00 0x5559887c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 --2- 172.21.15.71:0/863285375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559887b91e0 0x5559887c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d5ea00640 0 client.27137 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:17.302 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.301+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 >> 172.21.15.71:0/863285375 conn(0x5559887b5750 msgr2=0x5559887bfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.302+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 shutdown_connections 2024-09-05T12:43:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.302+0000 7f8d87031740 1 -- 172.21.15.71:0/863285375 wait complete. 2024-09-05T12:43:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.302+0000 7f8d87031740 20 client.27137 trim_cache size 0 max 16384 2024-09-05T12:43:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:43:17.303 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8d640a2e20 msgr2=0x7f8d640a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8d640a2e20 0x7f8d640a52e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c11d500 tx=0x7f8d6c009110 comp rx=0 tx=0).stop 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 msgr2=0x7f8d7c1a72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c1a72a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009430 tx=0x7f8d74003f90 comp rx=0 tx=0).stop 2024-09-05T12:43:17.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 shutdown_connections 2024-09-05T12:43:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8d640a2e20 0x7f8d640a52e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d7c11b140 0x7f8d7c1abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d7c1095d0 0x7f8d7c1a77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.303+0000 7f8d84000640 1 --2- 172.21.15.71:0/1982783099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7c1091f0 0x7f8d7c1a72a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:17.305 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.304+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 >> 172.21.15.71:0/1982783099 conn(0x7f8d7c06ce00 msgr2=0x7f8d7c11b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.305+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 shutdown_connections 2024-09-05T12:43:17.306 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:17.305+0000 7f8d84000640 1 -- 172.21.15.71:0/1982783099 wait complete. 2024-09-05T12:43:17.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 51068 48.33 MB/sec execute 11 sec latency 262.161 ms 2024-09-05T12:43:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:18 smithi174 ceph-mon[34313]: pgmap v490: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 734 op/s 2024-09-05T12:43:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:18 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:18.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 51895 47.80 MB/sec execute 12 sec latency 163.372 ms 2024-09-05T12:43:18.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:18 smithi071 ceph-mon[29845]: pgmap v490: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 734 op/s 2024-09-05T12:43:18.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:18 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:18 smithi123 ceph-mon[34321]: pgmap v490: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 734 op/s 2024-09-05T12:43:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:18 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:19.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:19 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:43:19.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 52850 48.11 MB/sec execute 13 sec latency 82.965 ms 2024-09-05T12:43:19.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:19 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:43:19.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:19 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:43:20.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 53847 49.12 MB/sec execute 14 sec latency 43.069 ms 2024-09-05T12:43:20.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:20 smithi071 ceph-mon[29845]: pgmap v491: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 734 op/s 2024-09-05T12:43:20.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:20 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:43:20.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:20 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:43:20.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:20 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:43:20.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:20 smithi123 ceph-mon[34321]: pgmap v491: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 734 op/s 2024-09-05T12:43:20.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:20 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:43:20.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:20 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:43:20.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:20 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:43:21.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:20 smithi174 ceph-mon[34313]: pgmap v491: 129 pgs: 129 active+clean; 591 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 734 op/s 2024-09-05T12:43:21.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:20 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:43:21.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:20 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:43:21.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:20 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:43:21.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 54470 47.97 MB/sec execute 15 sec latency 97.700 ms 2024-09-05T12:43:22.733 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:22 smithi071 ceph-mon[29845]: pgmap v492: 129 pgs: 129 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-05T12:43:22.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 54630 45.47 MB/sec execute 16 sec latency 886.333 ms 2024-09-05T12:43:22.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:22 smithi123 ceph-mon[34321]: pgmap v492: 129 pgs: 129 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-05T12:43:23.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:22 smithi174 ceph-mon[34313]: pgmap v492: 129 pgs: 129 active+clean; 606 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2024-09-05T12:43:23.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 54633 42.79 MB/sec execute 17 sec latency 1886.404 ms 2024-09-05T12:43:24.706 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:24 smithi071 ceph-mon[29845]: pgmap v493: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 776 op/s 2024-09-05T12:43:24.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 54634 40.42 MB/sec execute 18 sec latency 2743.245 ms 2024-09-05T12:43:24.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:24 smithi123 ceph-mon[34321]: pgmap v493: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 776 op/s 2024-09-05T12:43:25.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:24 smithi174 ceph-mon[34313]: pgmap v493: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 776 op/s 2024-09-05T12:43:25.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 54634 38.29 MB/sec execute 19 sec latency 3743.327 ms 2024-09-05T12:43:26.723 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:26 smithi071 ceph-mon[29845]: pgmap v494: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 691 op/s 2024-09-05T12:43:26.776 INFO:tasks.workunit.client.0.smithi071.stdout: 10 54781 36.81 MB/sec execute 20 sec latency 4395.528 ms 2024-09-05T12:43:26.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:26 smithi123 ceph-mon[34321]: pgmap v494: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 691 op/s 2024-09-05T12:43:27.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:26 smithi174 ceph-mon[34313]: pgmap v494: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 691 op/s 2024-09-05T12:43:27.780 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55038 35.40 MB/sec execute 21 sec latency 695.733 ms 2024-09-05T12:43:28.718 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:28 smithi071 ceph-mon[29845]: pgmap v495: 129 pgs: 129 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 827 op/s 2024-09-05T12:43:28.788 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55041 33.78 MB/sec execute 22 sec latency 1704.354 ms 2024-09-05T12:43:28.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:28 smithi123 ceph-mon[34321]: pgmap v495: 129 pgs: 129 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 827 op/s 2024-09-05T12:43:29.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:28 smithi174 ceph-mon[34313]: pgmap v495: 129 pgs: 129 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 827 op/s 2024-09-05T12:43:29.287 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:43:29.288 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.0 2024-09-05T12:43:29.288 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:43:29.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55041 32.31 MB/sec execute 23 sec latency 2704.451 ms 2024-09-05T12:43:29.816 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:43:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 -- 172.21.15.71:0/4075324392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0076720 msgr2=0x7f3bb0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 --2- 172.21.15.71:0/4075324392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0076720 0x7f3bb0076b00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009900 tx=0x7f3ba002a410 comp rx=0 tx=0).stop 2024-09-05T12:43:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 -- 172.21.15.71:0/4075324392 shutdown_connections 2024-09-05T12:43:30.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 --2- 172.21.15.71:0/4075324392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0112980 0x7f3bb0114d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 --2- 172.21.15.71:0/4075324392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0077040 0x7f3bb0112440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 --2- 172.21.15.71:0/4075324392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0076720 0x7f3bb0076b00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 -- 172.21.15.71:0/4075324392 >> 172.21.15.71:0/4075324392 conn(0x7f3bb00ff9e0 msgr2=0x7f3bb0101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 -- 172.21.15.71:0/4075324392 shutdown_connections 2024-09-05T12:43:30.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 -- 172.21.15.71:0/4075324392 wait complete. 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.436+0000 7f3bb7200640 1 Processor -- start 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 -- start start 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0076720 0x7f3bb01a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 0x7f3bb01a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0112980 0x7f3bb019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01185c0 con 0x7f3bb0112980 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0118440 con 0x7f3bb0077040 2024-09-05T12:43:30.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb0118740 con 0x7f3bb0076720 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 0x7f3bb01a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0076720 0x7f3bb01a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0112980 0x7f3bb019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 0x7f3bb01a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51762/0 (socket says 172.21.15.71:51762) 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0076720 0x7f3bb01a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33102/0 (socket says 172.21.15.71:33102) 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 -- 172.21.15.71:0/2575206617 learned_addr learned my addr 172.21.15.71:0/2575206617 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:30.441 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 -- 172.21.15.71:0/2575206617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0076720 msgr2=0x7f3bb01a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0076720 0x7f3bb01a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 -- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0112980 msgr2=0x7f3bb019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0112980 0x7f3bb019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 -- 172.21.15.71:0/2575206617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb019f010 con 0x7f3bb0077040 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3baf400640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 0x7f3bb01a4c20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00e9f0 tx=0x7f3b9c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bacc00640 1 -- 172.21.15.71:0/2575206617 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00cd80 con 0x7f3bb0077040 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb019f300 con 0x7f3bb0077040 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bacc00640 1 -- 172.21.15.71:0/2575206617 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3b9c005500 con 0x7f3bb0077040 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bacc00640 1 -- 172.21.15.71:0/2575206617 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c010690 con 0x7f3bb0077040 2024-09-05T12:43:30.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.438+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01ab5a0 con 0x7f3bb0077040 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 Processor -- start 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- start start 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bacc00640 1 -- 172.21.15.71:0/2575206617 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3b9c0046c0 con 0x7f3bb0077040 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 0x561f407d15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d1b00 0x561f407dbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561f40737100 con 0x561f407dc3a0 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f406df1b0 con 0x561f407d11e0 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f407330c0 con 0x561f407d1b00 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 0x561f407d15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d1b00 0x561f407dbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 0x561f407d15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:51766/0 (socket says 172.21.15.71:51766) 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 -- 172.21.15.71:0/2456144161 learned_addr learned my addr 172.21.15.71:0/2456144161 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3baf400640 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 -- 172.21.15.71:0/2456144161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d1b00 msgr2=0x561f407dbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d1b00 0x561f407dbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 -- 172.21.15.71:0/2456144161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 msgr2=0x561f407de790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:30.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407de790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 -- 172.21.15.71:0/2456144161 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f4044a210 con 0x561f407d11e0 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 0x561f407d15c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800a4b0 tx=0x7f3ba8009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87160d4cd53053b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3ba5600640 1 -- 172.21.15.71:0/2456144161 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8013790 con 0x561f407d11e0 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3ba5600640 1 -- 172.21.15.71:0/2456144161 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba8019070 con 0x561f407d11e0 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3ba5600640 1 -- 172.21.15.71:0/2456144161 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8014dd0 con 0x561f407d11e0 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bacc00640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b940a2da0 0x7f3b940a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bacc00640 1 -- 172.21.15.71:0/2575206617 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f3b9c0cfc60 con 0x7f3bb0077040 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b940a2da0 0x7f3b940a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2456144161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 msgr2=0x561f407d15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 0x561f407d15c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800a4b0 tx=0x7f3ba8009650 comp rx=0 tx=0).stop 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2456144161 shutdown_connections 2024-09-05T12:43:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407de790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d1b00 0x561f407dbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2456144161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d11e0 0x561f407d15c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2456144161 >> 172.21.15.71:0/2456144161 conn(0x561f407cd750 msgr2=0x561f407cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2456144161 shutdown_connections 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2456144161 wait complete. 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b940a2da0 0x7f3b940a5260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006010 tx=0x7f3ba0000bc0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- start start 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 Processor -- start 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407d9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561f407defd0 con 0x561f407dc3a0 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f406c0090 con 0x561f407d1b00 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f407df150 con 0x561f407d11e0 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bba0c0740 10 client.0 _ll_register_callbacks cb 0x561f40714f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407d9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36626/0 (socket says 172.21.15.71:36626) 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 -- 172.21.15.71:0/1184079654 learned_addr learned my addr 172.21.15.71:0/1184079654 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407d9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3baf400640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 -- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 msgr2=0x561f407d9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407d9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 -- 172.21.15.71:0/1184079654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 msgr2=0x561f407d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407d9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bafe00640 1 -- 172.21.15.71:0/1184079654 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f40516620 con 0x561f407d11e0 2024-09-05T12:43:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3bb5200640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.444+0000 7f3baf400640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407d9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:30.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.497+0000 7f3bafe00640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407d9300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009230 tx=0x7f3ba0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641aeb07360e33f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:30.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.497+0000 7f3b9a000640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:43:30.510 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 1 -- 172.21.15.71:0/1184079654 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba004e070 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba8019070 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 1 -- 172.21.15.71:0/1184079654 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f3ba00048c0 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 1 -- 172.21.15.71:0/1184079654 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3ba005a050 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 10 client.27193 fetch_fsmap learned FSMap version 66 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561f407eab50 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 1 -- 172.21.15.71:0/1184079654 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba8019070 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 1 -- 172.21.15.71:0/1184079654 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 66) ==== 8422+0+0 (secure 0 0 0) 0x7f3ba00035f0 con 0x561f407d11e0 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 10 client.27193 fetch_fsmap finished waiting for FSMap version 66 2024-09-05T12:43:30.511 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 10 client.27193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 20 client.27193 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f407ec010 0x561f407ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 4 client.27193 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f40744830 con 0x561f407ec010 2024-09-05T12:43:30.512 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3baf400640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f407ec010 0x561f407ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3baf400640 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f407ec010 0x561f407ee490 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0fab430d2d33f0 server_cookie=19b0d9186c36a696 in_seq=0 out_seq=0 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 10 client.27193 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 1 -- 172.21.15.71:0/1184079654 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f40744830 con 0x561f407ec010 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 10 client.27193 handle_command_reply: tid=0 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 client.27193 shutdown 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 msgr2=0x561f407d9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.514 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407d9300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009230 tx=0x7f3ba0009260 comp rx=0 tx=0).stop 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 shutdown_connections 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f407ec010 0x561f407ee490 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407dc3a0 0x561f407d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407d9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3b9a000640 0 client.27193 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/1184079654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407d9300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 >> 172.21.15.71:0/1184079654 conn(0x561f407cd750 msgr2=0x561f407d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 shutdown_connections 2024-09-05T12:43:30.515 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 1 -- 172.21.15.71:0/1184079654 wait complete. 2024-09-05T12:43:30.516 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.508+0000 7f3bba0c0740 20 client.27193 trim_cache size 0 max 16384 2024-09-05T12:43:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 Processor -- start 2024-09-05T12:43:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- start start 2024-09-05T12:43:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d1b00 0x561f407f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407dc3a0 0x561f407d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01b54d0 con 0x561f407d1b00 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01b71f0 con 0x561f407dc3a0 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01b5c00 con 0x561f407d11e0 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3baf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d1b00 0x561f407f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33122/0 (socket says 172.21.15.71:33122) 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 -- 172.21.15.71:0/2000642005 learned_addr learned my addr 172.21.15.71:0/2000642005 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 -- 172.21.15.71:0/2000642005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407dc3a0 msgr2=0x561f407d2c40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407dc3a0 0x561f407d2c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 -- 172.21.15.71:0/2000642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d1b00 msgr2=0x561f407f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:30.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d1b00 0x561f407f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 -- 172.21.15.71:0/2000642005 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f407deb00 con 0x561f407d11e0 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407f3da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3ba002aa00 tx=0x7f3ba0003780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9f4e98e1465765 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3ba4c00640 1 -- 172.21.15.71:0/2000642005 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba004e070 con 0x561f407d11e0 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3ba4c00640 1 -- 172.21.15.71:0/2000642005 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0063070 con 0x561f407d11e0 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3ba4c00640 1 -- 172.21.15.71:0/2000642005 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0035040 con 0x561f407d11e0 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2000642005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 msgr2=0x561f407f3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407f3da0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3ba002aa00 tx=0x7f3ba0003780 comp rx=0 tx=0).stop 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2000642005 shutdown_connections 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407dc3a0 0x561f407d2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d1b00 0x561f407f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2000642005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407d11e0 0x561f407f3da0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2000642005 >> 172.21.15.71:0/2000642005 conn(0x561f407cd750 msgr2=0x561f407d83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:30.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2000642005 shutdown_connections 2024-09-05T12:43:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2000642005 wait complete. 2024-09-05T12:43:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 -- start start 2024-09-05T12:43:30.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.526+0000 7f3bba0c0740 1 Processor -- start 2024-09-05T12:43:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 0x561f407e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407dc3a0 0x561f407dff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01b6b20 con 0x561f407d11e0 2024-09-05T12:43:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01b6390 con 0x561f407d1b00 2024-09-05T12:43:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb01b34d0 con 0x561f407dc3a0 2024-09-05T12:43:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bba0c0740 10 client.0 _ll_register_callbacks cb 0x7f3ba0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 0x561f407e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bafe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 0x561f407e46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:36644/0 (socket says 172.21.15.71:36644) 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bafe00640 1 -- 172.21.15.71:0/2735317858 learned_addr learned my addr 172.21.15.71:0/2735317858 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 -- 172.21.15.71:0/2735317858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407dc3a0 msgr2=0x561f407dff00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407dc3a0 0x561f407dff00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 -- 172.21.15.71:0/2735317858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 msgr2=0x561f407e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 0x561f407e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 -- 172.21.15.71:0/2735317858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f407e04c0 con 0x561f407d1b00 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 0x561f407e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3baf400640 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407df9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0ccfc0 tx=0x7f3b9c09c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43e1f1a03df9185 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3b9a000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3b9a000640 1 -- 172.21.15.71:0/2735317858 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0cd440 con 0x561f407d1b00 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.531+0000 7f3b9a000640 1 -- 172.21.15.71:0/2735317858 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f3b9c0a4570 con 0x561f407d1b00 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba0063070 con 0x561f407d1b00 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3b9a000640 1 -- 172.21.15.71:0/2735317858 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0a4a00 con 0x561f407d1b00 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 10 client.27149 fetch_fsmap learned FSMap version 66 2024-09-05T12:43:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x561f40856bb0 con 0x561f407d1b00 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3b9a000640 1 -- 172.21.15.71:0/2735317858 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0063070 con 0x561f407d1b00 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3b9a000640 1 -- 172.21.15.71:0/2735317858 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 66) ==== 8422+0+0 (secure 0 0 0) 0x7f3b9c09ac50 con 0x561f407d1b00 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 10 client.27149 fetch_fsmap finished waiting for FSMap version 66 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 10 client.27149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 20 client.27149 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f408573d0 0x561f408597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 4 client.27149 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f4073eb80 con 0x561f408573d0 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f408573d0 0x561f408597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3b9a000640 10 client.27149 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.532+0000 7f3bafe00640 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f408573d0 0x561f408597f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb20125360a9c9f4 server_cookie=7d0d5287f7173bd4 in_seq=0 out_seq=0 2024-09-05T12:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.671+0000 7f3b9a000640 1 -- 172.21.15.71:0/2735317858 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f4073eb80 con 0x561f408573d0 2024-09-05T12:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3b9a000640 10 client.27149 handle_command_reply: tid=0 2024-09-05T12:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 client.27149 shutdown 2024-09-05T12:43:30.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 msgr2=0x561f407df9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407df9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0ccfc0 tx=0x7f3b9c09c550 comp rx=0 tx=0).stop 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 shutdown_connections 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561f408573d0 0x561f408597f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f407dc3a0 0x561f407dff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f407d1b00 0x561f407df9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3b9a000640 0 client.27149 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 --2- 172.21.15.71:0/2735317858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561f407d11e0 0x561f407e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 >> 172.21.15.71:0/2735317858 conn(0x561f407cd750 msgr2=0x561f407d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 shutdown_connections 2024-09-05T12:43:30.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 1 -- 172.21.15.71:0/2735317858 wait complete. 2024-09-05T12:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bba0c0740 20 client.27149 trim_cache size 0 max 16384 2024-09-05T12:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:43:30.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b940a2da0 msgr2=0x7f3b940a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b940a2da0 0x7f3b940a5260 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0006010 tx=0x7f3ba0000bc0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 msgr2=0x7f3bb01a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 0x7f3bb01a4c20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00e9f0 tx=0x7f3b9c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 shutdown_connections 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f3b940a2da0 0x7f3b940a5260 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0112980 0x7f3bb019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb0077040 0x7f3bb01a4c20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 --2- 172.21.15.71:0/2575206617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb0076720 0x7f3bb01a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 >> 172.21.15.71:0/2575206617 conn(0x7f3bb00ff9e0 msgr2=0x7f3bb0101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 shutdown_connections 2024-09-05T12:43:30.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:30.672+0000 7f3bb7200640 1 -- 172.21.15.71:0/2575206617 wait complete. 2024-09-05T12:43:30.687 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:30 smithi071 ceph-mon[29845]: pgmap v496: 129 pgs: 129 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 522 op/s 2024-09-05T12:43:30.788 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55041 30.97 MB/sec execute 24 sec latency 3704.505 ms 2024-09-05T12:43:30.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:30 smithi123 ceph-mon[34321]: pgmap v496: 129 pgs: 129 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 522 op/s 2024-09-05T12:43:31.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:30 smithi174 ceph-mon[34313]: pgmap v496: 129 pgs: 129 active+clean; 632 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.5 MiB/s wr, 522 op/s 2024-09-05T12:43:31.788 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55041 29.73 MB/sec execute 25 sec latency 4704.545 ms 2024-09-05T12:43:32.683 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:32 smithi071 ceph-mon[29845]: pgmap v497: 129 pgs: 129 active+clean; 680 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 775 op/s 2024-09-05T12:43:32.788 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55041 28.59 MB/sec execute 26 sec latency 5704.597 ms 2024-09-05T12:43:32.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:32 smithi123 ceph-mon[34321]: pgmap v497: 129 pgs: 129 active+clean; 680 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 775 op/s 2024-09-05T12:43:33.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:32 smithi174 ceph-mon[34313]: pgmap v497: 129 pgs: 129 active+clean; 680 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 775 op/s 2024-09-05T12:43:33.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55043 27.53 MB/sec execute 27 sec latency 6704.652 ms 2024-09-05T12:43:34.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55071 26.70 MB/sec execute 28 sec latency 7704.695 ms 2024-09-05T12:43:34.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:34 smithi071 ceph-mon[29845]: pgmap v498: 129 pgs: 129 active+clean; 697 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 575 op/s 2024-09-05T12:43:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:34 smithi123 ceph-mon[34321]: pgmap v498: 129 pgs: 129 active+clean; 697 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 575 op/s 2024-09-05T12:43:35.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:34 smithi174 ceph-mon[34313]: pgmap v498: 129 pgs: 129 active+clean; 697 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 15 MiB/s wr, 575 op/s 2024-09-05T12:43:35.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55071 25.78 MB/sec execute 29 sec latency 8704.752 ms 2024-09-05T12:43:36.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55071 24.92 MB/sec execute 30 sec latency 9704.809 ms 2024-09-05T12:43:36.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:36 smithi071 ceph-mon[29845]: pgmap v499: 129 pgs: 129 active+clean; 697 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 467 op/s 2024-09-05T12:43:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:36 smithi123 ceph-mon[34321]: pgmap v499: 129 pgs: 129 active+clean; 697 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 467 op/s 2024-09-05T12:43:37.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:36 smithi174 ceph-mon[34313]: pgmap v499: 129 pgs: 129 active+clean; 697 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 467 op/s 2024-09-05T12:43:37.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55071 24.11 MB/sec execute 31 sec latency 10704.864 ms 2024-09-05T12:43:37.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:37 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:37.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:37 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:38.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:37 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:38.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55071 23.36 MB/sec execute 32 sec latency 11704.902 ms 2024-09-05T12:43:38.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:38 smithi071 ceph-mon[29845]: pgmap v500: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 565 op/s 2024-09-05T12:43:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:38 smithi123 ceph-mon[34321]: pgmap v500: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 565 op/s 2024-09-05T12:43:39.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:38 smithi174 ceph-mon[34313]: pgmap v500: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 14 MiB/s wr, 565 op/s 2024-09-05T12:43:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 -- 172.21.15.71:0/1096616085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 msgr2=0x7fbd6c10def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1096616085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c10def0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009900 tx=0x7fbd5802a4b0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 -- 172.21.15.71:0/1096616085 shutdown_connections 2024-09-05T12:43:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1096616085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c10e430 0x7fbd6c110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1096616085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c10def0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1096616085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c0745c0 0x7fbd6c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 -- 172.21.15.71:0/1096616085 >> 172.21.15.71:0/1096616085 conn(0x7fbd6c0fd8b0 msgr2=0x7fbd6c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 -- 172.21.15.71:0/1096616085 shutdown_connections 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.631+0000 7fbd73e00640 1 -- 172.21.15.71:0/1096616085 wait complete. 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.633+0000 7fbd73e00640 1 Processor -- start 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- start start 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c0745c0 0x7fbd6c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 0x7fbd6c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c114250 con 0x7fbd6c074ee0 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1140d0 con 0x7fbd6c0745c0 2024-09-05T12:43:39.669 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c1143d0 con 0x7fbd6c10e430 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 0x7fbd6c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 0x7fbd6c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40704/0 (socket says 172.21.15.71:40704) 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 -- 172.21.15.71:0/1925436239 learned_addr learned my addr 172.21.15.71:0/1925436239 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71400640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c0745c0 0x7fbd6c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd70a00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 -- 172.21.15.71:0/1925436239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c0745c0 msgr2=0x7fbd6c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c0745c0 0x7fbd6c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 -- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 msgr2=0x7fbd6c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 -- 172.21.15.71:0/1925436239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c19f6e0 con 0x7fbd6c10e430 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd70a00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd71e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 0x7fbd6c19f000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b9e0 tx=0x7fbd6400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:39.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64019070 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd64014440 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c19f9d0 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640125c0 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c1a72b0 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c0749a0 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.634+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fbd6401c070 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.640+0000 7fbd6a000640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd540a2d70 0x7fbd540a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.640+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7fbd640cf120 con 0x7fbd6c10e430 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.640+0000 7fbd71400640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd540a2d70 0x7fbd540a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.640+0000 7fbd71400640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd540a2d70 0x7fbd540a5230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009bd0 tx=0x7fbd60005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:39.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.640+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd6409b410 con 0x7fbd6c10e430 2024-09-05T12:43:39.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55117 22.79 MB/sec execute 33 sec latency 12704.943 ms 2024-09-05T12:43:39.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.861+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd6c109610 con 0x7fbd6c10e430 2024-09-05T12:43:39.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.862+0000 7fbd6a000640 1 -- 172.21.15.71:0/1925436239 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+11988 (secure 0 0 0) 0x7fbd6409f710 con 0x7fbd6c10e430 2024-09-05T12:43:39.864 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:43:39.864 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":67,"btime":"2024-09-05T12:43:36:568563+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":67,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:43:35.798228+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:43:39.864 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 67 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd540a2d70 msgr2=0x7fbd540a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd540a2d70 0x7fbd540a5230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009bd0 tx=0x7fbd60005f90 comp rx=0 tx=0).stop 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 msgr2=0x7fbd6c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 0x7fbd6c19f000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400b9e0 tx=0x7fbd6400bea0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 shutdown_connections 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd6c10e430 0x7fbd6c19f000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fbd540a2d70 0x7fbd540a5230 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd6c074ee0 0x7fbd6c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 --2- 172.21.15.71:0/1925436239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c0745c0 0x7fbd6c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:39.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 >> 172.21.15.71:0/1925436239 conn(0x7fbd6c0fd8b0 msgr2=0x7fbd6c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 shutdown_connections 2024-09-05T12:43:39.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:39.864+0000 7fbd73e00640 1 -- 172.21.15.71:0/1925436239 wait complete. 2024-09-05T12:43:40.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:40 smithi071 ceph-mon[29845]: pgmap v501: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-09-05T12:43:40.686 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1925436239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:43:40.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55191 22.28 MB/sec execute 34 sec latency 13704.985 ms 2024-09-05T12:43:40.862 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0' 2024-09-05T12:43:40.862 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0 2024-09-05T12:43:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:40 smithi123 ceph-mon[34321]: pgmap v501: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-09-05T12:43:40.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1925436239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:43:41.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:40 smithi174 ceph-mon[34313]: pgmap v501: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 11 MiB/s wr, 429 op/s 2024-09-05T12:43:41.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1925436239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:43:41.300 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:43:41.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 21.84 MB/sec execute 35 sec latency 14705.058 ms 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 -- 172.21.15.71:0/591262827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 msgr2=0x7fb910102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 --2- 172.21.15.71:0/591262827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb910102f70 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb904009900 tx=0x7fb90402a480 comp rx=0 tx=0).stop 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 -- 172.21.15.71:0/591262827 shutdown_connections 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 --2- 172.21.15.71:0/591262827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb91010bfc0 0x7fb91010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 --2- 172.21.15.71:0/591262827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9101034b0 0x7fb91010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 --2- 172.21.15.71:0/591262827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb910102f70 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 -- 172.21.15.71:0/591262827 >> 172.21.15.71:0/591262827 conn(0x7fb9100fc610 msgr2=0x7fb9100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 -- 172.21.15.71:0/591262827 shutdown_connections 2024-09-05T12:43:41.990 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 -- 172.21.15.71:0/591262827 wait complete. 2024-09-05T12:43:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 Processor -- start 2024-09-05T12:43:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.984+0000 7fb916400640 1 -- start start 2024-09-05T12:43:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb916400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb91019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb916400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9101034b0 0x7fb91019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:41.992 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb916400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb91010bfc0 0x7fb91019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb916400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb910111e10 con 0x7fb910102b90 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb916400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb910111c90 con 0x7fb91010bfc0 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb916400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb910111f90 con 0x7fb9101034b0 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb91019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb91019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45746/0 (socket says 172.21.15.71:45746) 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915000640 1 -- 172.21.15.71:0/2990439920 learned_addr learned my addr 172.21.15.71:0/2990439920 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb91010bfc0 0x7fb91019ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 -- 172.21.15.71:0/2990439920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9101034b0 msgr2=0x7fb91019aa90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9101034b0 0x7fb91019aa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 -- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 msgr2=0x7fb91019a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:41.993 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb91019a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 -- 172.21.15.71:0/2990439920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91019f500 con 0x7fb91010bfc0 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.989+0000 7fb915a00640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb91010bfc0 0x7fb91019ee20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb90800da10 tx=0x7fb90800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90d600640 1 -- 172.21.15.71:0/2990439920 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90800bd70 con 0x7fb91010bfc0 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb91019f7f0 con 0x7fb91010bfc0 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90d600640 1 -- 172.21.15.71:0/2990439920 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb908003e50 con 0x7fb91010bfc0 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90d600640 1 -- 172.21.15.71:0/2990439920 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080106a0 con 0x7fb91010bfc0 2024-09-05T12:43:41.994 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a70d0 con 0x7fb91010bfc0 2024-09-05T12:43:41.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90d600640 1 -- 172.21.15.71:0/2990439920 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fb908003970 con 0x7fb91010bfc0 2024-09-05T12:43:41.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 Processor -- start 2024-09-05T12:43:41.995 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 -- start start 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324560 0x55c9ef324940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 0x55c9ef32f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef331b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c9ef28a530 con 0x55c9ef324560 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9ef231860 con 0x55c9ef324e80 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9ef2864f0 con 0x55c9ef32f740 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 0x55c9ef32f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:41.997 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324560 0x55c9ef324940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 0x55c9ef32f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49448/0 (socket says 172.21.15.71:49448) 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 -- 172.21.15.71:0/1282976101 learned_addr learned my addr 172.21.15.71:0/1282976101 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef331b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 -- 172.21.15.71:0/1282976101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 msgr2=0x55c9ef331b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef331b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 -- 172.21.15.71:0/1282976101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324560 msgr2=0x55c9ef324940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324560 0x55c9ef324940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 -- 172.21.15.71:0/1282976101 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9eeffbb90 con 0x55c9ef324e80 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324560 0x55c9ef324940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb915000640 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 0x55c9ef32f200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb904005e80 tx=0x7fb904005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e5e442f04242363 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:41.998 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90d600640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb8f80a2cf0 0x7fb8f80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb902000640 1 -- 172.21.15.71:0/1282976101 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904035040 con 0x55c9ef324e80 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb902000640 1 -- 172.21.15.71:0/1282976101 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9040044b0 con 0x55c9ef324e80 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb902000640 1 -- 172.21.15.71:0/1282976101 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904003850 con 0x55c9ef324e80 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90d600640 1 -- 172.21.15.71:0/2990439920 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7fb9080cfc30 con 0x7fb91010bfc0 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.991+0000 7fb90fe00640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb8f80a2cf0 0x7fb8f80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1282976101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 msgr2=0x55c9ef32f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:42.001 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 0x55c9ef32f200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb904005e80 tx=0x7fb904005c00 comp rx=0 tx=0).stop 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1282976101 shutdown_connections 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef331b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324e80 0x55c9ef32f200 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1282976101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324560 0x55c9ef324940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1282976101 >> 172.21.15.71:0/1282976101 conn(0x55c9ef320af0 msgr2=0x55c9ef320ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb90fe00640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb8f80a2cf0 0x7fb8f80a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55c9ef321e70 tx=0x7fb8fc0096d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1282976101 shutdown_connections 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1282976101 wait complete. 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- start start 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 Processor -- start 2024-09-05T12:43:42.002 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef324e80 0x55c9ef32cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef32f740 0x55c9ef32d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c9ef3322b0 con 0x55c9ef32f740 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9ef244e60 con 0x55c9ef324560 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9ef332430 con 0x55c9ef324e80 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 10 client.0 _ll_register_callbacks cb 0x55c9ef249cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49450/0 (socket says 172.21.15.71:49450) 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 -- 172.21.15.71:0/1355063394 learned_addr learned my addr 172.21.15.71:0/1355063394 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915a00640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef32f740 0x55c9ef32d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 -- 172.21.15.71:0/1355063394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef324e80 msgr2=0x55c9ef32cb50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef324e80 0x55c9ef32cb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.003 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 -- 172.21.15.71:0/1355063394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef32f740 msgr2=0x55c9ef32d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef32f740 0x55c9ef32d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 -- 172.21.15.71:0/1355063394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ef250990 con 0x55c9ef324560 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915a00640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef32f740 0x55c9ef32d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb915000640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32c610 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb904031550 tx=0x7fb904004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1178f758f38a5e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb8f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1355063394 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904053060 con 0x55c9ef324560 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1355063394 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7fb9040042b0 con 0x55c9ef324560 2024-09-05T12:43:42.004 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9040044b0 con 0x55c9ef324560 2024-09-05T12:43:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1355063394 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb904035050 con 0x55c9ef324560 2024-09-05T12:43:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 10 client.27164 fetch_fsmap learned FSMap version 67 2024-09-05T12:43:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55c9ef33deb0 con 0x55c9ef324560 2024-09-05T12:43:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1355063394 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9040044b0 con 0x55c9ef324560 2024-09-05T12:43:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1355063394 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 67) ==== 8414+0+0 (secure 0 0 0) 0x7fb904031b40 con 0x55c9ef324560 2024-09-05T12:43:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 10 client.27164 fetch_fsmap finished waiting for FSMap version 67 2024-09-05T12:43:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 10 client.27164 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:43:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 20 client.27164 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef33f200 0x55c9ef3415f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 4 client.27164 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:42.007 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9ef22d310 con 0x55c9ef33f200 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:41.996+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef33f200 0x55c9ef3415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.001+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef33f200 0x55c9ef3415f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef14e317cc42b450 server_cookie=9fe7120bf709cb08 in_seq=0 out_seq=0 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.001+0000 7fb8f6a00640 10 client.27164 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1355063394 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9ef22d310 con 0x55c9ef33f200 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb8f6a00640 10 client.27164 handle_command_reply: tid=0 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 client.27164 shutdown 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 msgr2=0x55c9ef32c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32c610 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb904031550 tx=0x7fb904004aa0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 shutdown_connections 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef33f200 0x55c9ef3415f0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef32f740 0x55c9ef32d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.008 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef324e80 0x55c9ef32cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1355063394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32c610 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 >> 172.21.15.71:0/1355063394 conn(0x55c9ef320af0 msgr2=0x55c9ef31fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:42.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.002+0000 7fb8f6a00640 0 client.27164 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:42.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.014+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 shutdown_connections 2024-09-05T12:43:42.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.014+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1355063394 wait complete. 2024-09-05T12:43:42.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.014+0000 7fb91a4c4740 20 client.27164 trim_cache size 0 max 16384 2024-09-05T12:43:42.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.026+0000 7fb91a4c4740 1 Processor -- start 2024-09-05T12:43:42.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 -- start start 2024-09-05T12:43:42.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef326fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef346c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b1090 con 0x55c9ef324e80 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b2d50 con 0x55c9ef324560 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b17c0 con 0x55c9ef32f740 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49464/0 (socket says 172.21.15.71:49464) 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 -- 172.21.15.71:0/1803007406 learned_addr learned my addr 172.21.15.71:0/1803007406 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef326fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915a00640 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef346c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 -- 172.21.15.71:0/1803007406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 msgr2=0x55c9ef346c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef346c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 -- 172.21.15.71:0/1803007406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 msgr2=0x55c9ef326fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:42.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef326fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb915000640 1 -- 172.21.15.71:0/1803007406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9ef31b160 con 0x55c9ef324560 2024-09-05T12:43:42.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.032+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef326fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:42.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32b550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb904006010 tx=0x7fb904005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c56aa56da37e3524 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:42.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb901600640 1 -- 172.21.15.71:0/1803007406 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904053040 con 0x55c9ef324560 2024-09-05T12:43:42.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb901600640 1 -- 172.21.15.71:0/1803007406 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb904004d50 con 0x55c9ef324560 2024-09-05T12:43:42.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb901600640 1 -- 172.21.15.71:0/1803007406 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904005490 con 0x55c9ef324560 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1803007406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 msgr2=0x55c9ef32b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32b550 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb904006010 tx=0x7fb904005c00 comp rx=0 tx=0).stop 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1803007406 shutdown_connections 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef346c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef326fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1803007406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef32b550 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1803007406 >> 172.21.15.71:0/1803007406 conn(0x55c9ef320af0 msgr2=0x55c9ef32e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1803007406 shutdown_connections 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1803007406 wait complete. 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- start start 2024-09-05T12:43:42.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 Processor -- start 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef337f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef333260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef3339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b2680 con 0x55c9ef324e80 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b1ef0 con 0x55c9ef324560 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101af090 con 0x55c9ef32f740 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 10 client.0 _ll_register_callbacks cb 0x7fb904005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:42.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef337f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef337f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49474/0 (socket says 172.21.15.71:49474) 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 -- 172.21.15.71:0/1722984755 learned_addr learned my addr 172.21.15.71:0/1722984755 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915a00640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef3339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef333260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 -- 172.21.15.71:0/1722984755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 msgr2=0x55c9ef3339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef3339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 -- 172.21.15.71:0/1722984755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 msgr2=0x55c9ef333260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef333260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 -- 172.21.15.71:0/1722984755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9ef325ae0 con 0x55c9ef324560 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb915000640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef337f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb904031550 tx=0x7fb904005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1295731fdcc01e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:42.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1722984755 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904053040 con 0x55c9ef324560 2024-09-05T12:43:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1722984755 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7fb904003b40 con 0x55c9ef324560 2024-09-05T12:43:42.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb904004d50 con 0x55c9ef324560 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1722984755 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7fb90406f050 con 0x55c9ef324560 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 10 client.27170 fetch_fsmap learned FSMap version 67 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x55c9ef3a9b60 con 0x55c9ef324560 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1722984755 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb904004d50 con 0x55c9ef324560 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1722984755 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 67) ==== 8414+0+0 (secure 0 0 0) 0x7fb904031950 con 0x55c9ef324560 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 10 client.27170 fetch_fsmap finished waiting for FSMap version 67 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 10 client.27170 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 20 client.27170 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef3aa180 0x55c9ef3ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 4 client.27170 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}] 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}) -- 0x55c9ef297c60 con 0x55c9ef3aa180 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef3aa180 0x55c9ef3ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:42.047 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb90fe00640 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef3aa180 0x55c9ef3ac5a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9009930a89053fd3 server_cookie=3ce7faf7d07c3576 in_seq=0 out_seq=0 2024-09-05T12:43:42.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:42.038+0000 7fb8f6a00640 10 client.27170 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:42.723 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:42 smithi071 ceph-mon[29845]: pgmap v502: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-09-05T12:43:42.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 21.23 MB/sec execute 36 sec latency 15705.126 ms 2024-09-05T12:43:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:42 smithi123 ceph-mon[34321]: pgmap v502: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-09-05T12:43:43.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:42 smithi174 ceph-mon[34313]: pgmap v502: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-09-05T12:43:43.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 20.66 MB/sec execute 37 sec latency 16705.175 ms 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2840217114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341078c0 msgr2=0x7feb34107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- 172.21.15.71:0/2840217114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341078c0 0x7feb34107ca0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7feb24009930 tx=0x7feb2402a420 comp rx=0 tx=0).stop 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2840217114 shutdown_connections 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- 172.21.15.71:0/2840217114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb34102250 0x7feb3410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- 172.21.15.71:0/2840217114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb341018b0 0x7feb34101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- 172.21.15.71:0/2840217114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341078c0 0x7feb34107ca0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2840217114 >> 172.21.15.71:0/2840217114 conn(0x7feb340fd5d0 msgr2=0x7feb340ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2840217114 shutdown_connections 2024-09-05T12:43:43.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2840217114 wait complete. 2024-09-05T12:43:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 Processor -- start 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- start start 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 0x7feb3419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb34102250 0x7feb3419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 0x7feb3419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb34111d20 con 0x7feb341018b0 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb34111ba0 con 0x7feb34102250 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb34111ea0 con 0x7feb341078c0 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 0x7feb3419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 0x7feb3419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46242/0 (socket says 172.21.15.71:46242) 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 -- 172.21.15.71:0/2611681304 learned_addr learned my addr 172.21.15.71:0/2611681304 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33e00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 0x7feb3419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb2be00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb34102250 0x7feb3419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 -- 172.21.15.71:0/2611681304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 msgr2=0x7feb3419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 0x7feb3419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 -- 172.21.15.71:0/2611681304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb34102250 msgr2=0x7feb3419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb34102250 0x7feb3419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 -- 172.21.15.71:0/2611681304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3419f480 con 0x7feb341018b0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33e00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 0x7feb3419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb33400640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 0x7feb3419a490 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb24009230 tx=0x7feb24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb30c00640 1 -- 172.21.15.71:0/2611681304 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2403f030 con 0x7feb341018b0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb30c00640 1 -- 172.21.15.71:0/2611681304 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb24003ad0 con 0x7feb341018b0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3419f710 con 0x7feb341018b0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb30c00640 1 -- 172.21.15.71:0/2611681304 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb24035070 con 0x7feb341018b0 2024-09-05T12:43:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.793+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb341a7000 con 0x7feb341018b0 2024-09-05T12:43:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 Processor -- start 2024-09-05T12:43:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 -- start start 2024-09-05T12:43:43.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb30c00640 1 -- 172.21.15.71:0/2611681304 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7feb24042030 con 0x7feb341018b0 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 0x56100ff8b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ff96030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff98960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56100fef1100 con 0x56100ff8bcd0 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56100fe991b0 con 0x56100ff8b3b0 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb3ce93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56100feed0c0 con 0x56100ff96570 2024-09-05T12:43:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb33400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ff96030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.799+0000 7feb33e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 0x56100ff8b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 0x56100ff8b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:56358/0 (socket says 172.21.15.71:56358) 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ff96030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46256/0 (socket says 172.21.15.71:46256) 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/1567806058 learned_addr learned my addr 172.21.15.71:0/1567806058 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb2be00640 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff98960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/1567806058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 msgr2=0x56100ff98960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff98960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/1567806058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 msgr2=0x56100ff96030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ff96030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/1567806058 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56100fc04210 con 0x56100ff8b3b0 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 0x56100ff8b790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb2c011790 tx=0x7feb2c011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b2a5d9d8827058b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb23e00640 1 -- 172.21.15.71:0/1567806058 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003c30 con 0x56100ff8b3b0 2024-09-05T12:43:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb23e00640 1 -- 172.21.15.71:0/1567806058 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb2c00fe80 con 0x56100ff8b3b0 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb23e00640 1 -- 172.21.15.71:0/1567806058 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0193e0 con 0x56100ff8b3b0 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb30c00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feb180a2cf0 0x7feb180a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb30c00640 1 -- 172.21.15.71:0/2611681304 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7feb240efe80 con 0x7feb341018b0 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33400640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feb180a2cf0 0x7feb180a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/1567806058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 msgr2=0x56100ff8b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 0x56100ff8b790 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feb2c011790 tx=0x7feb2c011c50 comp rx=0 tx=0).stop 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/1567806058 shutdown_connections 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff98960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ff96030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- 172.21.15.71:0/1567806058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8b3b0 0x56100ff8b790 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/1567806058 >> 172.21.15.71:0/1567806058 conn(0x56100ff87920 msgr2=0x56100ff87d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/1567806058 shutdown_connections 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/1567806058 wait complete. 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33400640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feb180a2cf0 0x7feb180a51b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb240b8980 tx=0x7feb240f3930 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- start start 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 Processor -- start 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 0x56100ff93610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 0x56100ff93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56100ff98ff0 con 0x56100ff8b3b0 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56100fe7a090 con 0x56100ff8bcd0 2024-09-05T12:43:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56100ff99170 con 0x56100ff96570 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 10 client.0 _ll_register_callbacks cb 0x56100fecef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 0x56100ff93610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 0x56100ff93610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46266/0 (socket says 172.21.15.71:46266) 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/655364121 learned_addr learned my addr 172.21.15.71:0/655364121 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/655364121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 msgr2=0x56100ff94090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff94090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 msgr2=0x56100ff93b50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 0x56100ff93b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 -- 172.21.15.71:0/655364121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56100fcd0620 con 0x56100ff8b3b0 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb33e00640 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 0x56100ff93610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00a010 tx=0x7feb2c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b495b298cf4eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb20c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:43:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb20c00640 1 -- 172.21.15.71:0/655364121 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c01f040 con 0x56100ff8b3b0 2024-09-05T12:43:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb20c00640 1 -- 172.21.15.71:0/655364121 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7feb2c011250 con 0x56100ff8b3b0 2024-09-05T12:43:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56100fefe830 con 0x56100ff8b3b0 2024-09-05T12:43:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb20c00640 1 -- 172.21.15.71:0/655364121 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7feb2c004780 con 0x56100ff8b3b0 2024-09-05T12:43:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 10 client.17187 fetch_fsmap learned FSMap version 67 2024-09-05T12:43:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.800+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x56100ffa4dd0 con 0x56100ff8b3b0 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb20c00640 1 -- 172.21.15.71:0/655364121 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56100fefe830 con 0x56100ff8b3b0 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb20c00640 1 -- 172.21.15.71:0/655364121 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) ==== 8414+0+0 (secure 0 0 0) 0x7feb2c018070 con 0x56100ff8b3b0 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb3ce93740 10 client.17187 fetch_fsmap finished waiting for FSMap version 67 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb3ce93740 10 client.17187 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb3ce93740 20 client.17187 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb3ce93740 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56100ffa61e0 0x56100ffa85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb3ce93740 4 client.17187 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.805+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56100fef8b80 con 0x56100ffa61e0 2024-09-05T12:43:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.829+0000 7feb2be00640 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56100ffa61e0 0x56100ffa85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.888+0000 7feb2be00640 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56100ffa61e0 0x56100ffa85d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d705a2c60a132fd server_cookie=449dcaf01476c65b in_seq=0 out_seq=0 2024-09-05T12:43:43.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.888+0000 7feb20c00640 10 client.17187 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb20c00640 1 -- 172.21.15.71:0/655364121 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56100fef8b80 con 0x56100ffa61e0 2024-09-05T12:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb20c00640 10 client.17187 handle_command_reply: tid=0 2024-09-05T12:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 client.17187 shutdown 2024-09-05T12:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 msgr2=0x56100ff93610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 0x56100ff93610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00a010 tx=0x7feb2c0046f0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 shutdown_connections 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56100ffa61e0 0x56100ffa85d0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff96570 0x56100ff94090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 0x56100ff93b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 --2- 172.21.15.71:0/655364121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8b3b0 0x56100ff93610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 >> 172.21.15.71:0/655364121 conn(0x56100ff87920 msgr2=0x56100ff8d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb20c00640 0 client.17187 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 shutdown_connections 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 1 -- 172.21.15.71:0/655364121 wait complete. 2024-09-05T12:43:43.915 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.912+0000 7feb3ce93740 20 client.17187 trim_cache size 0 max 16384 2024-09-05T12:43:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 Processor -- start 2024-09-05T12:43:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 -- start start 2024-09-05T12:43:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ffade90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ffae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff96570 0x56100ff8ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b1050 con 0x56100ff8bcd0 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b2d10 con 0x56100ff96570 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb3ce93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b1780 con 0x56100ff8b3b0 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ffae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ffae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46274/0 (socket says 172.21.15.71:46274) 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 -- 172.21.15.71:0/530893438 learned_addr learned my addr 172.21.15.71:0/530893438 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb33e00640 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ffade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb33e00640 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff96570 0x56100ff8ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 -- 172.21.15.71:0/530893438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 msgr2=0x56100ffade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ffade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 -- 172.21.15.71:0/530893438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff96570 msgr2=0x56100ff8ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff96570 0x56100ff8ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb2be00640 1 -- 172.21.15.71:0/530893438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56100ff945e0 con 0x56100ff8bcd0 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.924+0000 7feb33e00640 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff96570 0x56100ff8ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:43:43.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb2be00640 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ffae3d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000c00 tx=0x7feb1c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ab7c38022650fc server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb23400640 1 -- 172.21.15.71:0/530893438 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c005020 con 0x56100ff8bcd0 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb23400640 1 -- 172.21.15.71:0/530893438 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb1c0047a0 con 0x56100ff8bcd0 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb23400640 1 -- 172.21.15.71:0/530893438 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c01b610 con 0x56100ff8bcd0 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/530893438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 msgr2=0x56100ffae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ffae3d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7feb1c000c00 tx=0x7feb1c0093d0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/530893438 shutdown_connections 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff96570 0x56100ff8ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff8bcd0 0x56100ffae3d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- 172.21.15.71:0/530893438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ffade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/530893438 >> 172.21.15.71:0/530893438 conn(0x56100ff87920 msgr2=0x56100ff8e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/530893438 shutdown_connections 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/530893438 wait complete. 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- start start 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 Processor -- start 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ff99be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 0x56100ff9a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 0x561010008e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b2640 con 0x56100ff96570 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb341b1eb0 con 0x56100ff8bcd0 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb341af050 con 0x56100ff8b3b0 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 10 client.0 _ll_register_callbacks cb 0x7feb240f3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 0x561010008e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 0x561010008e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46288/0 (socket says 172.21.15.71:46288) 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 -- 172.21.15.71:0/3564119671 learned_addr learned my addr 172.21.15.71:0/3564119671 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ff99be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 -- 172.21.15.71:0/3564119671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 msgr2=0x56100ff99be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ff99be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 -- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 msgr2=0x56100ff9a120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 0x56100ff9a120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 -- 172.21.15.71:0/3564119671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610100094d0 con 0x56100ff96570 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb33e00640 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 0x561010008e90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0046c0 tx=0x7feb2c0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0d9e3ca7eb624d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb20c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb20c00640 1 -- 172.21.15.71:0/3564119671 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c039070 con 0x56100ff96570 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb20c00640 1 -- 172.21.15.71:0/3564119671 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7feb2c005620 con 0x56100ff96570 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb1c0047a0 con 0x56100ff96570 2024-09-05T12:43:43.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb20c00640 1 -- 172.21.15.71:0/3564119671 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7feb2c005240 con 0x56100ff96570 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 10 client.17193 fetch_fsmap learned FSMap version 67 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x5610100107e0 con 0x56100ff96570 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb20c00640 1 -- 172.21.15.71:0/3564119671 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb1c0047a0 con 0x56100ff96570 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb20c00640 1 -- 172.21.15.71:0/3564119671 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) ==== 8414+0+0 (secure 0 0 0) 0x7feb2c018070 con 0x56100ff96570 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 10 client.17193 fetch_fsmap finished waiting for FSMap version 67 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 20 client.17193 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561010010f60 0x561010013380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 4 client.17193 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.929+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56100fe757c0 con 0x561010010f60 2024-09-05T12:43:43.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.934+0000 7feb2be00640 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561010010f60 0x561010013380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.934+0000 7feb20c00640 10 client.17193 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:43.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.934+0000 7feb2be00640 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561010010f60 0x561010013380 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a006ecedd0b426 server_cookie=f4f26d90dc28ac4e in_seq=0 out_seq=0 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.984+0000 7feb20c00640 1 -- 172.21.15.71:0/3564119671 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56100fe757c0 con 0x561010010f60 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.984+0000 7feb20c00640 10 client.17193 handle_command_reply: tid=0 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 client.17193 shutdown 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 msgr2=0x561010008e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 0x561010008e90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0046c0 tx=0x7feb2c0046f0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 shutdown_connections 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561010010f60 0x561010013380 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56100ff96570 0x561010008e90 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.991 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb20c00640 0 client.17193 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56100ff8bcd0 0x56100ff9a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 --2- 172.21.15.71:0/3564119671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56100ff8b3b0 0x56100ff99be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 >> 172.21.15.71:0/3564119671 conn(0x56100ff87920 msgr2=0x56100ff861a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 shutdown_connections 2024-09-05T12:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 1 -- 172.21.15.71:0/3564119671 wait complete. 2024-09-05T12:43:43.992 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb3ce93740 20 client.17193 trim_cache size 0 max 16384 2024-09-05T12:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:43:43.995 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feb180a2cf0 msgr2=0x7feb180a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feb180a2cf0 0x7feb180a51b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feb240b8980 tx=0x7feb240f3930 comp rx=0 tx=0).stop 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 msgr2=0x7feb3419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 0x7feb3419a490 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7feb24009230 tx=0x7feb24004270 comp rx=0 tx=0).stop 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 shutdown_connections 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb341078c0 0x7feb3419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb34102250 0x7feb3419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.996 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7feb180a2cf0 0x7feb180a51b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 --2- 172.21.15.71:0/2611681304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feb341018b0 0x7feb3419a490 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 >> 172.21.15.71:0/2611681304 conn(0x7feb340fd5d0 msgr2=0x7feb340fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 shutdown_connections 2024-09-05T12:43:43.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:43.988+0000 7feb38a00640 1 -- 172.21.15.71:0/2611681304 wait complete. 2024-09-05T12:43:44.658 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:44 smithi071 ceph-mon[29845]: pgmap v503: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-09-05T12:43:44.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 20.11 MB/sec execute 38 sec latency 17705.248 ms 2024-09-05T12:43:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:44 smithi123 ceph-mon[34321]: pgmap v503: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-09-05T12:43:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:44 smithi174 ceph-mon[34313]: pgmap v503: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 3.0 MiB/s wr, 188 op/s 2024-09-05T12:43:45.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 19.60 MB/sec execute 39 sec latency 18705.298 ms 2024-09-05T12:43:46.616 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:46 smithi071 ceph-mon[29845]: pgmap v504: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 403 KiB/s wr, 110 op/s 2024-09-05T12:43:46.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 19.11 MB/sec execute 40 sec latency 19705.363 ms 2024-09-05T12:43:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:46 smithi123 ceph-mon[34321]: pgmap v504: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 403 KiB/s wr, 110 op/s 2024-09-05T12:43:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:46 smithi174 ceph-mon[34313]: pgmap v504: 129 pgs: 129 active+clean; 655 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 403 KiB/s wr, 110 op/s 2024-09-05T12:43:47.789 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 18.64 MB/sec execute 41 sec latency 20705.445 ms 2024-09-05T12:43:48.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 18.20 MB/sec execute 42 sec latency 21705.492 ms 2024-09-05T12:43:48.856 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:48 smithi071 ceph-mon[29845]: pgmap v505: 129 pgs: 129 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 866 KiB/s wr, 214 op/s 2024-09-05T12:43:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:48 smithi123 ceph-mon[34321]: pgmap v505: 129 pgs: 129 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 866 KiB/s wr, 214 op/s 2024-09-05T12:43:49.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:48 smithi174 ceph-mon[34313]: pgmap v505: 129 pgs: 129 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 913 KiB/s rd, 866 KiB/s wr, 214 op/s 2024-09-05T12:43:49.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 17.77 MB/sec execute 43 sec latency 22493.189 ms 2024-09-05T12:43:50.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 17.37 MB/sec execute 44 sec latency 23376.621 ms 2024-09-05T12:43:50.877 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:50 smithi071 ceph-mon[29845]: pgmap v506: 129 pgs: 129 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 539 KiB/s wr, 116 op/s 2024-09-05T12:43:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:50 smithi123 ceph-mon[34321]: pgmap v506: 129 pgs: 129 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 539 KiB/s wr, 116 op/s 2024-09-05T12:43:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:50 smithi174 ceph-mon[34313]: pgmap v506: 129 pgs: 129 active+clean; 621 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 539 KiB/s wr, 116 op/s 2024-09-05T12:43:51.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55327 16.98 MB/sec execute 45 sec latency 24376.706 ms 2024-09-05T12:43:52.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55328 16.62 MB/sec execute 46 sec latency 25376.788 ms 2024-09-05T12:43:52.828 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:52 smithi071 ceph-mon[29845]: pgmap v507: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 587 KiB/s wr, 122 op/s 2024-09-05T12:43:52.829 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:52 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:52 smithi123 ceph-mon[34321]: pgmap v507: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 587 KiB/s wr, 122 op/s 2024-09-05T12:43:52.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:52 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:52 smithi174 ceph-mon[34313]: pgmap v507: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 587 KiB/s wr, 122 op/s 2024-09-05T12:43:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:52 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:53.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb8f6a00640 1 -- 172.21.15.71:0/1722984755 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c9ef297c60 con 0x55c9ef3aa180 2024-09-05T12:43:53.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb8f6a00640 10 client.27170 handle_command_reply: tid=0 2024-09-05T12:43:53.281 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 client.27170 shutdown 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 msgr2=0x55c9ef337f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef337f30 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb904031550 tx=0x7fb904005990 comp rx=0 tx=0).stop 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 shutdown_connections 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c9ef3aa180 0x55c9ef3ac5a0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9ef32f740 0x55c9ef3339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c9ef324e80 0x55c9ef333260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 --2- 172.21.15.71:0/1722984755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9ef324560 0x55c9ef337f30 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb8f6a00640 0 client.27170 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.278+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 >> 172.21.15.71:0/1722984755 conn(0x55c9ef320af0 msgr2=0x55c9ef31f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 shutdown_connections 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb91a4c4740 1 -- 172.21.15.71:0/1722984755 wait complete. 2024-09-05T12:43:53.282 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb91a4c4740 20 client.27170 trim_cache size 0 max 16384 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 68, 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 50, 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "ccf99444": { 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 50, 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:43:53.283 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 140.0, 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:53.284 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb8f80a2cf0 msgr2=0x7fb8f80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb916400640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb8f80a2cf0 0x7fb8f80a51b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55c9ef321e70 tx=0x7fb8fc0096d0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb91010bfc0 msgr2=0x7fb91019ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb916400640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb91010bfc0 0x7fb91019ee20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb90800da10 tx=0x7fb90800ded0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.279+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 shutdown_connections 2024-09-05T12:43:53.285 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb91010bfc0 0x7fb91019ee20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9101034b0 0x7fb91019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fb8f80a2cf0 0x7fb8f80a51b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 --2- 172.21.15.71:0/2990439920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb910102b90 0x7fb91019a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 >> 172.21.15.71:0/2990439920 conn(0x7fb9100fc610 msgr2=0x7fb9100fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 shutdown_connections 2024-09-05T12:43:53.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:53.280+0000 7fb916400640 1 -- 172.21.15.71:0/2990439920 wait complete. 2024-09-05T12:43:53.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55332 16.26 MB/sec execute 47 sec latency 26081.111 ms 2024-09-05T12:43:54.105 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ccf99444', quiesce took 24.8 seconds. Will release after: -4.800000000000001 2024-09-05T12:43:54.106 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:43:54.479 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:43:54.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55332 15.92 MB/sec execute 48 sec latency 27081.181 ms 2024-09-05T12:43:54.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:54 smithi071 ceph-mon[29845]: pgmap v508: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 513 KiB/s wr, 112 op/s 2024-09-05T12:43:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:54 smithi123 ceph-mon[34321]: pgmap v508: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 513 KiB/s wr, 112 op/s 2024-09-05T12:43:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:54 smithi174 ceph-mon[34313]: pgmap v508: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 513 KiB/s wr, 112 op/s 2024-09-05T12:43:55.148 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 -- 172.21.15.71:0/3966318940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9640101df0 msgr2=0x7f96401021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 --2- 172.21.15.71:0/3966318940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9640101df0 0x7f96401021d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9630009900 tx=0x7f963002a3f0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 -- 172.21.15.71:0/3966318940 shutdown_connections 2024-09-05T12:43:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 --2- 172.21.15.71:0/3966318940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 0x7f964010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 --2- 172.21.15.71:0/3966318940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96401027a0 0x7f96401067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 --2- 172.21.15.71:0/3966318940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9640101df0 0x7f96401021d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.149 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 -- 172.21.15.71:0/3966318940 >> 172.21.15.71:0/3966318940 conn(0x7f96400fd8b0 msgr2=0x7f96400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:55.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 -- 172.21.15.71:0/3966318940 shutdown_connections 2024-09-05T12:43:55.150 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.146+0000 7f9646a00640 1 -- 172.21.15.71:0/3966318940 wait complete. 2024-09-05T12:43:55.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.147+0000 7f9646a00640 1 Processor -- start 2024-09-05T12:43:55.151 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.147+0000 7f9646a00640 1 -- start start 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9646a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9640101df0 0x7f9640198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9646a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 0x7f9640198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9646a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 0x7f964019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9646a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9640111f80 con 0x7f96401027a0 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9646a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9640111e00 con 0x7f9640106f20 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9646a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9640112100 con 0x7f9640101df0 2024-09-05T12:43:55.152 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f963f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 0x7f9640198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f963f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 0x7f9640198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53354/0 (socket says 172.21.15.71:53354) 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f963f400640 1 -- 172.21.15.71:0/2834993449 learned_addr learned my addr 172.21.15.71:0/2834993449 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9644a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 0x7f964019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f963fe00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9640101df0 0x7f9640198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9644a00640 1 -- 172.21.15.71:0/2834993449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9640101df0 msgr2=0x7f9640198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9644a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9640101df0 0x7f9640198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9644a00640 1 -- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 msgr2=0x7f9640198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9644a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 0x7f9640198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f9644a00640 1 -- 172.21.15.71:0/2834993449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f964019d3d0 con 0x7f9640106f20 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.148+0000 7f963f400640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 0x7f9640198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.149+0000 7f9644a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 0x7f964019ccf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f963800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.149+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800ce20 con 0x7f9640106f20 2024-09-05T12:43:55.153 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.149+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f964019d6c0 con 0x7f9640106f20 2024-09-05T12:43:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.149+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9638005020 con 0x7f9640106f20 2024-09-05T12:43:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.149+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638010720 con 0x7f9640106f20 2024-09-05T12:43:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.149+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96401a4fa0 con 0x7f9640106f20 2024-09-05T12:43:55.154 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.150+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96401021d0 con 0x7f9640106f20 2024-09-05T12:43:55.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.152+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9638005540 con 0x7f9640106f20 2024-09-05T12:43:55.155 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.153+0000 7f963cc00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96240a2d70 0x7f96240a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:55.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.154+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f96380cfe00 con 0x7f9640106f20 2024-09-05T12:43:55.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.154+0000 7f963fe00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96240a2d70 0x7f96240a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:55.156 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.154+0000 7f963fe00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96240a2d70 0x7f96240a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9630009230 tx=0x7f96300058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:55.159 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.158+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f963809c630 con 0x7f9640106f20 2024-09-05T12:43:55.368 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.366+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9640109b70 con 0x7f9640106f20 2024-09-05T12:43:55.369 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.368+0000 7f963cc00640 1 -- 172.21.15.71:0/2834993449 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+11986 (secure 0 0 0) 0x7f963809bf30 con 0x7f9640106f20 2024-09-05T12:43:55.370 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:43:55.370 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":68,"btime":"2024-09-05T12:43:51:607972+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":68,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:43:50.799118+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[0,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:43:55.370 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 68 2024-09-05T12:43:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96240a2d70 msgr2=0x7f96240a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96240a2d70 0x7f96240a5230 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9630009230 tx=0x7f96300058d0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 msgr2=0x7f964019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 0x7f964019ccf0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9638009fd0 tx=0x7f963800ef10 comp rx=0 tx=0).stop 2024-09-05T12:43:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 shutdown_connections 2024-09-05T12:43:55.372 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9640106f20 0x7f964019ccf0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f96240a2d70 0x7f96240a5230 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96401027a0 0x7f9640198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 --2- 172.21.15.71:0/2834993449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9640101df0 0x7f9640198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 >> 172.21.15.71:0/2834993449 conn(0x7f96400fd8b0 msgr2=0x7f96400fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 shutdown_connections 2024-09-05T12:43:55.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:55.370+0000 7f9646a00640 1 -- 172.21.15.71:0/2834993449 wait complete. 2024-09-05T12:43:55.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55332 15.60 MB/sec execute 49 sec latency 28081.284 ms 2024-09-05T12:43:55.794 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:55 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2834993449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:43:55.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:55 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2834993449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:43:56.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:55 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2834993449' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:43:56.396 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id ccf99444 --release --await' 2024-09-05T12:43:56.396 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id ccf99444 --release --await 2024-09-05T12:43:56.769 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:43:56.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55332 15.29 MB/sec execute 50 sec latency 29081.330 ms 2024-09-05T12:43:56.908 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:56 smithi071 ceph-mon[29845]: pgmap v509: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 510 KiB/s wr, 110 op/s 2024-09-05T12:43:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:56 smithi123 ceph-mon[34321]: pgmap v509: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 510 KiB/s wr, 110 op/s 2024-09-05T12:43:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:56 smithi174 ceph-mon[34313]: pgmap v509: 129 pgs: 129 active+clean; 622 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 510 KiB/s wr, 110 op/s 2024-09-05T12:43:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 -- 172.21.15.71:0/2029801798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074f40 msgr2=0x7ff9cc10dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2029801798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074f40 0x7ff9cc10dbb0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8009900 tx=0x7ff9b802a480 comp rx=0 tx=0).stop 2024-09-05T12:43:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 -- 172.21.15.71:0/2029801798 shutdown_connections 2024-09-05T12:43:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2029801798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9cc10e0f0 0x7ff9cc110540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2029801798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074f40 0x7ff9cc10dbb0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2029801798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074620 0x7ff9cc074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 -- 172.21.15.71:0/2029801798 >> 172.21.15.71:0/2029801798 conn(0x7ff9cc0fd5d0 msgr2=0x7ff9cc0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 -- 172.21.15.71:0/2029801798 shutdown_connections 2024-09-05T12:43:57.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.107+0000 7ff9d1000640 1 -- 172.21.15.71:0/2029801798 wait complete. 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 Processor -- start 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 -- start start 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074620 0x7ff9cc19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 0x7ff9cc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9cc10e0f0 0x7ff9cc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc113ef0 con 0x7ff9cc074620 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc113d70 con 0x7ff9cc074f40 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.108+0000 7ff9d1000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc114070 con 0x7ff9cc10e0f0 2024-09-05T12:43:57.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074620 0x7ff9cc19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 0x7ff9cc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 0x7ff9cc19a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49230/0 (socket says 172.21.15.71:49230) 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 -- 172.21.15.71:0/2798239465 learned_addr learned my addr 172.21.15.71:0/2798239465 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9cc10e0f0 0x7ff9cc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 -- 172.21.15.71:0/2798239465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9cc10e0f0 msgr2=0x7ff9cc19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9cc10e0f0 0x7ff9cc19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 -- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074620 msgr2=0x7ff9cc19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074620 0x7ff9cc19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 -- 172.21.15.71:0/2798239465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc19f410 con 0x7ff9cc074f40 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9cb400640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074620 0x7ff9cc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.109+0000 7ff9caa00640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 0x7ff9cc19a960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8004770 tx=0x7ff9b8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.110+0000 7ff9c3e00640 1 -- 172.21.15.71:0/2798239465 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b803f070 con 0x7ff9cc074f40 2024-09-05T12:43:57.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.110+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9cc19f6a0 con 0x7ff9cc074f40 2024-09-05T12:43:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.110+0000 7ff9c3e00640 1 -- 172.21.15.71:0/2798239465 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9b8031ca0 con 0x7ff9cc074f40 2024-09-05T12:43:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.110+0000 7ff9c3e00640 1 -- 172.21.15.71:0/2798239465 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8035030 con 0x7ff9cc074f40 2024-09-05T12:43:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.110+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9cc1a6f90 con 0x7ff9cc074f40 2024-09-05T12:43:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 Processor -- start 2024-09-05T12:43:57.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 -- start start 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a2990b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990bcd0 0x558a29916030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a29916570 0x558a29918960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558a29871100 con 0x558a2990bcd0 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558a298191b0 con 0x558a2990b3b0 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a2986d0c0 con 0x558a29916570 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9c3e00640 1 -- 172.21.15.71:0/2798239465 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff9b8042070 con 0x7ff9cc074f40 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a2990b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990bcd0 0x558a29916030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.113+0000 7ff9cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a2990b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49232/0 (socket says 172.21.15.71:49232) 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 -- 172.21.15.71:0/3756099724 learned_addr learned my addr 172.21.15.71:0/3756099724 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9caa00640 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a29916570 0x558a29918960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 -- 172.21.15.71:0/3756099724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a29916570 msgr2=0x558a29918960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a29916570 0x558a29918960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 -- 172.21.15.71:0/3756099724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990bcd0 msgr2=0x558a29916030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990bcd0 0x558a29916030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 -- 172.21.15.71:0/3756099724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a29584210 con 0x558a2990b3b0 2024-09-05T12:43:57.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.114+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a2990b790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009f70 tx=0x7ff9c4010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e4a32fbf9a1887 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9c0c00640 1 -- 172.21.15.71:0/3756099724 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4016070 con 0x558a2990b3b0 2024-09-05T12:43:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9c0c00640 1 -- 172.21.15.71:0/3756099724 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4004d20 con 0x558a2990b3b0 2024-09-05T12:43:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9c0c00640 1 -- 172.21.15.71:0/3756099724 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4012850 con 0x558a2990b3b0 2024-09-05T12:43:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9c3e00640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff9b00a2d70 0x7ff9b00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9c3e00640 1 -- 172.21.15.71:0/2798239465 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7ff9b80efe10 con 0x7ff9cc074f40 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9cb400640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff9b00a2d70 0x7ff9b00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9d54d6740 1 -- 172.21.15.71:0/3756099724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 msgr2=0x558a2990b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.115+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a2990b790 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009f70 tx=0x7ff9c4010800 comp rx=0 tx=0).stop 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 -- 172.21.15.71:0/3756099724 shutdown_connections 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a29916570 0x558a29918960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990bcd0 0x558a29916030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/3756099724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a2990b790 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 -- 172.21.15.71:0/3756099724 >> 172.21.15.71:0/3756099724 conn(0x558a29907920 msgr2=0x558a29907d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 -- 172.21.15.71:0/3756099724 shutdown_connections 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 -- 172.21.15.71:0/3756099724 wait complete. 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9cb400640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff9b00a2d70 0x7ff9b00a5230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc012640 tx=0x7ff9bc005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.116+0000 7ff9d54d6740 1 -- start start 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 Processor -- start 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a299136d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a29913c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a29914150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558a29918ff0 con 0x558a2990b3b0 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558a297fa090 con 0x558a29916570 2024-09-05T12:43:57.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a29919170 con 0x558a2990bcd0 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.117+0000 7ff9d54d6740 10 client.0 _ll_register_callbacks cb 0x558a2984ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a299136d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a299136d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53388/0 (socket says 172.21.15.71:53388) 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 -- 172.21.15.71:0/1395016011 learned_addr learned my addr 172.21.15.71:0/1395016011 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a29914150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9caa00640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a29913c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 -- 172.21.15.71:0/1395016011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 msgr2=0x558a29913c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a29913c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 -- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 msgr2=0x558a29914150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a29914150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 -- 172.21.15.71:0/1395016011 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a29650620 con 0x558a2990b3b0 2024-09-05T12:43:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a29914150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9cb400640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a299136d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005bc0 tx=0x7ff9bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417d49460ab74a02 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.118+0000 7ff9b5600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9b5600640 1 -- 172.21.15.71:0/1395016011 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00f950 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9b5600640 1 -- 172.21.15.71:0/1395016011 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7ff9bc00faf0 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c4004d20 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9b5600640 1 -- 172.21.15.71:0/1395016011 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc00fcf0 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9d54d6740 10 client.17211 fetch_fsmap learned FSMap version 68 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x558a29924e70 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9b5600640 1 -- 172.21.15.71:0/1395016011 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c4004d20 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9b5600640 1 -- 172.21.15.71:0/1395016011 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 68) ==== 8410+0+0 (secure 0 0 0) 0x7ff9bc027070 con 0x558a2990b3b0 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9d54d6740 10 client.17211 fetch_fsmap finished waiting for FSMap version 68 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.119+0000 7ff9d54d6740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9d54d6740 20 client.17211 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29926380 0x558a29928840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9d54d6740 4 client.17211 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a2987e830 con 0x558a29926380 2024-09-05T12:43:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9caa00640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29926380 0x558a29928840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9caa00640 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29926380 0x558a29928840 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce4797e0439675e server_cookie=33476b325e808bff in_seq=0 out_seq=0 2024-09-05T12:43:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.120+0000 7ff9b5600640 10 client.17211 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9b5600640 1 -- 172.21.15.71:0/1395016011 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a2987e830 con 0x558a29926380 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9b5600640 10 client.17211 handle_command_reply: tid=0 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9d54d6740 1 client.17211 shutdown 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 msgr2=0x558a299136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a299136d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc005bc0 tx=0x7ff9bc004730 comp rx=0 tx=0).stop 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 shutdown_connections 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.121+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29926380 0x558a29928840 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.128 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a29914150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a29913c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9b5600640 0 client.17211 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1395016011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a299136d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 >> 172.21.15.71:0/1395016011 conn(0x558a29907920 msgr2=0x558a2990d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 shutdown_connections 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1395016011 wait complete. 2024-09-05T12:43:57.129 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.122+0000 7ff9d54d6740 20 client.17211 trim_cache size 0 max 16384 2024-09-05T12:43:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 Processor -- start 2024-09-05T12:43:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 -- start start 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a2990c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2992e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a2992e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b0fe0 con 0x558a2990b3b0 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b2c70 con 0x558a29916570 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b1710 con 0x558a2990bcd0 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a2990c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a2990c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53394/0 (socket says 172.21.15.71:53394) 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9cb400640 1 -- 172.21.15.71:0/1986965877 learned_addr learned my addr 172.21.15.71:0/1986965877 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.138+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a2992e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9caa00640 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2992e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cb400640 1 -- 172.21.15.71:0/1986965877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 msgr2=0x558a2992e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cb400640 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2992e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cb400640 1 -- 172.21.15.71:0/1986965877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 msgr2=0x558a2992e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cb400640 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a2992e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cb400640 1 -- 172.21.15.71:0/1986965877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a299089d0 con 0x558a2990b3b0 2024-09-05T12:43:57.142 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a2992e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9cb400640 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a2990c550 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0092e0 tx=0x7ff9bc0044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804f7a12c2731001 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9b7e00640 1 -- 172.21.15.71:0/1986965877 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00f950 con 0x558a2990b3b0 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9b7e00640 1 -- 172.21.15.71:0/1986965877 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc00faf0 con 0x558a2990b3b0 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.139+0000 7ff9b7e00640 1 -- 172.21.15.71:0/1986965877 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0054e0 con 0x558a2990b3b0 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1986965877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 msgr2=0x558a2990c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a2990c550 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0092e0 tx=0x7ff9bc0044d0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1986965877 shutdown_connections 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a29916570 0x558a2992e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2992e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/1986965877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a2990b3b0 0x558a2990c550 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1986965877 >> 172.21.15.71:0/1986965877 conn(0x558a29907920 msgr2=0x558a299127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1986965877 shutdown_connections 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 -- 172.21.15.71:0/1986965877 wait complete. 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.140+0000 7ff9d54d6740 1 -- start start 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 Processor -- start 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a29919b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2991a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a29916570 0x558a2998f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b25a0 con 0x558a29916570 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1b1e10 con 0x558a2990b3b0 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc1aefe0 con 0x558a2990bcd0 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9d54d6740 10 client.0 _ll_register_callbacks cb 0x7ff9bc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:57.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a29919b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9cb400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a29919b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49254/0 (socket says 172.21.15.71:49254) 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9cbe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a29916570 0x558a2998f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9cb400640 1 -- 172.21.15.71:0/2940899871 learned_addr learned my addr 172.21.15.71:0/2940899871 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.141+0000 7ff9caa00640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2991a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cb400640 1 -- 172.21.15.71:0/2940899871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 msgr2=0x558a2991a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cb400640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2991a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cb400640 1 -- 172.21.15.71:0/2940899871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a29916570 msgr2=0x558a2998f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cb400640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a29916570 0x558a2998f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cb400640 1 -- 172.21.15.71:0/2940899871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a2998f750 con 0x558a2990b3b0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cbe00640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a29916570 0x558a2998f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9cb400640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a29919b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00fcc0 tx=0x7ff9bc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeee60f68c8bbd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9b5600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9b5600640 1 -- 172.21.15.71:0/2940899871 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc00f950 con 0x558a2990b3b0 2024-09-05T12:43:57.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9bc00faf0 con 0x558a2990b3b0 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.142+0000 7ff9b5600640 1 -- 172.21.15.71:0/2940899871 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7ff9bc012250 con 0x558a2990b3b0 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9b5600640 1 -- 172.21.15.71:0/2940899871 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7ff9bc048050 con 0x558a2990b3b0 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 10 client.27200 fetch_fsmap learned FSMap version 68 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x558a29990a60 con 0x558a2990b3b0 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9b5600640 1 -- 172.21.15.71:0/2940899871 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9bc00faf0 con 0x558a2990b3b0 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9b5600640 1 -- 172.21.15.71:0/2940899871 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 68) ==== 8410+0+0 (secure 0 0 0) 0x7ff9bc027070 con 0x558a2990b3b0 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 10 client.27200 fetch_fsmap finished waiting for FSMap version 68 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 10 client.27200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 20 client.27200 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29991170 0x558a29993590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 4 client.27200 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.143+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a29878b80 con 0x558a29991170 2024-09-05T12:43:57.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.144+0000 7ff9caa00640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29991170 0x558a29993590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.144+0000 7ff9caa00640 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29991170 0x558a29993590 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d10c249da5dfb5 server_cookie=4c201e65f0d8502b in_seq=0 out_seq=0 2024-09-05T12:43:57.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.144+0000 7ff9b5600640 10 client.27200 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9b5600640 1 -- 172.21.15.71:0/2940899871 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a29878b80 con 0x558a29991170 2024-09-05T12:43:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9b5600640 10 client.27200 handle_command_reply: tid=0 2024-09-05T12:43:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 client.27200 shutdown 2024-09-05T12:43:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 msgr2=0x558a29919b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a29919b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00fcc0 tx=0x7ff9bc017000 comp rx=0 tx=0).stop 2024-09-05T12:43:57.179 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 shutdown_connections 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558a29991170 0x558a29993590 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a29916570 0x558a2998f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a2990bcd0 0x558a2991a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 --2- 172.21.15.71:0/2940899871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a2990b3b0 0x558a29919b70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9b5600640 0 client.27200 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 >> 172.21.15.71:0/2940899871 conn(0x558a29907920 msgr2=0x558a299127a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 shutdown_connections 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 1 -- 172.21.15.71:0/2940899871 wait complete. 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.177+0000 7ff9d54d6740 20 client.27200 trim_cache size 0 max 16384 2024-09-05T12:43:57.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff9b00a2d70 msgr2=0x7ff9b00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.181 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff9b00a2d70 0x7ff9b00a5230 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc012640 tx=0x7ff9bc005f90 comp rx=0 tx=0).stop 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 msgr2=0x7ff9cc19a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 0x7ff9cc19a960 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8004770 tx=0x7ff9b8003c10 comp rx=0 tx=0).stop 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 shutdown_connections 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9cc10e0f0 0x7ff9cc19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9cc074f40 0x7ff9cc19a960 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff9b00a2d70 0x7ff9b00a5230 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 --2- 172.21.15.71:0/2798239465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9cc074620 0x7ff9cc19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 >> 172.21.15.71:0/2798239465 conn(0x7ff9cc0fd5d0 msgr2=0x7ff9cc0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 shutdown_connections 2024-09-05T12:43:57.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:43:57.178+0000 7ff9d1000640 1 -- 172.21.15.71:0/2798239465 wait complete. 2024-09-05T12:43:57.384 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 -- 172.21.15.71:0/567317455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68181047e0 msgr2=0x7f6818108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 --2- 172.21.15.71:0/567317455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68181047e0 0x7f6818108830 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6804009900 tx=0x7f680402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 -- 172.21.15.71:0/567317455 shutdown_connections 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 --2- 172.21.15.71:0/567317455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6818108f60 0x7f6818110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 --2- 172.21.15.71:0/567317455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68181047e0 0x7f6818108830 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 --2- 172.21.15.71:0/567317455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818103e30 0x7f6818104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 -- 172.21.15.71:0/567317455 >> 172.21.15.71:0/567317455 conn(0x7f68180ff8d0 msgr2=0x7f6818101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 -- 172.21.15.71:0/567317455 shutdown_connections 2024-09-05T12:43:57.385 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.382+0000 7f6817200640 1 -- 172.21.15.71:0/567317455 wait complete. 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.383+0000 7f6817200640 1 Processor -- start 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.383+0000 7f6817200640 1 -- start start 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6817200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6818103e30 0x7f681819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6817200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68181047e0 0x7f681819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6817200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 0x7f681819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6817200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6818114010 con 0x7f68181047e0 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6817200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6818113e90 con 0x7f6818103e30 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6817200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6818114190 con 0x7f6818108f60 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6815400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68181047e0 0x7f681819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.387 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6816800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 0x7f681819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6815e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6818103e30 0x7f681819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6815e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6818103e30 0x7f681819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49272/0 (socket says 172.21.15.71:49272) 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6816800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 0x7f681819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58656/0 (socket says 172.21.15.71:58656) 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6815e00640 1 -- 172.21.15.71:0/3948168739 learned_addr learned my addr 172.21.15.71:0/3948168739 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6816800640 1 -- 172.21.15.71:0/3948168739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6818103e30 msgr2=0x7f681819a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6816800640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6818103e30 0x7f681819a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6816800640 1 -- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68181047e0 msgr2=0x7f681819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.384+0000 7f6816800640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68181047e0 0x7f681819aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f6816800640 1 -- 172.21.15.71:0/3948168739 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f681819f4c0 con 0x7f6818108f60 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f6816800640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 0x7f681819ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f681000ec50 tx=0x7f681000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f680ea00640 1 -- 172.21.15.71:0/3948168739 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810003b70 con 0x7f6818108f60 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f681819f7b0 con 0x7f6818108f60 2024-09-05T12:43:57.388 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f680ea00640 1 -- 172.21.15.71:0/3948168739 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6810003d10 con 0x7f6818108f60 2024-09-05T12:43:57.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f680ea00640 1 -- 172.21.15.71:0/3948168739 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6810010890 con 0x7f6818108f60 2024-09-05T12:43:57.389 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.385+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68181a7090 con 0x7f6818108f60 2024-09-05T12:43:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.388+0000 7f681f67d740 1 Processor -- start 2024-09-05T12:43:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.388+0000 7f680ea00640 1 -- 172.21.15.71:0/3948168739 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6810003fc0 con 0x7f6818108f60 2024-09-05T12:43:57.390 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.388+0000 7f681f67d740 1 -- start start 2024-09-05T12:43:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f681f67d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f681f67d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7e40 0x55c31cef21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f681f67d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cef2700 0x55c31cef4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f681f67d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c31ce4d650 con 0x55c31cef2700 2024-09-05T12:43:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f681f67d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c31cdf4f10 con 0x55c31cee7e40 2024-09-05T12:43:57.391 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f681f67d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c31ce49610 con 0x55c31cee7520 2024-09-05T12:43:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f6815400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f6815400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:58668/0 (socket says 172.21.15.71:58668) 2024-09-05T12:43:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f6815400640 1 -- 172.21.15.71:0/2665388287 learned_addr learned my addr 172.21.15.71:0/2665388287 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f6815e00640 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7e40 0x55c31cef21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.389+0000 7f6815400640 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cef2700 0x55c31cef4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.392 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815e00640 1 -- 172.21.15.71:0/2665388287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 msgr2=0x55c31cee7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815e00640 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815e00640 1 -- 172.21.15.71:0/2665388287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cef2700 msgr2=0x55c31cef4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815e00640 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cef2700 0x55c31cef4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815e00640 1 -- 172.21.15.71:0/2665388287 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31cb5c580 con 0x55c31cee7e40 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815e00640 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7e40 0x55c31cef21c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6808009d20 tx=0x7f6808011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3c7202e1d3f14a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6803000640 1 -- 172.21.15.71:0/2665388287 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808005020 con 0x55c31cee7e40 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f680ea00640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67fc0a2cf0 0x7f67fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6803000640 1 -- 172.21.15.71:0/2665388287 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68080047a0 con 0x55c31cee7e40 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6803000640 1 -- 172.21.15.71:0/2665388287 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6808019670 con 0x55c31cee7e40 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f680ea00640 1 -- 172.21.15.71:0/3948168739 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f68100cfd80 con 0x7f6818108f60 2024-09-05T12:43:57.393 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.390+0000 7f6815400640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67fc0a2cf0 0x7f67fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 -- 172.21.15.71:0/2665388287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7e40 msgr2=0x55c31cef21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7e40 0x55c31cef21c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6808009d20 tx=0x7f6808011810 comp rx=0 tx=0).stop 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 -- 172.21.15.71:0/2665388287 shutdown_connections 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cef2700 0x55c31cef4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7e40 0x55c31cef21c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 --2- 172.21.15.71:0/2665388287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 -- 172.21.15.71:0/2665388287 >> 172.21.15.71:0/2665388287 conn(0x55c31cee3ab0 msgr2=0x55c31cee3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 -- 172.21.15.71:0/2665388287 shutdown_connections 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f6815400640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67fc0a2cf0 0x7f67fc0a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6804005c40 tx=0x7f680402a3f0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.391+0000 7f681f67d740 1 -- 172.21.15.71:0/2665388287 wait complete. 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 -- start start 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 Processor -- start 2024-09-05T12:43:57.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31ceef650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31ceefb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cef00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c31cef52d0 con 0x55c31cee7e40 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c31ce08350 con 0x55c31cef2700 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c31cef5450 con 0x55c31cee7520 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.392+0000 7f681f67d740 10 client.0 _ll_register_callbacks cb 0x55c31ce2b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cef00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cef00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49300/0 (socket says 172.21.15.71:49300) 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 -- 172.21.15.71:0/609493527 learned_addr learned my addr 172.21.15.71:0/609493527 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6815400640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31ceefb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6815e00640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31ceef650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 -- 172.21.15.71:0/609493527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 msgr2=0x55c31ceef650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31ceef650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 -- 172.21.15.71:0/609493527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 msgr2=0x55c31ceefb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31ceefb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 -- 172.21.15.71:0/609493527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31cc26960 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6815400640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31ceefb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f6816800640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cef00d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6810098310 tx=0x7f681009bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9afb4e8969a469 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f67f7400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.393+0000 7f67f7400640 1 -- 172.21.15.71:0/609493527 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681009beb0 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68080047a0 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f67f7400640 1 -- 172.21.15.71:0/609493527 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f681009c110 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f67f7400640 1 -- 172.21.15.71:0/609493527 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f6810082050 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 10 client.27212 fetch_fsmap learned FSMap version 68 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55c31cf01000 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f67f7400640 1 -- 172.21.15.71:0/609493527 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68080047a0 con 0x55c31cef2700 2024-09-05T12:43:57.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f67f7400640 1 -- 172.21.15.71:0/609493527 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 68) ==== 8410+0+0 (secure 0 0 0) 0x7f68100a4720 con 0x55c31cef2700 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 10 client.27212 fetch_fsmap finished waiting for FSMap version 68 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 10 client.27212 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 20 client.27212 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf02460 0x55c31cf04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 4 client.27212 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.394+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c31ce5abc0 con 0x55c31cf02460 2024-09-05T12:43:57.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.395+0000 7f6815e00640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf02460 0x55c31cf04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.395+0000 7f6815e00640 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf02460 0x55c31cf04900 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfbcd5742dd3751b server_cookie=9184f9a99c34484e in_seq=0 out_seq=0 2024-09-05T12:43:57.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.395+0000 7f67f7400640 10 client.27212 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f67f7400640 1 -- 172.21.15.71:0/609493527 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c31ce5abc0 con 0x55c31cf02460 2024-09-05T12:43:57.401 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f67f7400640 10 client.27212 handle_command_reply: tid=0 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 client.27212 shutdown 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 msgr2=0x55c31cef00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cef00d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6810098310 tx=0x7f681009bab0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 shutdown_connections 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf02460 0x55c31cf04900 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cef00d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31ceefb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 --2- 172.21.15.71:0/609493527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31ceef650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f67f7400640 0 client.27212 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 >> 172.21.15.71:0/609493527 conn(0x55c31cee3ab0 msgr2=0x55c31cee2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 shutdown_connections 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.396+0000 7f681f67d740 1 -- 172.21.15.71:0/609493527 wait complete. 2024-09-05T12:43:57.402 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.397+0000 7f681f67d740 20 client.27212 trim_cache size 0 max 16384 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 Processor -- start 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 -- start start 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cf0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cf0a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68181b0fe0 con 0x55c31cee7e40 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68181b2c70 con 0x55c31cef2700 2024-09-05T12:43:57.416 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.414+0000 7f681f67d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68181b1710 con 0x55c31cee7520 2024-09-05T12:43:57.417 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cf0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cf0a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53458/0 (socket says 172.21.15.71:53458) 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 -- 172.21.15.71:0/2290496553 learned_addr learned my addr 172.21.15.71:0/2290496553 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6816800640 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cf0a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815e00640 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 -- 172.21.15.71:0/2290496553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 msgr2=0x55c31cee9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 -- 172.21.15.71:0/2290496553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 msgr2=0x55c31cf0a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cf0a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 -- 172.21.15.71:0/2290496553 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c31ceea4c0 con 0x55c31cee7e40 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6816800640 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cf0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.415+0000 7f6815400640 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cf0a1d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6804006010 tx=0x7f680403f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c099a99ece3a795d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.418 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f6802600640 1 -- 172.21.15.71:0/2290496553 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804004980 con 0x55c31cee7e40 2024-09-05T12:43:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f6802600640 1 -- 172.21.15.71:0/2290496553 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6804051070 con 0x55c31cee7e40 2024-09-05T12:43:57.419 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f6802600640 1 -- 172.21.15.71:0/2290496553 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804032e60 con 0x55c31cee7e40 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 -- 172.21.15.71:0/2290496553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 msgr2=0x55c31cf0a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cf0a1d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6804006010 tx=0x7f680403f070 comp rx=0 tx=0).stop 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 -- 172.21.15.71:0/2290496553 shutdown_connections 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cef2700 0x55c31cf0a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cf0a1d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 --2- 172.21.15.71:0/2290496553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cee7520 0x55c31cee9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 -- 172.21.15.71:0/2290496553 >> 172.21.15.71:0/2290496553 conn(0x55c31cee3ab0 msgr2=0x55c31cef15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.416+0000 7f681f67d740 1 -- 172.21.15.71:0/2290496553 shutdown_connections 2024-09-05T12:43:57.421 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.417+0000 7f681f67d740 1 -- 172.21.15.71:0/2290496553 wait complete. 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.417+0000 7f681f67d740 1 -- start start 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.417+0000 7f681f67d740 1 Processor -- start 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.417+0000 7f681f67d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7520 0x55c31cef7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.417+0000 7f681f67d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cef7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f681f67d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cef2700 0x55c31cef80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f681f67d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f68181b25a0 con 0x55c31cee7e40 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f681f67d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68181b1e10 con 0x55c31cee7520 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f681f67d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68181aefe0 con 0x55c31cef2700 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f681f67d740 10 client.0 _ll_register_callbacks cb 0x7f6810003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cef7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cef7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53460/0 (socket says 172.21.15.71:53460) 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 -- 172.21.15.71:0/1394494351 learned_addr learned my addr 172.21.15.71:0/1394494351 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:43:57.422 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815e00640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7520 0x55c31cef7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6816800640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cef2700 0x55c31cef80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 -- 172.21.15.71:0/1394494351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cef2700 msgr2=0x55c31cef80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cef2700 0x55c31cef80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 -- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7520 msgr2=0x55c31cef7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7520 0x55c31cef7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.418+0000 7f6815400640 1 -- 172.21.15.71:0/1394494351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c31cf6d1b0 con 0x55c31cee7e40 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f6815400640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cef7bb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6804031550 tx=0x7f6804009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6e6f9eef0e67ab server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f67f7400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f67f7400640 1 -- 172.21.15.71:0/1394494351 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6804005170 con 0x55c31cee7e40 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f67f7400640 1 -- 172.21.15.71:0/1394494351 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f68040325c0 con 0x55c31cee7e40 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6804051070 con 0x55c31cee7e40 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f67f7400640 1 -- 172.21.15.71:0/1394494351 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7f680406b050 con 0x55c31cee7e40 2024-09-05T12:43:57.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f681f67d740 10 client.17235 fetch_fsmap learned FSMap version 68 2024-09-05T12:43:57.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.419+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x55c31cf6e3d0 con 0x55c31cee7e40 2024-09-05T12:43:57.424 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f67f7400640 1 -- 172.21.15.71:0/1394494351 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6804051070 con 0x55c31cee7e40 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f67f7400640 1 -- 172.21.15.71:0/1394494351 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 68) ==== 8410+0+0 (secure 0 0 0) 0x7f6804048070 con 0x55c31cee7e40 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f681f67d740 10 client.17235 fetch_fsmap finished waiting for FSMap version 68 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f681f67d740 10 client.17235 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f681f67d740 20 client.17235 populate_metadata read hostname 'smithi071' 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f681f67d740 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf6eb60 0x55c31cf70f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f681f67d740 4 client.17235 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ccf99444", "release": true, "await": true}] 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ccf99444", "release": true, "await": true}) -- 0x55c31cee3380 con 0x55c31cf6eb60 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.420+0000 7f6815e00640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf6eb60 0x55c31cf70f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.421+0000 7f6815e00640 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf6eb60 0x55c31cf70f80 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ef8da5e0be418f server_cookie=7fa59f0226400967 in_seq=0 out_seq=0 2024-09-05T12:43:57.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.421+0000 7f67f7400640 10 client.17235 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f67f7400640 1 -- 172.21.15.71:0/1394494351 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c31cee3380 con 0x55c31cf6eb60 2024-09-05T12:43:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f67f7400640 10 client.17235 handle_command_reply: tid=0 2024-09-05T12:43:57.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 client.17235 shutdown 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 msgr2=0x55c31cef7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cef7bb0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6804031550 tx=0x7f6804009640 comp rx=0 tx=0).stop 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 shutdown_connections 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55c31cf6eb60 0x55c31cf70f80 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c31cef2700 0x55c31cef80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c31cee7e40 0x55c31cef7bb0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 --2- 172.21.15.71:0/1394494351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c31cee7520 0x55c31cef7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 >> 172.21.15.71:0/1394494351 conn(0x55c31cee3ab0 msgr2=0x55c31ceef610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.443+0000 7f67f7400640 0 client.17235 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.444+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 shutdown_connections 2024-09-05T12:43:57.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.444+0000 7f681f67d740 1 -- 172.21.15.71:0/1394494351 wait complete. 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.444+0000 7f681f67d740 20 client.17235 trim_cache size 0 max 16384 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 68, 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 52, 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:43:57.447 INFO:teuthology.orchestra.run.smithi071.stdout: "ccf99444": { 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 52, 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 140.0, 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:43:57.448 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:43:57.449 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67fc0a2cf0 msgr2=0x7f67fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67fc0a2cf0 0x7f67fc0a51b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6804005c40 tx=0x7f680402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 msgr2=0x7f681819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 0x7f681819ede0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f681000ec50 tx=0x7f681000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 shutdown_connections 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6818108f60 0x7f681819ede0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f67fc0a2cf0 0x7f67fc0a51b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f68181047e0 0x7f681819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 --2- 172.21.15.71:0/3948168739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6818103e30 0x7f681819a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:43:57.450 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.447+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 >> 172.21.15.71:0/3948168739 conn(0x7f68180ff8d0 msgr2=0x7f6818100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:43:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.448+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 shutdown_connections 2024-09-05T12:43:57.451 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:43:57.448+0000 7f6817200640 1 -- 172.21.15.71:0/3948168739 wait complete. 2024-09-05T12:43:57.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 55613 15.11 MB/sec execute 51 sec latency 29736.379 ms 2024-09-05T12:43:58.498 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ccf99444', total seconds elapsed: 29.2 2024-09-05T12:43:58.498 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.0 2024-09-05T12:43:58.498 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:43:58.790 INFO:tasks.workunit.client.0.smithi071.stdout: 10 56188 15.55 MB/sec execute 52 sec latency 445.929 ms 2024-09-05T12:43:58.868 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:58 smithi071 ceph-mon[29845]: pgmap v510: 129 pgs: 129 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 817 KiB/s wr, 150 op/s 2024-09-05T12:43:58.868 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:43:58 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:58 smithi123 ceph-mon[34321]: pgmap v510: 129 pgs: 129 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 817 KiB/s wr, 150 op/s 2024-09-05T12:43:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:43:58 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:58 smithi174 ceph-mon[34313]: pgmap v510: 129 pgs: 129 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 817 KiB/s wr, 150 op/s 2024-09-05T12:43:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:43:58 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:43:59.066 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:43:59.791 INFO:tasks.workunit.client.0.smithi071.stdout: 10 57145 16.06 MB/sec execute 53 sec latency 358.508 ms 2024-09-05T12:44:00.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cbd600640 1 -- 172.21.15.71:0/2448694715 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00319b0 con 0x7f2cc0100930 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2448694715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc0100930 msgr2=0x7f2cc0100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2448694715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc0100930 0x7f2cc0100d10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0009900 tx=0x7f2cb002a420 comp rx=0 tx=0).stop 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2448694715 shutdown_connections 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2448694715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc010e4e0 0x7f2cc01108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2448694715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc01012e0 0x7f2cc010dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2448694715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc0100930 0x7f2cc0100d10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2448694715 >> 172.21.15.71:0/2448694715 conn(0x7f2cc00fc7a0 msgr2=0x7f2cc00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:00.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2448694715 shutdown_connections 2024-09-05T12:44:00.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.606+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2448694715 wait complete. 2024-09-05T12:44:00.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.609+0000 7f2cc5e00640 1 Processor -- start 2024-09-05T12:44:00.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.611+0000 7f2cc5e00640 1 -- start start 2024-09-05T12:44:00.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.613+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 0x7f2cc00736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:00.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.613+0000 7f2cc5e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc01012e0 0x7f2cc0073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.613+0000 7f2cbea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 0x7f2cc00736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.613+0000 7f2cbea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 0x7f2cc00736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49350/0 (socket says 172.21.15.71:49350) 2024-09-05T12:44:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.613+0000 7f2cbea00640 1 -- 172.21.15.71:0/2281548428 learned_addr learned my addr 172.21.15.71:0/2281548428 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.614+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc010e4e0 0x7f2cc006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.614+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc006c0e0 con 0x7f2cc01012e0 2024-09-05T12:44:00.616 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.614+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc006bf60 con 0x7f2cc0100930 2024-09-05T12:44:00.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.614+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc006c260 con 0x7f2cc010e4e0 2024-09-05T12:44:00.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.614+0000 7f2cbf400640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc010e4e0 0x7f2cc006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:00.617 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.614+0000 7f2cbe000640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc01012e0 0x7f2cc0073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.616+0000 7f2cbea00640 1 -- 172.21.15.71:0/2281548428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc010e4e0 msgr2=0x7f2cc006d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.616+0000 7f2cbea00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc010e4e0 0x7f2cc006d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.616+0000 7f2cbea00640 1 -- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc01012e0 msgr2=0x7f2cc0073c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.616+0000 7f2cbea00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc01012e0 0x7f2cc0073c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.616+0000 7f2cbea00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc006dff0 con 0x7f2cc0100930 2024-09-05T12:44:00.618 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.616+0000 7f2cbf400640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc010e4e0 0x7f2cc006d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.617+0000 7f2cbea00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 0x7f2cc00736d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0004770 tx=0x7f2cb0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.617+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb003f070 con 0x7f2cc0100930 2024-09-05T12:44:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.617+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc006e280 con 0x7f2cc0100930 2024-09-05T12:44:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.617+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01afad0 con 0x7f2cc0100930 2024-09-05T12:44:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.617+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cb0031ca0 con 0x7f2cc0100930 2024-09-05T12:44:00.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.617+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0035030 con 0x7f2cc0100930 2024-09-05T12:44:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.624+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc0100d10 con 0x7f2cc0100930 2024-09-05T12:44:00.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.625+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2cb0042070 con 0x7f2cc0100930 2024-09-05T12:44:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.627+0000 7f2cc4a00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2ca40a2cf0 0x7f2ca40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.627+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9438+0+0 (secure 0 0 0) 0x7f2cb00efbe0 con 0x7f2cc0100930 2024-09-05T12:44:00.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.627+0000 7f2cbe000640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2ca40a2cf0 0x7f2ca40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:00.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.630+0000 7f2cbe000640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2ca40a2cf0 0x7f2ca40a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cac006fd0 tx=0x7f2cac008040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:00.637 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:00.635+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2cb00bc410 con 0x7f2cc0100930 2024-09-05T12:44:00.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 57952 16.82 MB/sec execute 54 sec latency 56.043 ms 2024-09-05T12:44:00.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:00 smithi071 ceph-mon[29845]: pgmap v511: 129 pgs: 129 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 355 KiB/s wr, 47 op/s 2024-09-05T12:44:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:00 smithi123 ceph-mon[34321]: pgmap v511: 129 pgs: 129 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 355 KiB/s wr, 47 op/s 2024-09-05T12:44:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:00 smithi174 ceph-mon[34313]: pgmap v511: 129 pgs: 129 active+clean; 615 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 355 KiB/s wr, 47 op/s 2024-09-05T12:44:01.024 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.022+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cc0109670 con 0x7f2cc0100930 2024-09-05T12:44:01.026 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:44:01.026 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.024+0000 7f2cc4a00640 1 -- 172.21.15.71:0/2281548428 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11984 (secure 0 0 0) 0x7f2cb00bbd10 con 0x7f2cc0100930 2024-09-05T12:44:01.027 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 69 2024-09-05T12:44:01.027 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":69,"btime":"2024-09-05T12:43:57:653357+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:43:56.841307+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2ca40a2cf0 msgr2=0x7f2ca40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2ca40a2cf0 0x7f2ca40a51b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2cac006fd0 tx=0x7f2cac008040 comp rx=0 tx=0).stop 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 msgr2=0x7f2cc00736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 0x7f2cc00736d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0004770 tx=0x7f2cb0003c10 comp rx=0 tx=0).stop 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 shutdown_connections 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cc010e4e0 0x7f2cc006d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:01.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f2ca40a2cf0 0x7f2ca40a51b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:01.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cc01012e0 0x7f2cc0073c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:01.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 --2- 172.21.15.71:0/2281548428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2cc0100930 0x7f2cc00736d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:01.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.028+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 >> 172.21.15.71:0/2281548428 conn(0x7f2cc00fc7a0 msgr2=0x7f2cc00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:01.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.029+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 shutdown_connections 2024-09-05T12:44:01.031 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:01.029+0000 7f2cc5e00640 1 -- 172.21.15.71:0/2281548428 wait complete. 2024-09-05T12:44:01.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58280 16.67 MB/sec execute 55 sec latency 377.478 ms 2024-09-05T12:44:01.887 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:01 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2281548428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:44:01.954 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0' 2024-09-05T12:44:01.954 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0 2024-09-05T12:44:02.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:01 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2281548428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:44:02.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:01 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2281548428' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:44:02.590 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:44:02.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58331 16.42 MB/sec execute 56 sec latency 424.476 ms 2024-09-05T12:44:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:02 smithi174 ceph-mon[34313]: pgmap v512: 129 pgs: 129 active+clean; 625 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 210 op/s 2024-09-05T12:44:03.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:02 smithi071 ceph-mon[29845]: pgmap v512: 129 pgs: 129 active+clean; 625 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 210 op/s 2024-09-05T12:44:03.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:02 smithi123 ceph-mon[34321]: pgmap v512: 129 pgs: 129 active+clean; 625 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.1 MiB/s wr, 210 op/s 2024-09-05T12:44:03.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58337 16.15 MB/sec execute 57 sec latency 931.448 ms 2024-09-05T12:44:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.807+0000 7fae91a00640 1 -- 172.21.15.71:0/2613138624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c112b90 msgr2=0x7fae8c114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:03.809 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.807+0000 7fae91a00640 1 --2- 172.21.15.71:0/2613138624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c112b90 0x7fae8c114f80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fae78009900 tx=0x7fae7802a4b0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 -- 172.21.15.71:0/2613138624 shutdown_connections 2024-09-05T12:44:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 --2- 172.21.15.71:0/2613138624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c112b90 0x7fae8c114f80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 --2- 172.21.15.71:0/2613138624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c074eb0 0x7fae8c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 --2- 172.21.15.71:0/2613138624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae8c074590 0x7fae8c074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 -- 172.21.15.71:0/2613138624 >> 172.21.15.71:0/2613138624 conn(0x7fae8c0fd8c0 msgr2=0x7fae8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:03.810 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 -- 172.21.15.71:0/2613138624 shutdown_connections 2024-09-05T12:44:03.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.808+0000 7fae91a00640 1 -- 172.21.15.71:0/2613138624 wait complete. 2024-09-05T12:44:03.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.809+0000 7fae91a00640 1 Processor -- start 2024-09-05T12:44:03.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.810+0000 7fae91a00640 1 -- start start 2024-09-05T12:44:03.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.812+0000 7fae91a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae8c074590 0x7fae8c112270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.812+0000 7fae91a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 0x7fae8c10b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.812+0000 7fae91a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c112b90 0x7fae8c10b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.812+0000 7fae8a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 0x7fae8c10b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae8a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 0x7fae8c10b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55716/0 (socket says 172.21.15.71:55716) 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae8a000640 1 -- 172.21.15.71:0/2897666857 learned_addr learned my addr 172.21.15.71:0/2897666857 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c118630 con 0x7fae8c074590 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1184b0 con 0x7fae8c074eb0 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1187b0 con 0x7fae8c112b90 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae8b400640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c112b90 0x7fae8c10b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.813+0000 7fae8aa00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae8c074590 0x7fae8c112270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.814+0000 7fae8a000640 1 -- 172.21.15.71:0/2897666857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c112b90 msgr2=0x7fae8c10b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:03.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.814+0000 7fae8a000640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c112b90 0x7fae8c10b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.814+0000 7fae8a000640 1 -- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae8c074590 msgr2=0x7fae8c112270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:03.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.814+0000 7fae8a000640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae8c074590 0x7fae8c112270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.814+0000 7fae8a000640 1 -- 172.21.15.71:0/2897666857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c10bf70 con 0x7fae8c074eb0 2024-09-05T12:44:03.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.815+0000 7fae8b400640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c112b90 0x7fae8c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:44:03.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.815+0000 7fae8a000640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 0x7fae8c10b300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b4a0 tx=0x7fae7c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:03.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.815+0000 7fae83400640 1 -- 172.21.15.71:0/2897666857 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c014070 con 0x7fae8c074eb0 2024-09-05T12:44:03.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.815+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c10fda0 con 0x7fae8c074eb0 2024-09-05T12:44:03.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.815+0000 7fae83400640 1 -- 172.21.15.71:0/2897666857 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae7c010af0 con 0x7fae8c074eb0 2024-09-05T12:44:03.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.815+0000 7fae83400640 1 -- 172.21.15.71:0/2897666857 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c014070 con 0x7fae8c074eb0 2024-09-05T12:44:03.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.816+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1102e0 con 0x7fae8c074eb0 2024-09-05T12:44:03.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.820+0000 7fae83400640 1 -- 172.21.15.71:0/2897666857 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fae7c017070 con 0x7fae8c074eb0 2024-09-05T12:44:03.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.822+0000 7fae944c6740 1 Processor -- start 2024-09-05T12:44:03.824 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.822+0000 7fae944c6740 1 -- start start 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae83400640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fae700a3030 0x7fae700a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae944c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc5160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae944c6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc5160cd0 0x55abc516b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae944c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc516d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae944c6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55abc50c6530 con 0x55abc5160cd0 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae944c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abc506d860 con 0x55abc51603b0 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae944c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abc50c24f0 con 0x55abc516b590 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.823+0000 7fae83400640 1 -- 172.21.15.71:0/2897666857 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9526+0+0 (secure 0 0 0) 0x7fae7c0ce720 con 0x7fae8c074eb0 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc5160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc5160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55720/0 (socket says 172.21.15.71:55720) 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 -- 172.21.15.71:0/184165098 learned_addr learned my addr 172.21.15.71:0/184165098 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:03.826 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8b400640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc516d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8a000640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc5160cd0 0x55abc516b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 -- 172.21.15.71:0/184165098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 msgr2=0x55abc516d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc516d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 -- 172.21.15.71:0/184165098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc5160cd0 msgr2=0x55abc516b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc5160cd0 0x55abc516b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8aa00640 1 -- 172.21.15.71:0/184165098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc4e37b90 con 0x55abc51603b0 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8b400640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc516d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8a000640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc5160cd0 0x55abc516b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.824+0000 7fae8b400640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fae700a3030 0x7fae700a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.825+0000 7fae8aa00640 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc5160790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae8400a4b0 tx=0x7fae840096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8983903612d5aa35 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.825+0000 7fae77e00640 1 -- 172.21.15.71:0/184165098 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84018020 con 0x55abc51603b0 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.825+0000 7fae77e00640 1 -- 172.21.15.71:0/184165098 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae84014cf0 con 0x55abc51603b0 2024-09-05T12:44:03.828 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.825+0000 7fae77e00640 1 -- 172.21.15.71:0/184165098 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae84013de0 con 0x55abc51603b0 2024-09-05T12:44:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 -- 172.21.15.71:0/184165098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 msgr2=0x55abc5160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc5160790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae8400a4b0 tx=0x7fae840096d0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae8b400640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fae700a3030 0x7fae700a54f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae7802aaa0 tx=0x7fae78005fd0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 -- 172.21.15.71:0/184165098 shutdown_connections 2024-09-05T12:44:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc516d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc5160cd0 0x55abc516b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 --2- 172.21.15.71:0/184165098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc5160790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.826+0000 7fae944c6740 1 -- 172.21.15.71:0/184165098 >> 172.21.15.71:0/184165098 conn(0x55abc515c940 msgr2=0x55abc515cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.827+0000 7fae944c6740 1 -- 172.21.15.71:0/184165098 shutdown_connections 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.827+0000 7fae944c6740 1 -- 172.21.15.71:0/184165098 wait complete. 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.827+0000 7fae944c6740 1 -- start start 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.827+0000 7fae944c6740 1 Processor -- start 2024-09-05T12:44:03.830 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc51685a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 0x55abc5168ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc5169020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55abc516dfb0 con 0x55abc51603b0 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abc5080e60 con 0x55abc5160cd0 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abc516e130 con 0x55abc516b590 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae944c6740 10 client.0 _ll_register_callbacks cb 0x55abc5085cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.828+0000 7fae8a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 0x55abc5168ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 0x55abc5168ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55734/0 (socket says 172.21.15.71:55734) 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 -- 172.21.15.71:0/3126230570 learned_addr learned my addr 172.21.15.71:0/3126230570 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:03.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8b400640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc5169020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8aa00640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc51685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 -- 172.21.15.71:0/3126230570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 msgr2=0x55abc5169020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.833 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc5169020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 -- 172.21.15.71:0/3126230570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 msgr2=0x55abc51685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc51685a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.834 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.829+0000 7fae8a000640 1 -- 172.21.15.71:0/3126230570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc508c990 con 0x55abc5160cd0 2024-09-05T12:44:03.834 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:03 smithi071 ceph-mon[29845]: osdmap e117: 12 total, 12 up, 12 in 2024-09-05T12:44:03.834 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:03 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.835+0000 7fae8a000640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 0x55abc5168ae0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b470 tx=0x7fae7c09b0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2631f6c79bba16a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:03.837 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.835+0000 7fae74c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:44:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.836+0000 7fae74c00640 1 -- 172.21.15.71:0/3126230570 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0cd030 con 0x55abc5160cd0 2024-09-05T12:44:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.836+0000 7fae74c00640 1 -- 172.21.15.71:0/3126230570 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9526+0+0 (secure 0 0 0) 0x7fae7c096f70 con 0x55abc5160cd0 2024-09-05T12:44:03.838 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.837+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae84014cf0 con 0x55abc5160cd0 2024-09-05T12:44:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.837+0000 7fae74c00640 1 -- 172.21.15.71:0/3126230570 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fae7c0a3570 con 0x55abc5160cd0 2024-09-05T12:44:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.837+0000 7fae944c6740 10 client.27236 fetch_fsmap learned FSMap version 70 2024-09-05T12:44:03.839 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.837+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55abc5179cf0 con 0x55abc5160cd0 2024-09-05T12:44:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae74c00640 1 -- 172.21.15.71:0/3126230570 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae84014cf0 con 0x55abc5160cd0 2024-09-05T12:44:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae74c00640 1 -- 172.21.15.71:0/3126230570 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 70) ==== 8402+0+0 (secure 0 0 0) 0x7fae7c0b90c0 con 0x55abc5160cd0 2024-09-05T12:44:03.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae944c6740 10 client.27236 fetch_fsmap finished waiting for FSMap version 70 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae944c6740 10 client.27236 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae944c6740 20 client.27236 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae944c6740 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc517b670 0x55abc517db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae944c6740 4 client.27236 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.838+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc5069310 con 0x55abc517b670 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.839+0000 7fae8aa00640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc517b670 0x55abc517db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.840+0000 7fae74c00640 10 client.27236 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:03.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.841+0000 7fae8aa00640 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc517b670 0x55abc517db30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319938debe6543a3 server_cookie=910d409335e243d2 in_seq=0 out_seq=0 2024-09-05T12:44:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.842+0000 7fae74c00640 1 -- 172.21.15.71:0/3126230570 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abc5069310 con 0x55abc517b670 2024-09-05T12:44:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.842+0000 7fae74c00640 10 client.27236 handle_command_reply: tid=0 2024-09-05T12:44:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.842+0000 7fae944c6740 1 client.27236 shutdown 2024-09-05T12:44:03.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.842+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 msgr2=0x55abc5168ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.842+0000 7fae944c6740 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 0x55abc5168ae0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b470 tx=0x7fae7c09b0c0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 shutdown_connections 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc517b670 0x55abc517db30 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc516b590 0x55abc5169020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc5160cd0 0x55abc5168ae0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae74c00640 0 client.27236 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 --2- 172.21.15.71:0/3126230570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc51685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 >> 172.21.15.71:0/3126230570 conn(0x55abc515c940 msgr2=0x55abc515bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 shutdown_connections 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 1 -- 172.21.15.71:0/3126230570 wait complete. 2024-09-05T12:44:03.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.843+0000 7fae944c6740 20 client.27236 trim_cache size 0 max 16384 2024-09-05T12:44:03.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.876+0000 7fae944c6740 1 Processor -- start 2024-09-05T12:44:03.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.876+0000 7fae944c6740 1 -- start start 2024-09-05T12:44:03.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.876+0000 7fae944c6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc5183440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae944c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc5183980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae944c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 0x55abc5165800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae944c6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1be0a0 con 0x55abc51603b0 2024-09-05T12:44:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae944c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1bfd30 con 0x55abc516b590 2024-09-05T12:44:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae944c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1be7d0 con 0x55abc5160cd0 2024-09-05T12:44:03.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae8b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 0x55abc5165800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae8b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 0x55abc5165800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55736/0 (socket says 172.21.15.71:55736) 2024-09-05T12:44:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.877+0000 7fae8b400640 1 -- 172.21.15.71:0/1694694957 learned_addr learned my addr 172.21.15.71:0/1694694957 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8a000640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc5183980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8b400640 1 -- 172.21.15.71:0/1694694957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 msgr2=0x55abc5183980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8aa00640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc5183440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8b400640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc5183980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8b400640 1 -- 172.21.15.71:0/1694694957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 msgr2=0x55abc5183440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8b400640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc5183440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8b400640 1 -- 172.21.15.71:0/1694694957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc5161930 con 0x55abc516b590 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8a000640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc5183980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:03.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.878+0000 7fae8aa00640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc5183440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.879+0000 7fae8b400640 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 0x55abc5165800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae780315c0 tx=0x7fae78005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4442967e4e434709 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.879+0000 7fae77400640 1 -- 172.21.15.71:0/1694694957 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7804d070 con 0x55abc516b590 2024-09-05T12:44:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.880+0000 7fae77400640 1 -- 172.21.15.71:0/1694694957 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78043040 con 0x55abc516b590 2024-09-05T12:44:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.880+0000 7fae77400640 1 -- 172.21.15.71:0/1694694957 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae78042040 con 0x55abc516b590 2024-09-05T12:44:03.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.881+0000 7fae944c6740 1 -- 172.21.15.71:0/1694694957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 msgr2=0x55abc5165800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.881+0000 7fae944c6740 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 0x55abc5165800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fae780315c0 tx=0x7fae78005850 comp rx=0 tx=0).stop 2024-09-05T12:44:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.881+0000 7fae944c6740 1 -- 172.21.15.71:0/1694694957 shutdown_connections 2024-09-05T12:44:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.881+0000 7fae944c6740 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc516b590 0x55abc5165800 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.881+0000 7fae944c6740 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc5183980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.881+0000 7fae944c6740 1 --2- 172.21.15.71:0/1694694957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc51603b0 0x55abc5183440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.882+0000 7fae944c6740 1 -- 172.21.15.71:0/1694694957 >> 172.21.15.71:0/1694694957 conn(0x55abc515c940 msgr2=0x55abc515bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.882+0000 7fae944c6740 1 -- 172.21.15.71:0/1694694957 shutdown_connections 2024-09-05T12:44:03.884 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.882+0000 7fae944c6740 1 -- 172.21.15.71:0/1694694957 wait complete. 2024-09-05T12:44:03.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.883+0000 7fae944c6740 1 -- start start 2024-09-05T12:44:03.886 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.884+0000 7fae944c6740 1 Processor -- start 2024-09-05T12:44:03.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.885+0000 7fae944c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc516fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.885+0000 7fae944c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc516ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.887 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.885+0000 7fae944c6740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc516b590 0x55abc51df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae944c6740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1bf660 con 0x55abc516b590 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae944c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1beed0 con 0x55abc51603b0 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae944c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1bc0a0 con 0x55abc5160cd0 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae8a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc516ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae8a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc516ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42904/0 (socket says 172.21.15.71:42904) 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae8a000640 1 -- 172.21.15.71:0/3162494056 learned_addr learned my addr 172.21.15.71:0/3162494056 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae944c6740 10 client.0 _ll_register_callbacks cb 0x7fae7c096ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:03.888 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.886+0000 7fae8aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc516fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.887+0000 7fae8b400640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc516b590 0x55abc51df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.887+0000 7fae8a000640 1 -- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 msgr2=0x55abc516fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.887+0000 7fae8a000640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc516fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.887+0000 7fae8a000640 1 -- 172.21.15.71:0/3162494056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc516b590 msgr2=0x55abc51df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:03.889 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.887+0000 7fae8a000640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc516b590 0x55abc51df3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:03.890 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.888+0000 7fae8a000640 1 -- 172.21.15.71:0/3162494056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc51dfa20 con 0x55abc5160cd0 2024-09-05T12:44:03.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.888+0000 7fae8b400640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc516b590 0x55abc51df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:03.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.888+0000 7fae8aa00640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc516fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:44:03.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.888+0000 7fae8a000640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc516ffd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0972d0 tx=0x7fae7c09a7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e23f83396685d3af server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:03.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.888+0000 7fae74c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:44:03.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.889+0000 7fae74c00640 1 -- 172.21.15.71:0/3162494056 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0cd070 con 0x55abc5160cd0 2024-09-05T12:44:03.891 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.889+0000 7fae74c00640 1 -- 172.21.15.71:0/3162494056 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9526+0+0 (secure 0 0 0) 0x7fae7c0850b0 con 0x55abc5160cd0 2024-09-05T12:44:03.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.889+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae78043040 con 0x55abc5160cd0 2024-09-05T12:44:03.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.889+0000 7fae74c00640 1 -- 172.21.15.71:0/3162494056 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fae7c0852b0 con 0x55abc5160cd0 2024-09-05T12:44:03.892 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.891+0000 7fae944c6740 10 client.27286 fetch_fsmap learned FSMap version 70 2024-09-05T12:44:03.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.891+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x55abc51e6d00 con 0x55abc5160cd0 2024-09-05T12:44:03.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.891+0000 7fae74c00640 1 -- 172.21.15.71:0/3162494056 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae78043040 con 0x55abc5160cd0 2024-09-05T12:44:03.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.891+0000 7fae74c00640 1 -- 172.21.15.71:0/3162494056 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 70) ==== 8402+0+0 (secure 0 0 0) 0x7fae7c0b3b00 con 0x55abc5160cd0 2024-09-05T12:44:03.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.892+0000 7fae944c6740 10 client.27286 fetch_fsmap finished waiting for FSMap version 70 2024-09-05T12:44:03.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.892+0000 7fae944c6740 10 client.27286 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:44:03.893 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.892+0000 7fae944c6740 20 client.27286 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:03.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.892+0000 7fae944c6740 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc51e7480 0x55abc51e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:03.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.892+0000 7fae944c6740 4 client.27286 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}] 2024-09-05T12:44:03.894 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.892+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}) -- 0x55abc50d3c60 con 0x55abc51e7480 2024-09-05T12:44:03.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.893+0000 7fae8aa00640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc51e7480 0x55abc51e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:03.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.893+0000 7fae8aa00640 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc51e7480 0x55abc51e98a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4c3abb4e17007ee server_cookie=24d072d54f04d9a1 in_seq=0 out_seq=0 2024-09-05T12:44:03.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:03.893+0000 7fae74c00640 10 client.27286 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:03 smithi174 ceph-mon[34313]: osdmap e117: 12 total, 12 up, 12 in 2024-09-05T12:44:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:03 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:04.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:03 smithi123 ceph-mon[34321]: osdmap e117: 12 total, 12 up, 12 in 2024-09-05T12:44:04.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:03 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae74c00640 1 -- 172.21.15.71:0/3162494056 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55abc50d3c60 con 0x55abc51e7480 2024-09-05T12:44:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae74c00640 10 client.27286 handle_command_reply: tid=0 2024-09-05T12:44:04.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae944c6740 1 client.27286 shutdown 2024-09-05T12:44:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 msgr2=0x55abc516ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae944c6740 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc516ffd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0972d0 tx=0x7fae7c09a7d0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 shutdown_connections 2024-09-05T12:44:04.628 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.625+0000 7fae944c6740 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55abc51e7480 0x55abc51e98a0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abc516b590 0x55abc51df3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc5160cd0 0x55abc516ffd0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 1 --2- 172.21.15.71:0/3162494056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc51603b0 0x55abc516fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 >> 172.21.15.71:0/3162494056 conn(0x55abc515c940 msgr2=0x55abc5161e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae74c00640 0 client.27286 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 shutdown_connections 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 1 -- 172.21.15.71:0/3162494056 wait complete. 2024-09-05T12:44:04.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.626+0000 7fae944c6740 20 client.27286 trim_cache size 0 max 16384 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 70, 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 54, 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "a1d3f86b": { 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 54, 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:44:04.630 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 140.0, 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:44:04.631 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:04.632 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:04.632 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:44:04.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.627+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fae700a3030 msgr2=0x7fae700a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.627+0000 7fae91a00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fae700a3030 0x7fae700a54f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae7802aaa0 tx=0x7fae78005fd0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 msgr2=0x7fae8c10b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 0x7fae8c10b300 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b4a0 tx=0x7fae7c00b960 comp rx=0 tx=0).stop 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 shutdown_connections 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c112b90 0x7fae8c10b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c074eb0 0x7fae8c10b300 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fae700a3030 0x7fae700a54f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 --2- 172.21.15.71:0/2897666857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae8c074590 0x7fae8c112270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 >> 172.21.15.71:0/2897666857 conn(0x7fae8c0fd8c0 msgr2=0x7fae8c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 shutdown_connections 2024-09-05T12:44:04.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:04.628+0000 7fae91a00640 1 -- 172.21.15.71:0/2897666857 wait complete. 2024-09-05T12:44:04.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 15.89 MB/sec execute 58 sec latency 1517.647 ms 2024-09-05T12:44:04.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:04 smithi071 ceph-mon[29845]: pgmap v514: 129 pgs: 129 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-09-05T12:44:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:04 smithi174 ceph-mon[34313]: pgmap v514: 129 pgs: 129 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-09-05T12:44:05.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:04 smithi123 ceph-mon[34321]: pgmap v514: 129 pgs: 129 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-09-05T12:44:05.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 15.62 MB/sec execute 59 sec latency 1487.910 ms 2024-09-05T12:44:05.940 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a1d3f86b', quiesce took 7.4 seconds. Will release after: 12.6 2024-09-05T12:44:06.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 15.36 MB/sec execute 60 sec latency 2488.038 ms 2024-09-05T12:44:07.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:06 smithi174 ceph-mon[34313]: pgmap v515: 129 pgs: 129 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-09-05T12:44:07.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:06 smithi071 ceph-mon[29845]: pgmap v515: 129 pgs: 129 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-09-05T12:44:07.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:06 smithi123 ceph-mon[34321]: pgmap v515: 129 pgs: 129 active+clean; 628 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.1 MiB/s wr, 321 op/s 2024-09-05T12:44:07.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 15.11 MB/sec execute 61 sec latency 3488.147 ms 2024-09-05T12:44:08.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 14.86 MB/sec execute 62 sec latency 4488.256 ms 2024-09-05T12:44:08.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:08 smithi071 ceph-mon[29845]: pgmap v516: 129 pgs: 129 active+clean; 633 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-05T12:44:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:08 smithi174 ceph-mon[34313]: pgmap v516: 129 pgs: 129 active+clean; 633 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-05T12:44:09.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:08 smithi123 ceph-mon[34321]: pgmap v516: 129 pgs: 129 active+clean; 633 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-05T12:44:09.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 14.63 MB/sec execute 63 sec latency 5488.406 ms 2024-09-05T12:44:10.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:09 smithi071 ceph-mon[29845]: pgmap v517: 129 pgs: 129 active+clean; 633 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-05T12:44:10.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:09 smithi123 ceph-mon[34321]: pgmap v517: 129 pgs: 129 active+clean; 633 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-05T12:44:10.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:09 smithi174 ceph-mon[34313]: pgmap v517: 129 pgs: 129 active+clean; 633 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 489 op/s 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 -- 172.21.15.71:0/402455638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81018b0 msgr2=0x7f55d8101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 --2- 172.21.15.71:0/402455638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81018b0 0x7f55d8101d10 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009930 tx=0x7f55c402a420 comp rx=0 tx=0).stop 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 -- 172.21.15.71:0/402455638 shutdown_connections 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 --2- 172.21.15.71:0/402455638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d8102250 0x7f55d810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 --2- 172.21.15.71:0/402455638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81018b0 0x7f55d8101d10 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 --2- 172.21.15.71:0/402455638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d81078c0 0x7f55d8107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 -- 172.21.15.71:0/402455638 >> 172.21.15.71:0/402455638 conn(0x7f55d80fd5d0 msgr2=0x7f55d80ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 -- 172.21.15.71:0/402455638 shutdown_connections 2024-09-05T12:44:10.304 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.303+0000 7f55de200640 1 -- 172.21.15.71:0/402455638 wait complete. 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 Processor -- start 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 -- start start 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d81018b0 0x7f55d819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d8102250 0x7f55d819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81078c0 0x7f55d819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8111d00 con 0x7f55d81078c0 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8111b80 con 0x7f55d8102250 2024-09-05T12:44:10.307 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.304+0000 7f55de200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8111e80 con 0x7f55d81018b0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81078c0 0x7f55d819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dce00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d81018b0 0x7f55d819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d8102250 0x7f55d819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81078c0 0x7f55d819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43492/0 (socket says 172.21.15.71:43492) 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dd800640 1 -- 172.21.15.71:0/237382110 learned_addr learned my addr 172.21.15.71:0/237382110 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dce00640 1 -- 172.21.15.71:0/237382110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d8102250 msgr2=0x7f55d819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dce00640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d8102250 0x7f55d819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dce00640 1 -- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81078c0 msgr2=0x7f55d819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dce00640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81078c0 0x7f55d819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.305+0000 7f55dce00640 1 -- 172.21.15.71:0/237382110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d819f480 con 0x7f55d81018b0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.306+0000 7f55dce00640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d81018b0 0x7f55d819a490 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e9f0 tx=0x7f55cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.306+0000 7f55d5600640 1 -- 172.21.15.71:0/237382110 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc00ccc0 con 0x7f55d81018b0 2024-09-05T12:44:10.308 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.306+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d819f770 con 0x7f55d81018b0 2024-09-05T12:44:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.306+0000 7f55d5600640 1 -- 172.21.15.71:0/237382110 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55cc005020 con 0x7f55d81018b0 2024-09-05T12:44:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.306+0000 7f55d5600640 1 -- 172.21.15.71:0/237382110 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0049e0 con 0x7f55d81018b0 2024-09-05T12:44:10.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.306+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a7000 con 0x7f55d81018b0 2024-09-05T12:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.310+0000 7f55e26a1740 1 Processor -- start 2024-09-05T12:44:10.311 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.310+0000 7f55d5600640 1 -- 172.21.15.71:0/237382110 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f55cc005560 con 0x7f55d81018b0 2024-09-05T12:44:10.312 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 -- start start 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1debe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5641aa126100 con 0x5641aa1dc7f0 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641aa0ce1b0 con 0x5641aa1c9380 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.311+0000 7f55e26a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5641aa1220c0 con 0x5641aa1c8a60 2024-09-05T12:44:10.313 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.312+0000 7f55dd800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.312+0000 7f55dd800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42936/0 (socket says 172.21.15.71:42936) 2024-09-05T12:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.312+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.314 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.312+0000 7f55dd800640 1 -- 172.21.15.71:0/3204439342 learned_addr learned my addr 172.21.15.71:0/3204439342 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.313+0000 7f55dd800640 1 -- 172.21.15.71:0/3204439342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 msgr2=0x5641aa1c4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.313+0000 7f55dd800640 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.313+0000 7f55dd800640 1 -- 172.21.15.71:0/3204439342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 msgr2=0x5641aa1debe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.313+0000 7f55dd800640 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1debe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.313+0000 7f55dd800640 1 -- 172.21.15.71:0/3204439342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641a9e39210 con 0x5641aa1c8a60 2024-09-05T12:44:10.315 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.313+0000 7f55d7e00640 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.314+0000 7f55d5600640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f55c00a2cf0 0x7f55c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.314+0000 7f55dd800640 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c8e40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55d0011570 tx=0x7f55d0011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667ca71fe98f609f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.314+0000 7f55d5600640 1 -- 172.21.15.71:0/237382110 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9526+0+0 (secure 0 0 0) 0x7f55cc0cf730 con 0x7f55d81018b0 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.314+0000 7f55dce00640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f55c00a2cf0 0x7f55c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.315+0000 7f55dce00640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f55c00a2cf0 0x7f55c00a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5641aa1c9ae0 tx=0x7f55cc09b490 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.315+0000 7f55ca000640 1 -- 172.21.15.71:0/3204439342 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0004b20 con 0x5641aa1c8a60 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.315+0000 7f55ca000640 1 -- 172.21.15.71:0/3204439342 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0004cc0 con 0x5641aa1c8a60 2024-09-05T12:44:10.316 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.315+0000 7f55ca000640 1 -- 172.21.15.71:0/3204439342 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00054c0 con 0x5641aa1c8a60 2024-09-05T12:44:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 -- 172.21.15.71:0/3204439342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 msgr2=0x5641aa1c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.317 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c8e40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f55d0011570 tx=0x7f55d0011a30 comp rx=0 tx=0).stop 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 -- 172.21.15.71:0/3204439342 shutdown_connections 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3204439342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c8e40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 -- 172.21.15.71:0/3204439342 >> 172.21.15.71:0/3204439342 conn(0x5641aa1c5130 msgr2=0x5641aa1c5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 -- 172.21.15.71:0/3204439342 shutdown_connections 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.316+0000 7f55e26a1740 1 -- 172.21.15.71:0/3204439342 wait complete. 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 -- start start 2024-09-05T12:44:10.318 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 Processor -- start 2024-09-05T12:44:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c9380 0x5641aa1c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1dc7f0 0x5641aa1c2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.321 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5641aa1df1e0 con 0x5641aa1c9380 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5641aa0af090 con 0x5641aa1dc7f0 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5641aa1df360 con 0x5641aa1c8a60 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.317+0000 7f55e26a1740 10 client.0 _ll_register_callbacks cb 0x5641aa103f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:42948/0 (socket says 172.21.15.71:42948) 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 -- 172.21.15.71:0/2051146737 learned_addr learned my addr 172.21.15.71:0/2051146737 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c9380 0x5641aa1c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1dc7f0 0x5641aa1c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 -- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1dc7f0 msgr2=0x5641aa1c2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1dc7f0 0x5641aa1c2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.322 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 -- 172.21.15.71:0/2051146737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c9380 msgr2=0x5641aa1c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c9380 0x5641aa1c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 -- 172.21.15.71:0/2051146737 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641a9f05620 con 0x5641aa1c8a60 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55dd800640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c17f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55d001b040 tx=0x7f55d0005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7ad0a48244be2a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.318+0000 7f55bea00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55bea00640 1 -- 172.21.15.71:0/2051146737 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d002c020 con 0x5641aa1c8a60 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55bea00640 1 -- 172.21.15.71:0/2051146737 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f55d0016040 con 0x5641aa1c8a60 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d0004cc0 con 0x5641aa1c8a60 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55bea00640 1 -- 172.21.15.71:0/2051146737 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f55d0037050 con 0x5641aa1c8a60 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 10 client.27298 fetch_fsmap learned FSMap version 70 2024-09-05T12:44:10.323 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5641aa1eae20 con 0x5641aa1c8a60 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55bea00640 1 -- 172.21.15.71:0/2051146737 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0004cc0 con 0x5641aa1c8a60 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55bea00640 1 -- 172.21.15.71:0/2051146737 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 70) ==== 8402+0+0 (secure 0 0 0) 0x7f55d0028450 con 0x5641aa1c8a60 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 10 client.27298 fetch_fsmap finished waiting for FSMap version 70 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 10 client.27298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 20 client.27298 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa1ec280 0x5641aa1ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 4 client.27298 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.319+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641aa133830 con 0x5641aa1ec280 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.320+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa1ec280 0x5641aa1ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.320+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa1ec280 0x5641aa1ee670 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbe7b70c83a9b0c2 server_cookie=34a4493fc8f9a5cc in_seq=0 out_seq=0 2024-09-05T12:44:10.324 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.320+0000 7f55bea00640 10 client.27298 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55bea00640 1 -- 172.21.15.71:0/2051146737 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641aa133830 con 0x5641aa1ec280 2024-09-05T12:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55bea00640 10 client.27298 handle_command_reply: tid=0 2024-09-05T12:44:10.325 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 client.27298 shutdown 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 msgr2=0x5641aa1c17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c17f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55d001b040 tx=0x7f55d0005000 comp rx=0 tx=0).stop 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 shutdown_connections 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa1ec280 0x5641aa1ee670 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1dc7f0 0x5641aa1c2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c9380 0x5641aa1c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2051146737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c17f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55bea00640 0 client.27298 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 >> 172.21.15.71:0/2051146737 conn(0x5641aa1c5130 msgr2=0x5641aa1cae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.321+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 shutdown_connections 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.322+0000 7f55e26a1740 1 -- 172.21.15.71:0/2051146737 wait complete. 2024-09-05T12:44:10.326 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.322+0000 7f55e26a1740 20 client.27298 trim_cache size 0 max 16384 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.336+0000 7f55e26a1740 1 Processor -- start 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.336+0000 7f55e26a1740 1 -- start start 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55e26a1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81acc50 con 0x5641aa1dc7f0 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55e26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81ae8e0 con 0x5641aa1c9380 2024-09-05T12:44:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55e26a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81ad380 con 0x5641aa1c8a60 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1f3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43522/0 (socket says 172.21.15.71:43522) 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 -- 172.21.15.71:0/3273028928 learned_addr learned my addr 172.21.15.71:0/3273028928 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55d7e00640 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 -- 172.21.15.71:0/3273028928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 msgr2=0x5641aa1c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 -- 172.21.15.71:0/3273028928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 msgr2=0x5641aa1c20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.337+0000 7f55dd800640 1 -- 172.21.15.71:0/3273028928 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641aa1bd3b0 con 0x5641aa1dc7f0 2024-09-05T12:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.338+0000 7f55dd800640 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1f3db0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f55d0035040 tx=0x7f55d0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f6693adb1405ac server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.338+0000 7f55c9600640 1 -- 172.21.15.71:0/3273028928 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0041070 con 0x5641aa1dc7f0 2024-09-05T12:44:10.343 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.338+0000 7f55c9600640 1 -- 172.21.15.71:0/3273028928 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0009d10 con 0x5641aa1dc7f0 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.338+0000 7f55c9600640 1 -- 172.21.15.71:0/3273028928 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d002c040 con 0x5641aa1dc7f0 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.338+0000 7f55e26a1740 1 -- 172.21.15.71:0/3273028928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 msgr2=0x5641aa1f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.338+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1f3db0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f55d0035040 tx=0x7f55d0014040 comp rx=0 tx=0).stop 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 -- 172.21.15.71:0/3273028928 shutdown_connections 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1dc7f0 0x5641aa1f3db0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1c20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 --2- 172.21.15.71:0/3273028928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1c8a60 0x5641aa1c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 -- 172.21.15.71:0/3273028928 >> 172.21.15.71:0/3273028928 conn(0x5641aa1c5130 msgr2=0x5641aa1c00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 -- 172.21.15.71:0/3273028928 shutdown_connections 2024-09-05T12:44:10.344 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 -- 172.21.15.71:0/3273028928 wait complete. 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 -- start start 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.339+0000 7f55e26a1740 1 Processor -- start 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 0x5641aa1df5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1dfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1dc7f0 0x5641aa1e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81ae210 con 0x5641aa1c8a60 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81ada80 con 0x5641aa1c9380 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81aac90 con 0x5641aa1dc7f0 2024-09-05T12:44:10.345 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55e26a1740 10 client.0 _ll_register_callbacks cb 0x7f55d0004ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 0x5641aa1df5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 0x5641aa1df5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43526/0 (socket says 172.21.15.71:43526) 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 -- 172.21.15.71:0/2704068827 learned_addr learned my addr 172.21.15.71:0/2704068827 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1dfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1dc7f0 0x5641aa1e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 -- 172.21.15.71:0/2704068827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1dc7f0 msgr2=0x5641aa1e0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1dc7f0 0x5641aa1e0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 -- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 msgr2=0x5641aa1dfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1dfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 -- 172.21.15.71:0/2704068827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641aa255010 con 0x5641aa1c8a60 2024-09-05T12:44:10.346 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1dfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55dd800640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 0x5641aa1df5d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f55d003f040 tx=0x7f55d0028600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc471fa611fb6c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.340+0000 7f55bea00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55bea00640 1 -- 172.21.15.71:0/2704068827 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0041040 con 0x5641aa1c8a60 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d0009d10 con 0x5641aa1c8a60 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55bea00640 1 -- 172.21.15.71:0/2704068827 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f55d0028a40 con 0x5641aa1c8a60 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55bea00640 1 -- 172.21.15.71:0/2704068827 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f55d0051050 con 0x5641aa1c8a60 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 10 client.17262 fetch_fsmap learned FSMap version 70 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5641aa256320 con 0x5641aa1c8a60 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55bea00640 1 -- 172.21.15.71:0/2704068827 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55d0009d10 con 0x5641aa1c8a60 2024-09-05T12:44:10.347 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55bea00640 1 -- 172.21.15.71:0/2704068827 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) ==== 8402+0+0 (secure 0 0 0) 0x7f55d004b510 con 0x5641aa1c8a60 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 10 client.17262 fetch_fsmap finished waiting for FSMap version 70 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 20 client.17262 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa256920 0x5641aa258d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 4 client.17262 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.341+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5641aa12db80 con 0x5641aa256920 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.342+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa256920 0x5641aa258d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.342+0000 7f55d7e00640 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa256920 0x5641aa258d40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9960b43fd233435f server_cookie=a5b784fbd01bda5f in_seq=0 out_seq=0 2024-09-05T12:44:10.348 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.342+0000 7f55bea00640 10 client.17262 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.460+0000 7f55bea00640 1 -- 172.21.15.71:0/2704068827 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5641aa12db80 con 0x5641aa256920 2024-09-05T12:44:10.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.460+0000 7f55bea00640 10 client.17262 handle_command_reply: tid=0 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.460+0000 7f55e26a1740 1 client.17262 shutdown 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 msgr2=0x5641aa1df5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 0x5641aa1df5d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f55d003f040 tx=0x7f55d0028600 comp rx=0 tx=0).stop 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 shutdown_connections 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5641aa256920 0x5641aa258d40 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5641aa1dc7f0 0x5641aa1e0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5641aa1c9380 0x5641aa1dfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 --2- 172.21.15.71:0/2704068827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5641aa1c8a60 0x5641aa1df5d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 >> 172.21.15.71:0/2704068827 conn(0x5641aa1c5130 msgr2=0x5641aa1c36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55bea00640 0 client.17262 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 shutdown_connections 2024-09-05T12:44:10.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 1 -- 172.21.15.71:0/2704068827 wait complete. 2024-09-05T12:44:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.461+0000 7f55e26a1740 20 client.17262 trim_cache size 0 max 16384 2024-09-05T12:44:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:44:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:44:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:44:10.464 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f55c00a2cf0 msgr2=0x7f55c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f55c00a2cf0 0x7f55c00a51b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5641aa1c9ae0 tx=0x7f55cc09b490 comp rx=0 tx=0).stop 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d81018b0 msgr2=0x7f55d819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d81018b0 0x7f55d819a490 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f55cc00e9f0 tx=0x7f55cc00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 shutdown_connections 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f55c00a2cf0 0x7f55c00a51b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d81078c0 0x7f55d819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d8102250 0x7f55d819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 --2- 172.21.15.71:0/237382110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d81018b0 0x7f55d819a490 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 >> 172.21.15.71:0/237382110 conn(0x7f55d80fd5d0 msgr2=0x7f55d810cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 shutdown_connections 2024-09-05T12:44:10.465 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:10.462+0000 7f55de200640 1 -- 172.21.15.71:0/237382110 wait complete. 2024-09-05T12:44:10.792 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 14.40 MB/sec execute 64 sec latency 6488.481 ms 2024-09-05T12:44:11.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:11 smithi071 ceph-mon[29845]: osdmap e118: 12 total, 12 up, 12 in 2024-09-05T12:44:11.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:11 smithi123 ceph-mon[34321]: osdmap e118: 12 total, 12 up, 12 in 2024-09-05T12:44:11.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:11 smithi174 ceph-mon[34313]: osdmap e118: 12 total, 12 up, 12 in 2024-09-05T12:44:11.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 14.18 MB/sec execute 65 sec latency 7488.645 ms 2024-09-05T12:44:12.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:12 smithi071 ceph-mon[29845]: pgmap v519: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-09-05T12:44:12.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:12 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:12 smithi123 ceph-mon[34321]: pgmap v519: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-09-05T12:44:12.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:12 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:12 smithi174 ceph-mon[34313]: pgmap v519: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 309 op/s 2024-09-05T12:44:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:12 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:12.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 13.96 MB/sec execute 66 sec latency 8488.808 ms 2024-09-05T12:44:13.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 13.75 MB/sec execute 67 sec latency 9488.877 ms 2024-09-05T12:44:14.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:14 smithi071 ceph-mon[29845]: pgmap v520: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:44:14.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:14 smithi123 ceph-mon[34321]: pgmap v520: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:44:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:14 smithi174 ceph-mon[34313]: pgmap v520: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:44:14.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 13.55 MB/sec execute 68 sec latency 10489.042 ms 2024-09-05T12:44:15.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 13.36 MB/sec execute 69 sec latency 11489.152 ms 2024-09-05T12:44:16.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:16 smithi071 ceph-mon[29845]: pgmap v521: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:44:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:16 smithi123 ceph-mon[34321]: pgmap v521: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:44:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:16 smithi174 ceph-mon[34313]: pgmap v521: 129 pgs: 129 active+clean; 630 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 263 op/s 2024-09-05T12:44:16.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 13.16 MB/sec execute 70 sec latency 12489.296 ms 2024-09-05T12:44:17.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 12.98 MB/sec execute 71 sec latency 13489.423 ms 2024-09-05T12:44:18.541 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:44:18.593 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:18 smithi071 ceph-mon[29845]: pgmap v522: 129 pgs: 129 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-05T12:44:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:18 smithi123 ceph-mon[34321]: pgmap v522: 129 pgs: 129 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-05T12:44:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:18 smithi174 ceph-mon[34313]: pgmap v522: 129 pgs: 129 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-05T12:44:18.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 12.80 MB/sec execute 72 sec latency 14489.464 ms 2024-09-05T12:44:18.918 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:44:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1578363185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 msgr2=0x7ff2dc114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:19.483 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1578363185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc114570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009900 tx=0x7ff2c802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1578363185 shutdown_connections 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1578363185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc114ab0 0x7ff2dc116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1578363185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc114570 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1578363185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1578363185 >> 172.21.15.71:0/1578363185 conn(0x7ff2dc06e2d0 msgr2=0x7ff2dc06e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1578363185 shutdown_connections 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.475+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1578363185 wait complete. 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 Processor -- start 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 -- start start 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:19.484 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc1a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc114ab0 0x7ff2dc1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc11a760 con 0x7ff2dc109ba0 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc11a5e0 con 0x7ff2dc1091f0 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.476+0000 7ff2e3e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc11a8e0 con 0x7ff2dc114ab0 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc1a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e1400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:43124/0 (socket says 172.21.15.71:43124) 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e1400640 1 -- 172.21.15.71:0/1814801833 learned_addr learned my addr 172.21.15.71:0/1814801833 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e1e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc114ab0 0x7ff2dc1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 -- 172.21.15.71:0/1814801833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc114ab0 msgr2=0x7ff2dc1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc114ab0 0x7ff2dc1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 -- 172.21.15.71:0/1814801833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 msgr2=0x7ff2dc1a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:19.485 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 -- 172.21.15.71:0/1814801833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc1a1180 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e1400640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.477+0000 7ff2e0a00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc1a6d30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005940 tx=0x7ff2c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.478+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c803f030 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.478+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2c8031990 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.478+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8035070 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.478+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1a1410 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.478+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1ad6c0 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.479+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc1095d0 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.481+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff2c8042030 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.482+0000 7ff2da000640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2c40a2cf0 0x7ff2c40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.482+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7ff2c80efaf0 con 0x7ff2dc109ba0 2024-09-05T12:44:19.486 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.482+0000 7ff2e1400640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2c40a2cf0 0x7ff2c40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.483+0000 7ff2e1400640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2c40a2cf0 0x7ff2c40a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc1a1af0 tx=0x7ff2d0005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:19.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.486+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2c80bbcd0 con 0x7ff2dc109ba0 2024-09-05T12:44:19.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.697+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2dc1ad9f0 con 0x7ff2dc109ba0 2024-09-05T12:44:19.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.698+0000 7ff2da000640 1 -- 172.21.15.71:0/1814801833 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11982 (secure 0 0 0) 0x7ff2c80bffd0 con 0x7ff2dc109ba0 2024-09-05T12:44:19.699 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:44:19.699 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":71,"btime":"2024-09-05T12:44:11:308726+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:44:10.802225+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24485,24524,24490,24502,24514]},"id":1}]} 2024-09-05T12:44:19.699 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 71 2024-09-05T12:44:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2c40a2cf0 msgr2=0x7ff2c40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2c40a2cf0 0x7ff2c40a51b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc1a1af0 tx=0x7ff2d0005f90 comp rx=0 tx=0).stop 2024-09-05T12:44:19.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 msgr2=0x7ff2dc1a6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc1a6d30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8005940 tx=0x7ff2c80042f0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 shutdown_connections 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2dc114ab0 0x7ff2dc1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff2c40a2cf0 0x7ff2c40a51b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff2dc109ba0 0x7ff2dc1a6d30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 --2- 172.21.15.71:0/1814801833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1091f0 0x7ff2dc1a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 >> 172.21.15.71:0/1814801833 conn(0x7ff2dc06e2d0 msgr2=0x7ff2dc070930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 shutdown_connections 2024-09-05T12:44:19.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:19.700+0000 7ff2e3e00640 1 -- 172.21.15.71:0/1814801833 wait complete. 2024-09-05T12:44:19.793 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 12.62 MB/sec execute 73 sec latency 15489.536 ms 2024-09-05T12:44:20.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:20 smithi071 ceph-mon[29845]: pgmap v523: 129 pgs: 129 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-05T12:44:20.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:20 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1814801833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:44:20.527 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id a1d3f86b --release --await' 2024-09-05T12:44:20.528 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id a1d3f86b --release --await 2024-09-05T12:44:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:20 smithi123 ceph-mon[34321]: pgmap v523: 129 pgs: 129 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-05T12:44:20.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:20 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1814801833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:44:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:20 smithi174 ceph-mon[34313]: pgmap v523: 129 pgs: 129 active+clean; 638 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 1.1 MiB/s wr, 67 op/s 2024-09-05T12:44:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:20 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1814801833' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:44:20.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58339 12.45 MB/sec execute 74 sec latency 16489.587 ms 2024-09-05T12:44:20.944 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:44:21.354 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:21 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:44:21.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:21 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.687+0000 7f1db4000640 1 -- 172.21.15.71:0/3780293274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 msgr2=0x7f1dac1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.687+0000 7f1db4000640 1 --2- 172.21.15.71:0/3780293274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 0x7f1dac1067d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d9802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 -- 172.21.15.71:0/3780293274 shutdown_connections 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 --2- 172.21.15.71:0/3780293274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac106f00 0x7f1dac10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 --2- 172.21.15.71:0/3780293274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 0x7f1dac1067d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 --2- 172.21.15.71:0/3780293274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac101dd0 0x7f1dac1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 -- 172.21.15.71:0/3780293274 >> 172.21.15.71:0/3780293274 conn(0x7f1dac0fd890 msgr2=0x7f1dac0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 -- 172.21.15.71:0/3780293274 shutdown_connections 2024-09-05T12:44:21.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.688+0000 7f1db4000640 1 -- 172.21.15.71:0/3780293274 wait complete. 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 Processor -- start 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 -- start start 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 0x7f1dac19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 0x7f1dac19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac106f00 0x7f1dac19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac111f60 con 0x7f1dac101dd0 2024-09-05T12:44:21.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac111de0 con 0x7f1dac106f00 2024-09-05T12:44:21.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db4000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1120e0 con 0x7f1dac102780 2024-09-05T12:44:21.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.689+0000 7f1db1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 0x7f1dac19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 0x7f1dac19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:47570/0 (socket says 172.21.15.71:47570) 2024-09-05T12:44:21.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 0x7f1dac19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 -- 172.21.15.71:0/2106313133 learned_addr learned my addr 172.21.15.71:0/2106313133 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db2000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac106f00 0x7f1dac19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 -- 172.21.15.71:0/2106313133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 msgr2=0x7f1dac19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 0x7f1dac19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 -- 172.21.15.71:0/2106313133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac106f00 msgr2=0x7f1dac19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac106f00 0x7f1dac19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 -- 172.21.15.71:0/2106313133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac19f630 con 0x7f1dac101dd0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1db1600640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 0x7f1dac19a680 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1da000e9f0 tx=0x7f1da000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.690+0000 7f1daa000640 1 -- 172.21.15.71:0/2106313133 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da000cd80 con 0x7f1dac101dd0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.691+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac19f920 con 0x7f1dac101dd0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.691+0000 7f1daa000640 1 -- 172.21.15.71:0/2106313133 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da0005500 con 0x7f1dac101dd0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.691+0000 7f1daa000640 1 -- 172.21.15.71:0/2106313133 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0010690 con 0x7f1dac101dd0 2024-09-05T12:44:21.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.691+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac1a7200 con 0x7f1dac101dd0 2024-09-05T12:44:21.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.694+0000 7f1db6c2f740 1 Processor -- start 2024-09-05T12:44:21.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.694+0000 7f1daa000640 1 -- 172.21.15.71:0/2106313133 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1da00046c0 con 0x7f1dac101dd0 2024-09-05T12:44:21.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.694+0000 7f1db6c2f740 1 -- start start 2024-09-05T12:44:21.695 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f13646f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364c30 0x55b3f136efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 0x55b3f13718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b3f12ca650 con 0x55b3f1364310 2024-09-05T12:44:21.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1271f10 con 0x55b3f1364c30 2024-09-05T12:44:21.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b3f12c6610 con 0x55b3f136f4f0 2024-09-05T12:44:21.696 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 0x55b3f13718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 0x55b3f13718e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48638/0 (socket says 172.21.15.71:48638) 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 -- 172.21.15.71:0/1054955963 learned_addr learned my addr 172.21.15.71:0/1054955963 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db2000640 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f13646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db1600640 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364c30 0x55b3f136efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 -- 172.21.15.71:0/1054955963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364c30 msgr2=0x55b3f136efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364c30 0x55b3f136efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 -- 172.21.15.71:0/1054955963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 msgr2=0x55b3f13646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f13646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.697 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.695+0000 7f1db0c00640 1 -- 172.21.15.71:0/1054955963 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f0fd9580 con 0x55b3f136f4f0 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1daa000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d940a2cf0 0x7f1d940a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1db0c00640 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 0x55b3f13718e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d98005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50ed5f56dc686dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1daa000640 1 -- 172.21.15.71:0/2106313133 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f1da00cfbc0 con 0x7f1dac101dd0 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1db2000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d940a2cf0 0x7f1d940a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1d9e600640 1 -- 172.21.15.71:0/1054955963 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98040070 con 0x55b3f136f4f0 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1d9e600640 1 -- 172.21.15.71:0/1054955963 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98031870 con 0x55b3f136f4f0 2024-09-05T12:44:21.698 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.696+0000 7f1d9e600640 1 -- 172.21.15.71:0/1054955963 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98035040 con 0x55b3f136f4f0 2024-09-05T12:44:21.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1054955963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 msgr2=0x55b3f13718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 0x55b3f13718e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d98005be0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1054955963 shutdown_connections 2024-09-05T12:44:21.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f136f4f0 0x55b3f13718e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364c30 0x55b3f136efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1054955963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f13646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.697+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1054955963 >> 172.21.15.71:0/1054955963 conn(0x55b3f13608a0 msgr2=0x55b3f1360c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:21.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.698+0000 7f1db2000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d940a2cf0 0x7f1d940a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005fd0 tx=0x7f1da4009500 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:21.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.698+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1054955963 shutdown_connections 2024-09-05T12:44:21.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.698+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1054955963 wait complete. 2024-09-05T12:44:21.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.698+0000 7f1db6c2f740 1 -- start start 2024-09-05T12:44:21.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.698+0000 7f1db6c2f740 1 Processor -- start 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f136c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f136c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f136ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1372060 con 0x55b3f1364310 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b3f1285350 con 0x55b3f136f4f0 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b3f13721e0 con 0x55b3f1364c30 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db6c2f740 10 client.0 _ll_register_callbacks cb 0x55b3f12a8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:21.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f136ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db1600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f136c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f136ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:43176/0 (socket says 172.21.15.71:43176) 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db2000640 1 -- 172.21.15.71:0/2270686044 learned_addr learned my addr 172.21.15.71:0/2270686044 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.699+0000 7f1db0c00640 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f136c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1db2000640 1 -- 172.21.15.71:0/2270686044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 msgr2=0x55b3f136c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1db2000640 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f136c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1db2000640 1 -- 172.21.15.71:0/2270686044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 msgr2=0x55b3f136c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1db2000640 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f136c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1db2000640 1 -- 172.21.15.71:0/2270686044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f10a3960 con 0x55b3f136f4f0 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1db2000640 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f136ce60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1da4003fa0 tx=0x7f1da4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7452677dcb19ec server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:21.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.700+0000 7f1d92a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:44:21.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1d92a00640 1 -- 172.21.15.71:0/2270686044 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4020820 con 0x55b3f136f4f0 2024-09-05T12:44:21.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d98031870 con 0x55b3f136f4f0 2024-09-05T12:44:21.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1d92a00640 1 -- 172.21.15.71:0/2270686044 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f1da40209c0 con 0x55b3f136f4f0 2024-09-05T12:44:21.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1d92a00640 1 -- 172.21.15.71:0/2270686044 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f1da4020bc0 con 0x55b3f136f4f0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 10 client.27275 fetch_fsmap learned FSMap version 71 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55b3f137dd20 con 0x55b3f136f4f0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1d92a00640 1 -- 172.21.15.71:0/2270686044 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98031870 con 0x55b3f136f4f0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1d92a00640 1 -- 172.21.15.71:0/2270686044 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 71) ==== 8398+0+0 (secure 0 0 0) 0x7f1da4027070 con 0x55b3f136f4f0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 10 client.27275 fetch_fsmap finished waiting for FSMap version 71 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 10 client.27275 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 20 client.27275 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.701+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f137f2f0 0x55b3f13816e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.702+0000 7f1db6c2f740 4 client.27275 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.702+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3f12d7bc0 con 0x55b3f137f2f0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.702+0000 7f1db1600640 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f137f2f0 0x55b3f13816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.702+0000 7f1db1600640 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f137f2f0 0x55b3f13816e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1700afa900169b19 server_cookie=65c756786ce058fa in_seq=0 out_seq=0 2024-09-05T12:44:21.705 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.702+0000 7f1d92a00640 10 client.27275 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1d92a00640 1 -- 172.21.15.71:0/2270686044 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3f12d7bc0 con 0x55b3f137f2f0 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1d92a00640 10 client.27275 handle_command_reply: tid=0 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 client.27275 shutdown 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 msgr2=0x55b3f136ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f136ce60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1da4003fa0 tx=0x7f1da4004ab0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 shutdown_connections 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f137f2f0 0x55b3f13816e0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f136ce60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f136c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/2270686044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364310 0x55b3f136c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1d92a00640 0 client.27275 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.703+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 >> 172.21.15.71:0/2270686044 conn(0x55b3f13608a0 msgr2=0x55b3f13665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:21.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.704+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 shutdown_connections 2024-09-05T12:44:21.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.704+0000 7f1db6c2f740 1 -- 172.21.15.71:0/2270686044 wait complete. 2024-09-05T12:44:21.707 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.704+0000 7f1db6c2f740 20 client.27275 trim_cache size 0 max 16384 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.721+0000 7f1db6c2f740 1 Processor -- start 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.721+0000 7f1db6c2f740 1 -- start start 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364310 0x55b3f1386f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364c30 0x55b3f1387480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f1369660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b1200 con 0x55b3f1364c30 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b2e60 con 0x55b3f136f4f0 2024-09-05T12:44:21.723 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b1930 con 0x55b3f1364310 2024-09-05T12:44:21.724 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f1369660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364c30 0x55b3f1387480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f1369660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:43184/0 (socket says 172.21.15.71:43184) 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db1600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364310 0x55b3f1386f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.722+0000 7f1db2000640 1 -- 172.21.15.71:0/1317630329 learned_addr learned my addr 172.21.15.71:0/1317630329 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db1600640 1 -- 172.21.15.71:0/1317630329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 msgr2=0x55b3f1369660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db1600640 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f1369660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db1600640 1 -- 172.21.15.71:0/1317630329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364c30 msgr2=0x55b3f1387480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db1600640 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364c30 0x55b3f1387480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db1600640 1 -- 172.21.15.71:0/1317630329 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3f1381d10 con 0x55b3f1364310 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db2000640 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f1369660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:21.725 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db0c00640 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364c30 0x55b3f1387480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1db1600640 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364310 0x55b3f1386f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1da00ccf80 tx=0x7f1da0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb21f52cf849b326 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1d9dc00640 1 -- 172.21.15.71:0/1317630329 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da009b890 con 0x55b3f1364310 2024-09-05T12:44:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1d9dc00640 1 -- 172.21.15.71:0/1317630329 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da009c450 con 0x55b3f1364310 2024-09-05T12:44:21.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.723+0000 7f1d9dc00640 1 -- 172.21.15.71:0/1317630329 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da009b6b0 con 0x55b3f1364310 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1317630329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364310 msgr2=0x55b3f1386f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364310 0x55b3f1386f40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1da00ccf80 tx=0x7f1da0004a40 comp rx=0 tx=0).stop 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1317630329 shutdown_connections 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f136f4f0 0x55b3f1369660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f1364c30 0x55b3f1387480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/1317630329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364310 0x55b3f1386f40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1317630329 >> 172.21.15.71:0/1317630329 conn(0x55b3f13608a0 msgr2=0x55b3f13665d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1317630329 shutdown_connections 2024-09-05T12:44:21.729 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.724+0000 7f1db6c2f740 1 -- 172.21.15.71:0/1317630329 wait complete. 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 -- start start 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 Processor -- start 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364310 0x55b3f1377c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f1374400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f136f4f0 0x55b3f13749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b2790 con 0x55b3f136f4f0 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1b2000 con 0x55b3f1364310 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac1af200 con 0x55b3f1364c30 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.725+0000 7f1db6c2f740 10 client.0 _ll_register_callbacks cb 0x7f1da4009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f136f4f0 0x55b3f13749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f1374400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f1374400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:48656/0 (socket says 172.21.15.71:48656) 2024-09-05T12:44:21.730 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 -- 172.21.15.71:0/4174520953 learned_addr learned my addr 172.21.15.71:0/4174520953 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364310 0x55b3f1377c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 -- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364310 msgr2=0x55b3f1377c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364310 0x55b3f1377c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 -- 172.21.15.71:0/4174520953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f136f4f0 msgr2=0x55b3f13749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f136f4f0 0x55b3f13749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db0c00640 1 -- 172.21.15.71:0/4174520953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3f13751d0 con 0x55b3f1364c30 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.726+0000 7f1db2000640 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f136f4f0 0x55b3f13749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1db0c00640 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f1374400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d98005420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c4e75ce34f7cc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1d92a00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1d92a00640 1 -- 172.21.15.71:0/4174520953 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98040040 con 0x55b3f1364c30 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da009c450 con 0x55b3f1364c30 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1d92a00640 1 -- 172.21.15.71:0/4174520953 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f1d98004230 con 0x55b3f1364c30 2024-09-05T12:44:21.731 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1d92a00640 1 -- 172.21.15.71:0/4174520953 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f1d98035040 con 0x55b3f1364c30 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1db6c2f740 10 client.27328 fetch_fsmap learned FSMap version 71 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x55b3f13eb030 con 0x55b3f1364c30 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.727+0000 7f1d92a00640 1 -- 172.21.15.71:0/4174520953 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1da009c450 con 0x55b3f1364c30 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1d92a00640 1 -- 172.21.15.71:0/4174520953 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 71) ==== 8398+0+0 (secure 0 0 0) 0x7f1d9803c440 con 0x55b3f1364c30 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db6c2f740 10 client.27328 fetch_fsmap finished waiting for FSMap version 71 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db6c2f740 10 client.27328 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db6c2f740 20 client.27328 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f13eb3e0 0x55b3f13ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db6c2f740 4 client.27328 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a1d3f86b", "release": true, "await": true}] 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a1d3f86b", "release": true, "await": true}) -- 0x55b3f12d20d0 con 0x55b3f13eb3e0 2024-09-05T12:44:21.732 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.728+0000 7f1db1600640 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f13eb3e0 0x55b3f13ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:21.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.729+0000 7f1db1600640 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f13eb3e0 0x55b3f13ed800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da58df76980deddc server_cookie=32e77f8cad307ba0 in_seq=0 out_seq=0 2024-09-05T12:44:21.733 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.729+0000 7f1d92a00640 10 client.27328 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.747+0000 7f1d92a00640 1 -- 172.21.15.71:0/4174520953 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b3f12d20d0 con 0x55b3f13eb3e0 2024-09-05T12:44:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.747+0000 7f1d92a00640 10 client.27328 handle_command_reply: tid=0 2024-09-05T12:44:21.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.747+0000 7f1db6c2f740 1 client.27328 shutdown 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 msgr2=0x55b3f1374400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f1374400 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1d98009900 tx=0x7f1d98005420 comp rx=0 tx=0).stop 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 shutdown_connections 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55b3f13eb3e0 0x55b3f13ed800 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3f136f4f0 0x55b3f13749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1d92a00640 0 client.27328 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b3f1364c30 0x55b3f1374400 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 --2- 172.21.15.71:0/4174520953 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b3f1364310 0x55b3f1377c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.748+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 >> 172.21.15.71:0/4174520953 conn(0x55b3f13608a0 msgr2=0x55b3f135f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.749+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 shutdown_connections 2024-09-05T12:44:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.749+0000 7f1db6c2f740 1 -- 172.21.15.71:0/4174520953 wait complete. 2024-09-05T12:44:21.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.749+0000 7f1db6c2f740 20 client.27328 trim_cache size 0 max 16384 2024-09-05T12:44:21.752 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:44:21.752 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 71, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 56, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "a1d3f86b": { 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 56, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 140.0, 2024-09-05T12:44:21.753 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:44:21.754 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.752+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d940a2cf0 msgr2=0x7f1d940a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.752+0000 7f1db4000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d940a2cf0 0x7f1d940a51b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1da4005fd0 tx=0x7f1da4009500 comp rx=0 tx=0).stop 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.752+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 msgr2=0x7f1dac19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.752+0000 7f1db4000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 0x7f1dac19a680 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1da000e9f0 tx=0x7f1da000edf0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 shutdown_connections 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dac106f00 0x7f1dac19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dac102780 0x7f1dac19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d940a2cf0 0x7f1d940a51b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 --2- 172.21.15.71:0/2106313133 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dac101dd0 0x7f1dac19a680 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 >> 172.21.15.71:0/2106313133 conn(0x7f1dac0fd890 msgr2=0x7f1dac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.753+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 shutdown_connections 2024-09-05T12:44:21.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:44:21.754+0000 7f1db4000640 1 -- 172.21.15.71:0/2106313133 wait complete. 2024-09-05T12:44:21.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:21 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:44:21.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 58374 12.35 MB/sec execute 75 sec latency 17433.190 ms 2024-09-05T12:44:22.653 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:22 smithi071 ceph-mon[29845]: pgmap v524: 129 pgs: 129 active+clean; 647 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.5 MiB/s wr, 85 op/s 2024-09-05T12:44:22.653 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:22 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:44:22.653 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:22 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:44:22.653 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:22 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:44:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:22 smithi123 ceph-mon[34321]: pgmap v524: 129 pgs: 129 active+clean; 647 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.5 MiB/s wr, 85 op/s 2024-09-05T12:44:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:22 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:44:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:22 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:44:22.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:22 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:44:22.722 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a1d3f86b', total seconds elapsed: 24.2 2024-09-05T12:44:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:22 smithi174 ceph-mon[34313]: pgmap v524: 129 pgs: 129 active+clean; 647 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 2.5 MiB/s wr, 85 op/s 2024-09-05T12:44:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:22 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:44:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:22 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:44:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:22 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:44:22.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 59058 12.66 MB/sec execute 76 sec latency 68.019 ms 2024-09-05T12:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.645+0000 7f9eaaa00640 1 -- 172.21.15.71:0/3640291223 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c037070 con 0x7f9eac103f60 2024-09-05T12:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 -- 172.21.15.71:0/3640291223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 msgr2=0x7f9eac104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 --2- 172.21.15.71:0/3640291223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 0x7f9eac104340 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c009860 tx=0x7f9e9c02a360 comp rx=0 tx=0).stop 2024-09-05T12:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 -- 172.21.15.71:0/3640291223 shutdown_connections 2024-09-05T12:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 --2- 172.21.15.71:0/3640291223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac1090b0 0x7f9eac110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.647 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 --2- 172.21.15.71:0/3640291223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac104910 0x7f9eac108950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 --2- 172.21.15.71:0/3640291223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 0x7f9eac104340 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.646+0000 7f9eb2600640 1 -- 172.21.15.71:0/3640291223 >> 172.21.15.71:0/3640291223 conn(0x7f9eac0ff970 msgr2=0x7f9eac101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.647+0000 7f9eb2600640 1 -- 172.21.15.71:0/3640291223 shutdown_connections 2024-09-05T12:44:23.648 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.647+0000 7f9eb2600640 1 -- 172.21.15.71:0/3640291223 wait complete. 2024-09-05T12:44:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 Processor -- start 2024-09-05T12:44:23.649 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 -- start start 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 0x7f9eac0737c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 0x7f9eac073d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 0x7f9eac06dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac06c220 con 0x7f9eac103f60 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac06c0a0 con 0x7f9eac06d880 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eb2600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac06c3a0 con 0x7f9eac1090b0 2024-09-05T12:44:23.650 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.649+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 0x7f9eac06dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 0x7f9eac073d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 0x7f9eac06dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33360/0 (socket says 172.21.15.71:33360) 2024-09-05T12:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 0x7f9eac073d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:57324/0 (socket says 172.21.15.71:57324) 2024-09-05T12:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9eabe00640 1 -- 172.21.15.71:0/369046321 learned_addr learned my addr 172.21.15.71:0/369046321 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:23.651 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9eab400640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 0x7f9eac0737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 -- 172.21.15.71:0/369046321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 msgr2=0x7f9eac06dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 0x7f9eac06dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 -- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 msgr2=0x7f9eac0737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 0x7f9eac0737c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 -- 172.21.15.71:0/369046321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eac06e520 con 0x7f9eac1090b0 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9eabe00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 0x7f9eac06dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:23.652 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.650+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 0x7f9eac073d00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e94005fd0 tx=0x7f9e9400ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.651+0000 7f9ea8c00640 1 -- 172.21.15.71:0/369046321 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400cc90 con 0x7f9eac1090b0 2024-09-05T12:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.651+0000 7f9ea8c00640 1 -- 172.21.15.71:0/369046321 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9400ee70 con 0x7f9eac1090b0 2024-09-05T12:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.651+0000 7f9ea8c00640 1 -- 172.21.15.71:0/369046321 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004a00 con 0x7f9eac1090b0 2024-09-05T12:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.652+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eac1af920 con 0x7f9eac1090b0 2024-09-05T12:44:23.653 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.652+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eac1afd90 con 0x7f9eac1090b0 2024-09-05T12:44:23.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 Processor -- start 2024-09-05T12:44:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9ea8c00640 1 -- 172.21.15.71:0/369046321 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f9e94004ba0 con 0x7f9eac1090b0 2024-09-05T12:44:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 -- start start 2024-09-05T12:44:23.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 0x560782f834c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f83a00 0x560782f96670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560782ee9100 con 0x560782f830e0 2024-09-05T12:44:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560782e911b0 con 0x560782f96bb0 2024-09-05T12:44:23.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.658+0000 7f9eb549b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560782ee50c0 con 0x560782f83a00 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f98fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33364/0 (socket says 172.21.15.71:33364) 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 -- 172.21.15.71:0/1033122321 learned_addr learned my addr 172.21.15.71:0/1033122321 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9eab400640 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f83a00 0x560782f96670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 -- 172.21.15.71:0/1033122321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f83a00 msgr2=0x560782f96670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f83a00 0x560782f96670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 -- 172.21.15.71:0/1033122321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 msgr2=0x560782f834c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 0x560782f834c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.659+0000 7f9ea3e00640 1 -- 172.21.15.71:0/1033122321 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560782bfc210 con 0x560782f96bb0 2024-09-05T12:44:23.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.660+0000 7f9eab400640 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f83a00 0x560782f96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:23.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:23 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.661+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f98fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e940ccfa0 tx=0x7f9e940d87a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52377c8728c46ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.661+0000 7f9ea0c00640 1 -- 172.21.15.71:0/1033122321 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940d6860 con 0x560782f96bb0 2024-09-05T12:44:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.661+0000 7f9ea0c00640 1 -- 172.21.15.71:0/1033122321 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e940dd070 con 0x560782f96bb0 2024-09-05T12:44:23.662 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.662+0000 7f9ea0c00640 1 -- 172.21.15.71:0/1033122321 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940d53b0 con 0x560782f96bb0 2024-09-05T12:44:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.663+0000 7f9eb549b740 1 -- 172.21.15.71:0/1033122321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 msgr2=0x560782f98fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.663+0000 7f9eb549b740 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f98fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e940ccfa0 tx=0x7f9e940d87a0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.664 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.663+0000 7f9ea8c00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f9e900a2e20 0x7f9e900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9eabe00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f9e900a2e20 0x7f9e900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9ea8c00640 1 -- 172.21.15.71:0/369046321 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f9e940cfc70 con 0x7f9eac1090b0 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9eb549b740 1 -- 172.21.15.71:0/1033122321 shutdown_connections 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9eb549b740 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f98fa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9eb549b740 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f83a00 0x560782f96670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9eb549b740 1 --2- 172.21.15.71:0/1033122321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 0x560782f834c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.664+0000 7f9eb549b740 1 -- 172.21.15.71:0/1033122321 >> 172.21.15.71:0/1033122321 conn(0x560782f7f750 msgr2=0x560782f7fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.665+0000 7f9eb549b740 1 -- 172.21.15.71:0/1033122321 shutdown_connections 2024-09-05T12:44:23.666 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.665+0000 7f9eb549b740 1 -- 172.21.15.71:0/1033122321 wait complete. 2024-09-05T12:44:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.665+0000 7f9eabe00640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f9e900a2e20 0x7f9e900a52e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c4d0 tx=0x7f9ea400b040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.666+0000 7f9eb549b740 1 -- start start 2024-09-05T12:44:23.667 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.666+0000 7f9eb549b740 1 Processor -- start 2024-09-05T12:44:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 0x560782f93b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 0x560782f94070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782f945b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.668 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560782f997d0 con 0x560782f830e0 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560782e72090 con 0x560782f83a00 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560782f99950 con 0x560782f96bb0 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eb549b740 10 client.0 _ll_register_callbacks cb 0x560782ec6f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 0x560782f94070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9eabe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782f945b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9ea3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 0x560782f94070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33370/0 (socket says 172.21.15.71:33370) 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9ea3e00640 1 -- 172.21.15.71:0/3013384176 learned_addr learned my addr 172.21.15.71:0/3013384176 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9ea3e00640 1 -- 172.21.15.71:0/3013384176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 msgr2=0x560782f945b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782f945b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.669 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.667+0000 7f9ea3e00640 1 -- 172.21.15.71:0/3013384176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 msgr2=0x560782f93b30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:44:23.670 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.668+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 0x560782f93b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.668+0000 7f9ea3e00640 1 -- 172.21.15.71:0/3013384176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560782cc8620 con 0x560782f83a00 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.668+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 0x560782f94070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e940a4540 tx=0x7f9e940d9720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749c7f634a6e7d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.668+0000 7f9e99600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.669+0000 7f9e99600640 1 -- 172.21.15.71:0/3013384176 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940980a0 con 0x560782f83a00 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.669+0000 7f9e99600640 1 -- 172.21.15.71:0/3013384176 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f9e9409a100 con 0x560782f83a00 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.669+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e940dd070 con 0x560782f83a00 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.669+0000 7f9e99600640 1 -- 172.21.15.71:0/3013384176 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9e940056b0 con 0x560782f83a00 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.669+0000 7f9eb549b740 10 client.27290 fetch_fsmap learned FSMap version 72 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.669+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x560782f885b0 con 0x560782f83a00 2024-09-05T12:44:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.670+0000 7f9e99600640 1 -- 172.21.15.71:0/3013384176 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e940dd070 con 0x560782f83a00 2024-09-05T12:44:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.670+0000 7f9e99600640 1 -- 172.21.15.71:0/3013384176 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 72) ==== 8394+0+0 (secure 0 0 0) 0x7f9e9401f070 con 0x560782f83a00 2024-09-05T12:44:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eb549b740 10 client.27290 fetch_fsmap finished waiting for FSMap version 72 2024-09-05T12:44:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eb549b740 10 client.27290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eb549b740 20 client.27290 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560782f89a20 0x560782f8be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eb549b740 4 client.27290 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560782ef6830 con 0x560782f89a20 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.671+0000 7f9eab400640 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560782f89a20 0x560782f8be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.672+0000 7f9eab400640 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560782f89a20 0x560782f8be80 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f883598c8cce9929 server_cookie=948b5c4cc086792c in_seq=0 out_seq=0 2024-09-05T12:44:23.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.672+0000 7f9e99600640 10 client.27290 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.675+0000 7f9e99600640 1 -- 172.21.15.71:0/3013384176 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560782ef6830 con 0x560782f89a20 2024-09-05T12:44:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.675+0000 7f9e99600640 10 client.27290 handle_command_reply: tid=0 2024-09-05T12:44:23.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.675+0000 7f9eb549b740 1 client.27290 shutdown 2024-09-05T12:44:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 msgr2=0x560782f94070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 0x560782f94070 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e940a4540 tx=0x7f9e940d9720 comp rx=0 tx=0).stop 2024-09-05T12:44:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 shutdown_connections 2024-09-05T12:44:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560782f89a20 0x560782f8be80 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782f945b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f83a00 0x560782f94070 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3013384176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f830e0 0x560782f93b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 >> 172.21.15.71:0/3013384176 conn(0x560782f7f750 msgr2=0x560782f854b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9e99600640 0 client.27290 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 shutdown_connections 2024-09-05T12:44:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 1 -- 172.21.15.71:0/3013384176 wait complete. 2024-09-05T12:44:23.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.676+0000 7f9eb549b740 20 client.27290 trim_cache size 0 max 16384 2024-09-05T12:44:23.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:23 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.700+0000 7f9eb549b740 1 Processor -- start 2024-09-05T12:44:23.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.700+0000 7f9eb549b740 1 -- start start 2024-09-05T12:44:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.701+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 0x560782f96290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.701+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f943e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.701+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782fb8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.701+0000 7f9eb549b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b54c0 con 0x560782f83a00 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.701+0000 7f9eb549b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b70f0 con 0x560782f830e0 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.701+0000 7f9eb549b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b5bc0 con 0x560782f96bb0 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.702+0000 7f9eab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 0x560782f96290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.702+0000 7f9eab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 0x560782f96290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33380/0 (socket says 172.21.15.71:33380) 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.702+0000 7f9eab400640 1 -- 172.21.15.71:0/2012361289 learned_addr learned my addr 172.21.15.71:0/2012361289 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.702+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f943e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.702+0000 7f9eabe00640 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782fb8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.703+0000 7f9eab400640 1 -- 172.21.15.71:0/2012361289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 msgr2=0x560782fb8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.703+0000 7f9eab400640 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782fb8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.703+0000 7f9eab400640 1 -- 172.21.15.71:0/2012361289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 msgr2=0x560782f943e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.703+0000 7f9eab400640 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f943e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.703+0000 7f9eab400640 1 -- 172.21.15.71:0/2012361289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560782f92060 con 0x560782f830e0 2024-09-05T12:44:23.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.704+0000 7f9eab400640 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 0x560782f96290 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c02a920 tx=0x7f9e9c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba5893e131a163 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.705+0000 7f9e9be00640 1 -- 172.21.15.71:0/2012361289 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c037040 con 0x560782f830e0 2024-09-05T12:44:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.705+0000 7f9e9be00640 1 -- 172.21.15.71:0/2012361289 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c042040 con 0x560782f830e0 2024-09-05T12:44:23.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.705+0000 7f9e9be00640 1 -- 172.21.15.71:0/2012361289 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c030c10 con 0x560782f830e0 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.706+0000 7f9eb549b740 1 -- 172.21.15.71:0/2012361289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 msgr2=0x560782f96290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.706+0000 7f9eb549b740 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 0x560782f96290 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c02a920 tx=0x7f9e9c009500 comp rx=0 tx=0).stop 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 -- 172.21.15.71:0/2012361289 shutdown_connections 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f96bb0 0x560782fb8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f943e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 --2- 172.21.15.71:0/2012361289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f830e0 0x560782f96290 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 -- 172.21.15.71:0/2012361289 >> 172.21.15.71:0/2012361289 conn(0x560782f7f750 msgr2=0x560782f8e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 -- 172.21.15.71:0/2012361289 shutdown_connections 2024-09-05T12:44:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.707+0000 7f9eb549b740 1 -- 172.21.15.71:0/2012361289 wait complete. 2024-09-05T12:44:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.708+0000 7f9eb549b740 1 -- start start 2024-09-05T12:44:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.711+0000 7f9eb549b740 1 Processor -- start 2024-09-05T12:44:23.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.712+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 0x560782f99c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.712+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f9a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 0x560782f99c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 0x560782f99c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:57364/0 (socket says 172.21.15.71:57364) 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eb549b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f9a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eb549b740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b6a20 con 0x560782f83a00 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eb549b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b6290 con 0x560782f96bb0 2024-09-05T12:44:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eb549b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9eac1b3550 con 0x560782f830e0 2024-09-05T12:44:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eab400640 1 -- 172.21.15.71:0/3047533672 learned_addr learned my addr 172.21.15.71:0/3047533672 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.713+0000 7f9eb549b740 10 client.0 _ll_register_callbacks cb 0x7f9e940d3cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9eab400640 1 -- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 msgr2=0x560782f9a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9eab400640 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f9a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9eab400640 1 -- 172.21.15.71:0/3047533672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 msgr2=0x560782f9a1b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:44:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9eab400640 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f9a1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9eab400640 1 -- 172.21.15.71:0/3047533672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560783017fc0 con 0x560782f830e0 2024-09-05T12:44:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9eab400640 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 0x560782f99c70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c035040 tx=0x7f9e9c02a360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5467e4090fb9e2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.714+0000 7f9e99600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:44:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.715+0000 7f9e99600640 1 -- 172.21.15.71:0/3047533672 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c037040 con 0x560782f830e0 2024-09-05T12:44:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.715+0000 7f9e99600640 1 -- 172.21.15.71:0/3047533672 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f9e9c005040 con 0x560782f830e0 2024-09-05T12:44:23.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.715+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9c042040 con 0x560782f830e0 2024-09-05T12:44:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.716+0000 7f9e99600640 1 -- 172.21.15.71:0/3047533672 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f9e9c004300 con 0x560782f830e0 2024-09-05T12:44:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.716+0000 7f9eb549b740 10 client.27343 fetch_fsmap learned FSMap version 72 2024-09-05T12:44:23.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.716+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x560783019250 con 0x560782f830e0 2024-09-05T12:44:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9e99600640 1 -- 172.21.15.71:0/3047533672 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e9c042040 con 0x560782f830e0 2024-09-05T12:44:23.720 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9e99600640 1 -- 172.21.15.71:0/3047533672 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 72) ==== 8394+0+0 (secure 0 0 0) 0x7f9e9c004820 con 0x560782f830e0 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9eb549b740 10 client.27343 fetch_fsmap finished waiting for FSMap version 72 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9eb549b740 10 client.27343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9eb549b740 20 client.27343 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560783019980 0x56078301be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9eb549b740 4 client.27343 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.717+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560782ef0b80 con 0x560783019980 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.718+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560783019980 0x56078301be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.719+0000 7f9e99600640 10 client.27343 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:23.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.719+0000 7f9ea3e00640 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560783019980 0x56078301be40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8473a32176228d9 server_cookie=e9f08347b2b74b44 in_seq=0 out_seq=0 2024-09-05T12:44:23.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:23 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:23.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 59739 12.97 MB/sec execute 77 sec latency 65.138 ms 2024-09-05T12:44:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.893+0000 7f9e99600640 1 -- 172.21.15.71:0/3047533672 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560782ef0b80 con 0x560783019980 2024-09-05T12:44:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.893+0000 7f9e99600640 10 client.27343 handle_command_reply: tid=0 2024-09-05T12:44:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.893+0000 7f9eb549b740 1 client.27343 shutdown 2024-09-05T12:44:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.893+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 msgr2=0x560782f99c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.893+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 0x560782f99c70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c035040 tx=0x7f9e9c02a360 comp rx=0 tx=0).stop 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 shutdown_connections 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560783019980 0x56078301be40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560782f96bb0 0x560782f9a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560782f83a00 0x560782f9a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 --2- 172.21.15.71:0/3047533672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560782f830e0 0x560782f99c70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 >> 172.21.15.71:0/3047533672 conn(0x560782f7f750 msgr2=0x560782f7e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:23.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9e99600640 0 client.27343 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.894+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 shutdown_connections 2024-09-05T12:44:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.895+0000 7f9eb549b740 1 -- 172.21.15.71:0/3047533672 wait complete. 2024-09-05T12:44:23.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.895+0000 7f9eb549b740 20 client.27343 trim_cache size 0 max 16384 2024-09-05T12:44:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:44:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:44:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:44:23.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f9e900a2e20 msgr2=0x7f9e900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f9e900a2e20 0x7f9e900a52e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c4d0 tx=0x7f9ea400b040 comp rx=0 tx=0).stop 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 msgr2=0x7f9eac073d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 0x7f9eac073d00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e94005fd0 tx=0x7f9e9400ec20 comp rx=0 tx=0).stop 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 shutdown_connections 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9eac06d880 0x7f9eac06dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9eac1090b0 0x7f9eac073d00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f9e900a2e20 0x7f9e900a52e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 --2- 172.21.15.71:0/369046321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9eac103f60 0x7f9eac0737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 >> 172.21.15.71:0/369046321 conn(0x7f9eac0ff970 msgr2=0x7f9eac100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 shutdown_connections 2024-09-05T12:44:23.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:23.897+0000 7f9eb2600640 1 -- 172.21.15.71:0/369046321 wait complete. 2024-09-05T12:44:24.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:24 smithi071 ceph-mon[29845]: pgmap v525: 129 pgs: 129 active+clean; 649 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 2.3 MiB/s wr, 53 op/s 2024-09-05T12:44:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:24 smithi123 ceph-mon[34321]: pgmap v525: 129 pgs: 129 active+clean; 649 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 2.3 MiB/s wr, 53 op/s 2024-09-05T12:44:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:24 smithi174 ceph-mon[34313]: pgmap v525: 129 pgs: 129 active+clean; 649 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 237 KiB/s rd, 2.3 MiB/s wr, 53 op/s 2024-09-05T12:44:24.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 60618 13.50 MB/sec execute 78 sec latency 90.071 ms 2024-09-05T12:44:25.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 61521 13.91 MB/sec execute 79 sec latency 56.216 ms 2024-09-05T12:44:26.537 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:26 smithi071 ceph-mon[29845]: pgmap v526: 129 pgs: 129 active+clean; 649 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.3 MiB/s wr, 43 op/s 2024-09-05T12:44:26.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:26 smithi123 ceph-mon[34321]: pgmap v526: 129 pgs: 129 active+clean; 649 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.3 MiB/s wr, 43 op/s 2024-09-05T12:44:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:26 smithi174 ceph-mon[34313]: pgmap v526: 129 pgs: 129 active+clean; 649 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.3 MiB/s wr, 43 op/s 2024-09-05T12:44:26.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 62248 14.20 MB/sec execute 80 sec latency 64.912 ms 2024-09-05T12:44:27.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 63136 14.64 MB/sec execute 81 sec latency 61.406 ms 2024-09-05T12:44:28.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:28 smithi071 ceph-mon[29845]: pgmap v527: 129 pgs: 129 active+clean; 674 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-09-05T12:44:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:28 smithi123 ceph-mon[34321]: pgmap v527: 129 pgs: 129 active+clean; 674 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-09-05T12:44:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:28 smithi174 ceph-mon[34313]: pgmap v527: 129 pgs: 129 active+clean; 674 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-09-05T12:44:28.794 INFO:tasks.workunit.client.0.smithi071.stdout: 10 64176 15.04 MB/sec execute 82 sec latency 64.237 ms 2024-09-05T12:44:29.808 INFO:tasks.workunit.client.0.smithi071.stdout: 10 64710 15.31 MB/sec execute 83 sec latency 769.151 ms 2024-09-05T12:44:30.478 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:30 smithi071 ceph-mon[29845]: pgmap v528: 129 pgs: 129 active+clean; 674 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 308 op/s 2024-09-05T12:44:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:30 smithi123 ceph-mon[34321]: pgmap v528: 129 pgs: 129 active+clean; 674 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 308 op/s 2024-09-05T12:44:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:30 smithi174 ceph-mon[34313]: pgmap v528: 129 pgs: 129 active+clean; 674 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 308 op/s 2024-09-05T12:44:30.808 INFO:tasks.workunit.client.0.smithi071.stdout: 10 65168 15.38 MB/sec execute 84 sec latency 1344.082 ms 2024-09-05T12:44:31.808 INFO:tasks.workunit.client.0.smithi071.stdout: 10 66362 16.06 MB/sec execute 85 sec latency 64.188 ms 2024-09-05T12:44:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:32 smithi123 ceph-mon[34321]: pgmap v529: 129 pgs: 129 active+clean; 701 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.3 MiB/s wr, 520 op/s 2024-09-05T12:44:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:32 smithi174 ceph-mon[34313]: pgmap v529: 129 pgs: 129 active+clean; 701 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.3 MiB/s wr, 520 op/s 2024-09-05T12:44:32.808 INFO:tasks.workunit.client.0.smithi071.stdout: 10 67037 16.27 MB/sec execute 86 sec latency 142.495 ms 2024-09-05T12:44:32.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:32 smithi071 ceph-mon[29845]: pgmap v529: 129 pgs: 129 active+clean; 701 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 7.3 MiB/s wr, 520 op/s 2024-09-05T12:44:33.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:33 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:33.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:33 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:33.808 INFO:tasks.workunit.client.0.smithi071.stdout: 10 67869 16.54 MB/sec execute 87 sec latency 61.497 ms 2024-09-05T12:44:33.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:33 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:34 smithi174 ceph-mon[34313]: pgmap v530: 129 pgs: 129 active+clean; 707 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-05T12:44:34.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 68386 16.62 MB/sec execute 88 sec latency 691.021 ms 2024-09-05T12:44:34.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:34 smithi071 ceph-mon[29845]: pgmap v530: 129 pgs: 129 active+clean; 707 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-05T12:44:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:34 smithi123 ceph-mon[34321]: pgmap v530: 129 pgs: 129 active+clean; 707 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.6 MiB/s wr, 561 op/s 2024-09-05T12:44:35.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 68388 16.43 MB/sec execute 89 sec latency 1691.083 ms 2024-09-05T12:44:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:36 smithi174 ceph-mon[34313]: pgmap v531: 129 pgs: 129 active+clean; 707 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.4 MiB/s wr, 553 op/s 2024-09-05T12:44:36.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 69460 16.87 MB/sec execute 90 sec latency 1856.562 ms 2024-09-05T12:44:36.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:36 smithi071 ceph-mon[29845]: pgmap v531: 129 pgs: 129 active+clean; 707 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.4 MiB/s wr, 553 op/s 2024-09-05T12:44:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:36 smithi123 ceph-mon[34321]: pgmap v531: 129 pgs: 129 active+clean; 707 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.4 MiB/s wr, 553 op/s 2024-09-05T12:44:37.085 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.082+0000 7f5b88600640 1 -- 172.21.15.71:0/4107491566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80103f50 msgr2=0x7f5b80104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.082+0000 7f5b88600640 1 --2- 172.21.15.71:0/4107491566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80103f50 0x7f5b80104330 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009900 tx=0x7f5b7402a410 comp rx=0 tx=0).stop 2024-09-05T12:44:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 -- 172.21.15.71:0/4107491566 shutdown_connections 2024-09-05T12:44:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 --2- 172.21.15.71:0/4107491566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b801090a0 0x7f5b80110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 --2- 172.21.15.71:0/4107491566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b80104900 0x7f5b80108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 --2- 172.21.15.71:0/4107491566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b80103f50 0x7f5b80104330 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 -- 172.21.15.71:0/4107491566 >> 172.21.15.71:0/4107491566 conn(0x7f5b800ff9e0 msgr2=0x7f5b80101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 -- 172.21.15.71:0/4107491566 shutdown_connections 2024-09-05T12:44:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.083+0000 7f5b88600640 1 -- 172.21.15.71:0/4107491566 wait complete. 2024-09-05T12:44:37.087 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.084+0000 7f5b88600640 1 Processor -- start 2024-09-05T12:44:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 -- start start 2024-09-05T12:44:37.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b80103f50 0x7f5b8010f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b801090a0 0x7f5b8010f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8010fbb0 0x7f5b801aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80077430 con 0x7f5b801090a0 2024-09-05T12:44:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b800772b0 con 0x7f5b8010fbb0 2024-09-05T12:44:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.086+0000 7f5b88600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b800775b0 con 0x7f5b80103f50 2024-09-05T12:44:37.091 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.087+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b80103f50 0x7f5b8010f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.087+0000 7f5b85200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b801090a0 0x7f5b8010f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.087+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b80103f50 0x7f5b8010f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36318/0 (socket says 172.21.15.71:36318) 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.087+0000 7f5b85c00640 1 -- 172.21.15.71:0/1732443266 learned_addr learned my addr 172.21.15.71:0/1732443266 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.087+0000 7f5b86600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8010fbb0 0x7f5b801aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b86600640 1 -- 172.21.15.71:0/1732443266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b80103f50 msgr2=0x7f5b8010f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b86600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b80103f50 0x7f5b8010f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.092 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b86600640 1 -- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b801090a0 msgr2=0x7f5b8010f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b86600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b801090a0 0x7f5b8010f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b86600640 1 -- 172.21.15.71:0/1732443266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b801ab0a0 con 0x7f5b8010fbb0 2024-09-05T12:44:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b85200640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b801090a0 0x7f5b8010f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.088+0000 7f5b86600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8010fbb0 0x7f5b801aa8d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009fd0 tx=0x7f5b7801ff20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.089+0000 7f5b7e000640 1 -- 172.21.15.71:0/1732443266 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800ce40 con 0x7f5b8010fbb0 2024-09-05T12:44:37.093 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.089+0000 7f5b7e000640 1 -- 172.21.15.71:0/1732443266 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b78005020 con 0x7f5b8010fbb0 2024-09-05T12:44:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.089+0000 7f5b7e000640 1 -- 172.21.15.71:0/1732443266 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b78021750 con 0x7f5b8010fbb0 2024-09-05T12:44:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.089+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b801133a0 con 0x7f5b8010fbb0 2024-09-05T12:44:37.094 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.089+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b80113840 con 0x7f5b8010fbb0 2024-09-05T12:44:37.096 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.094+0000 7f5b8b656740 1 Processor -- start 2024-09-05T12:44:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b7e000640 1 -- 172.21.15.71:0/1732443266 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f5b78021930 con 0x7f5b8010fbb0 2024-09-05T12:44:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 -- start start 2024-09-05T12:44:37.097 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b51d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b527e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b5283a0 0x559d2b52a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559d2b483100 con 0x559d2b51d1e0 2024-09-05T12:44:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d2b42b1b0 con 0x559d2b51db00 2024-09-05T12:44:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.095+0000 7f5b8b656740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d2b47f0c0 con 0x559d2b5283a0 2024-09-05T12:44:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.096+0000 7f5b85200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b51d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.096+0000 7f5b85200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b51d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:34410/0 (socket says 172.21.15.71:34410) 2024-09-05T12:44:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85200640 1 -- 172.21.15.71:0/1218233783 learned_addr learned my addr 172.21.15.71:0/1218233783 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85c00640 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b527e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85200640 1 -- 172.21.15.71:0/1218233783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b5283a0 msgr2=0x559d2b52a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85200640 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b5283a0 0x559d2b52a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85200640 1 -- 172.21.15.71:0/1218233783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 msgr2=0x559d2b527e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85200640 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b527e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.097+0000 7f5b85200640 1 -- 172.21.15.71:0/1218233783 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2b196210 con 0x559d2b51d1e0 2024-09-05T12:44:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.098+0000 7f5b85200640 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b51d5c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0114f0 tx=0x7f5b6c0119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90b34eb99548929 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.099+0000 7f5b72a00640 1 -- 172.21.15.71:0/1218233783 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c004220 con 0x559d2b51d1e0 2024-09-05T12:44:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.099+0000 7f5b72a00640 1 -- 172.21.15.71:0/1218233783 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c0043c0 con 0x559d2b51d1e0 2024-09-05T12:44:37.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.099+0000 7f5b72a00640 1 -- 172.21.15.71:0/1218233783 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c004220 con 0x559d2b51d1e0 2024-09-05T12:44:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.099+0000 7f5b7e000640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b680a2e70 0x7f5b680a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.100+0000 7f5b8b656740 1 -- 172.21.15.71:0/1218233783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 msgr2=0x559d2b51d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.100+0000 7f5b8b656740 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b51d5c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0114f0 tx=0x7f5b6c0119b0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.100+0000 7f5b7e000640 1 -- 172.21.15.71:0/1732443266 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f5b780e0e90 con 0x7f5b8010fbb0 2024-09-05T12:44:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.101+0000 7f5b85c00640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b680a2e70 0x7f5b680a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.101+0000 7f5b8b656740 1 -- 172.21.15.71:0/1218233783 shutdown_connections 2024-09-05T12:44:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.101+0000 7f5b8b656740 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b5283a0 0x559d2b52a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.101+0000 7f5b8b656740 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b527e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.101+0000 7f5b8b656740 1 --2- 172.21.15.71:0/1218233783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b51d5c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.101+0000 7f5b8b656740 1 -- 172.21.15.71:0/1218233783 >> 172.21.15.71:0/1218233783 conn(0x559d2b519750 msgr2=0x559d2b519b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:37.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.102+0000 7f5b8b656740 1 -- 172.21.15.71:0/1218233783 shutdown_connections 2024-09-05T12:44:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.102+0000 7f5b8b656740 1 -- 172.21.15.71:0/1218233783 wait complete. 2024-09-05T12:44:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.102+0000 7f5b85c00640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b680a2e70 0x7f5b680a5330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b740096c0 tx=0x7f5b7402a910 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.102+0000 7f5b8b656740 1 -- start start 2024-09-05T12:44:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.103+0000 7f5b8b656740 1 Processor -- start 2024-09-05T12:44:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.104+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51d1e0 0x559d2b5251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.104+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b525730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.104+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b525c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.104+0000 7f5b8b656740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559d2b52afd0 con 0x559d2b5283a0 2024-09-05T12:44:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.104+0000 7f5b8b656740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d2b40c090 con 0x559d2b51d1e0 2024-09-05T12:44:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.105+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51d1e0 0x559d2b5251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.105+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51d1e0 0x559d2b5251f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45992/0 (socket says 172.21.15.71:45992) 2024-09-05T12:44:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.105+0000 7f5b85c00640 1 -- 172.21.15.71:0/363303914 learned_addr learned my addr 172.21.15.71:0/363303914 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.105+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d2b52b150 con 0x559d2b51db00 2024-09-05T12:44:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.105+0000 7f5b85200640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b525730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.105+0000 7f5b86600640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b525c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.106+0000 7f5b8b656740 10 client.0 _ll_register_callbacks cb 0x559d2b460f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.107+0000 7f5b86600640 1 -- 172.21.15.71:0/363303914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 msgr2=0x559d2b525730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.107+0000 7f5b86600640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b525730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.107+0000 7f5b86600640 1 -- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51d1e0 msgr2=0x559d2b5251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.107+0000 7f5b86600640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51d1e0 0x559d2b5251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.107+0000 7f5b86600640 1 -- 172.21.15.71:0/363303914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2b262620 con 0x559d2b5283a0 2024-09-05T12:44:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.108+0000 7f5b86600640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b525c70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5b780de1f0 tx=0x7f5b780055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3bf532a1d16d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.108+0000 7f5b67400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:44:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.108+0000 7f5b67400640 1 -- 172.21.15.71:0/363303914 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780de670 con 0x559d2b5283a0 2024-09-05T12:44:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.108+0000 7f5b67400640 1 -- 172.21.15.71:0/363303914 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f5b780a8b80 con 0x559d2b5283a0 2024-09-05T12:44:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.108+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6c0043c0 con 0x559d2b5283a0 2024-09-05T12:44:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.109+0000 7f5b67400640 1 -- 172.21.15.71:0/363303914 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5b780a8dc0 con 0x559d2b5283a0 2024-09-05T12:44:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.109+0000 7f5b8b656740 10 client.17301 fetch_fsmap learned FSMap version 73 2024-09-05T12:44:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.109+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x559d2b547b90 con 0x559d2b5283a0 2024-09-05T12:44:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b67400640 1 -- 172.21.15.71:0/363303914 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c0043c0 con 0x559d2b5283a0 2024-09-05T12:44:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b67400640 1 -- 172.21.15.71:0/363303914 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) ==== 8382+0+0 (secure 0 0 0) 0x7f5b780af070 con 0x559d2b5283a0 2024-09-05T12:44:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b8b656740 10 client.17301 fetch_fsmap finished waiting for FSMap version 73 2024-09-05T12:44:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b8b656740 10 client.17301 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b8b656740 20 client.17301 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:37.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b8b656740 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b549050 0x559d2b54b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b8b656740 4 client.17301 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:44:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.110+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d2b490830 con 0x559d2b549050 2024-09-05T12:44:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.112+0000 7f5b85c00640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b549050 0x559d2b54b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.113+0000 7f5b67400640 10 client.17301 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:37.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.113+0000 7f5b85c00640 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b549050 0x559d2b54b440 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eaaf3bf18263770 server_cookie=d1e61c7ced1b9aed in_seq=0 out_seq=0 2024-09-05T12:44:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.115+0000 7f5b67400640 1 -- 172.21.15.71:0/363303914 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d2b490830 con 0x559d2b549050 2024-09-05T12:44:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.115+0000 7f5b67400640 10 client.17301 handle_command_reply: tid=0 2024-09-05T12:44:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.115+0000 7f5b8b656740 1 client.17301 shutdown 2024-09-05T12:44:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.115+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 msgr2=0x559d2b525c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.115+0000 7f5b8b656740 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b525c70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5b780de1f0 tx=0x7f5b780055e0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 shutdown_connections 2024-09-05T12:44:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b8b656740 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b549050 0x559d2b54b440 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b8b656740 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b525c70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b8b656740 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b525730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b8b656740 1 --2- 172.21.15.71:0/363303914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51d1e0 0x559d2b5251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 >> 172.21.15.71:0/363303914 conn(0x559d2b519750 msgr2=0x559d2b51f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:37.119 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.116+0000 7f5b67400640 0 client.17301 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.118+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 shutdown_connections 2024-09-05T12:44:37.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.118+0000 7f5b8b656740 1 -- 172.21.15.71:0/363303914 wait complete. 2024-09-05T12:44:37.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.119+0000 7f5b8b656740 20 client.17301 trim_cache size 0 max 16384 2024-09-05T12:44:37.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.156+0000 7f5b8b656740 1 Processor -- start 2024-09-05T12:44:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.156+0000 7f5b8b656740 1 -- start start 2024-09-05T12:44:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.156+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51d1e0 0x559d2b550d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.156+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b551250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b51ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b8b656740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8006d6c0 con 0x559d2b5283a0 2024-09-05T12:44:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b8b656740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8006f260 con 0x559d2b51db00 2024-09-05T12:44:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b8b656740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8006ddc0 con 0x559d2b51d1e0 2024-09-05T12:44:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51d1e0 0x559d2b550d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51d1e0 0x559d2b550d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36340/0 (socket says 172.21.15.71:36340) 2024-09-05T12:44:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b85200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b551250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.157+0000 7f5b85c00640 1 -- 172.21.15.71:0/889513793 learned_addr learned my addr 172.21.15.71:0/889513793 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b85200640 1 -- 172.21.15.71:0/889513793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51d1e0 msgr2=0x559d2b550d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b85200640 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51d1e0 0x559d2b550d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b85200640 1 -- 172.21.15.71:0/889513793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 msgr2=0x559d2b51ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b85200640 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b51ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b85200640 1 -- 172.21.15.71:0/889513793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d2b519580 con 0x559d2b51db00 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b85200640 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b551250 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c006010 tx=0x7f5b6c00f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aafec3e64be60f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:37.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b72000640 1 -- 172.21.15.71:0/889513793 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c028020 con 0x559d2b51db00 2024-09-05T12:44:37.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.158+0000 7f5b72000640 1 -- 172.21.15.71:0/889513793 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c00fd00 con 0x559d2b51db00 2024-09-05T12:44:37.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.159+0000 7f5b72000640 1 -- 172.21.15.71:0/889513793 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c004220 con 0x559d2b51db00 2024-09-05T12:44:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.159+0000 7f5b8b656740 1 -- 172.21.15.71:0/889513793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 msgr2=0x559d2b551250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.159+0000 7f5b8b656740 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b551250 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c006010 tx=0x7f5b6c00f7b0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 -- 172.21.15.71:0/889513793 shutdown_connections 2024-09-05T12:44:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b5283a0 0x559d2b51ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b51db00 0x559d2b551250 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 --2- 172.21.15.71:0/889513793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51d1e0 0x559d2b550d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 -- 172.21.15.71:0/889513793 >> 172.21.15.71:0/889513793 conn(0x559d2b519750 msgr2=0x559d2b5242c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 -- 172.21.15.71:0/889513793 shutdown_connections 2024-09-05T12:44:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 -- 172.21.15.71:0/889513793 wait complete. 2024-09-05T12:44:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 -- start start 2024-09-05T12:44:37.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.160+0000 7f5b8b656740 1 Processor -- start 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.161+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b522240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b522780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b8b656740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b5283a0 0x559d2b522cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b522240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b85200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b522780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b85200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b522780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:36354/0 (socket says 172.21.15.71:36354) 2024-09-05T12:44:37.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b85200640 1 -- 172.21.15.71:0/2864360735 learned_addr learned my addr 172.21.15.71:0/2864360735 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b86600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b5283a0 0x559d2b522cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8006eb90 con 0x559d2b51d1e0 2024-09-05T12:44:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8006e490 con 0x559d2b5283a0 2024-09-05T12:44:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b80104330 con 0x559d2b51db00 2024-09-05T12:44:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.162+0000 7f5b8b656740 10 client.0 _ll_register_callbacks cb 0x7f5b780b5c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:37.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b86600640 1 -- 172.21.15.71:0/2864360735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 msgr2=0x559d2b522780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b86600640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b522780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b86600640 1 -- 172.21.15.71:0/2864360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 msgr2=0x559d2b522240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b86600640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b522240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b86600640 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d2b5457a0 con 0x559d2b5283a0 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b85200640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b522780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.163+0000 7f5b85c00640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b522240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:44:37.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.164+0000 7f5b86600640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b5283a0 0x559d2b522cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000f80 tx=0x7f5b780b2950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e646cd9acbc6c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.164+0000 7f5b67400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:44:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.164+0000 7f5b67400640 1 -- 172.21.15.71:0/2864360735 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b780aa2b0 con 0x559d2b5283a0 2024-09-05T12:44:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.164+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6c00fd00 con 0x559d2b5283a0 2024-09-05T12:44:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.164+0000 7f5b67400640 1 -- 172.21.15.71:0/2864360735 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f5b780a8b80 con 0x559d2b5283a0 2024-09-05T12:44:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.165+0000 7f5b67400640 1 -- 172.21.15.71:0/2864360735 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f5b780de670 con 0x559d2b5283a0 2024-09-05T12:44:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.166+0000 7f5b8b656740 10 client.27311 fetch_fsmap learned FSMap version 73 2024-09-05T12:44:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.166+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x559d2b5b3e90 con 0x559d2b5283a0 2024-09-05T12:44:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.166+0000 7f5b67400640 1 -- 172.21.15.71:0/2864360735 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b6c00fd00 con 0x559d2b5283a0 2024-09-05T12:44:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b67400640 1 -- 172.21.15.71:0/2864360735 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 73) ==== 8382+0+0 (secure 0 0 0) 0x7f5b780af070 con 0x559d2b5283a0 2024-09-05T12:44:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b8b656740 10 client.27311 fetch_fsmap finished waiting for FSMap version 73 2024-09-05T12:44:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b8b656740 10 client.27311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b8b656740 20 client.27311 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b8b656740 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b5b4480 0x559d2b5b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b8b656740 4 client.27311 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d2b48ab80 con 0x559d2b5b4480 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.167+0000 7f5b85c00640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b5b4480 0x559d2b5b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.168+0000 7f5b67400640 10 client.27311 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.168+0000 7f5b85c00640 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b5b4480 0x559d2b5b68a0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db5c1bd00adeeb9f server_cookie=6bd4a024f96b324 in_seq=0 out_seq=0 2024-09-05T12:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.469+0000 7f5b67400640 1 -- 172.21.15.71:0/2864360735 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d2b48ab80 con 0x559d2b5b4480 2024-09-05T12:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.469+0000 7f5b67400640 10 client.27311 handle_command_reply: tid=0 2024-09-05T12:44:37.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.470+0000 7f5b8b656740 1 client.27311 shutdown 2024-09-05T12:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b5283a0 msgr2=0x559d2b522cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b5283a0 0x559d2b522cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5b78000f80 tx=0x7f5b780b2950 comp rx=0 tx=0).stop 2024-09-05T12:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 shutdown_connections 2024-09-05T12:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559d2b5b4480 0x559d2b5b68a0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d2b5283a0 0x559d2b522cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d2b51db00 0x559d2b522780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 --2- 172.21.15.71:0/2864360735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559d2b51d1e0 0x559d2b522240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b67400640 0 client.27311 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.471+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 >> 172.21.15.71:0/2864360735 conn(0x559d2b519750 msgr2=0x559d2b51ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.472+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 shutdown_connections 2024-09-05T12:44:37.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.472+0000 7f5b8b656740 1 -- 172.21.15.71:0/2864360735 wait complete. 2024-09-05T12:44:37.475 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.473+0000 7f5b8b656740 20 client.27311 trim_cache size 0 max 16384 2024-09-05T12:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:44:37.476 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.475+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b680a2e70 msgr2=0x7f5b680a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.475+0000 7f5b88600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b680a2e70 0x7f5b680a5330 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5b740096c0 tx=0x7f5b7402a910 comp rx=0 tx=0).stop 2024-09-05T12:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.475+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8010fbb0 msgr2=0x7f5b801aa8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:37.477 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.475+0000 7f5b88600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8010fbb0 0x7f5b801aa8d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009fd0 tx=0x7f5b7801ff20 comp rx=0 tx=0).stop 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 shutdown_connections 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8010fbb0 0x7f5b801aa8d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f5b680a2e70 0x7f5b680a5330 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5b801090a0 0x7f5b8010f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 --2- 172.21.15.71:0/1732443266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b80103f50 0x7f5b8010f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 >> 172.21.15.71:0/1732443266 conn(0x7f5b800ff9e0 msgr2=0x7f5b80100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.476+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 shutdown_connections 2024-09-05T12:44:37.478 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:37.477+0000 7f5b88600640 1 -- 172.21.15.71:0/1732443266 wait complete. 2024-09-05T12:44:37.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 70317 17.34 MB/sec execute 91 sec latency 243.874 ms 2024-09-05T12:44:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:38 smithi174 ceph-mon[34313]: pgmap v532: 129 pgs: 129 active+clean; 733 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 9.4 MiB/s wr, 803 op/s 2024-09-05T12:44:38.809 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:38 smithi071 ceph-mon[29845]: pgmap v532: 129 pgs: 129 active+clean; 733 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 9.4 MiB/s wr, 803 op/s 2024-09-05T12:44:38.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 71027 17.45 MB/sec execute 92 sec latency 143.964 ms 2024-09-05T12:44:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:38 smithi123 ceph-mon[34321]: pgmap v532: 129 pgs: 129 active+clean; 733 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 9.4 MiB/s wr, 803 op/s 2024-09-05T12:44:39.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 72116 17.96 MB/sec execute 93 sec latency 78.531 ms 2024-09-05T12:44:40.474 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:40 smithi071 ceph-mon[29845]: pgmap v533: 129 pgs: 129 active+clean; 733 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 521 op/s 2024-09-05T12:44:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:40 smithi174 ceph-mon[34313]: pgmap v533: 129 pgs: 129 active+clean; 733 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 521 op/s 2024-09-05T12:44:40.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 73001 18.23 MB/sec execute 94 sec latency 161.818 ms 2024-09-05T12:44:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:40 smithi123 ceph-mon[34321]: pgmap v533: 129 pgs: 129 active+clean; 733 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.9 MiB/s wr, 521 op/s 2024-09-05T12:44:41.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 73838 18.56 MB/sec execute 95 sec latency 59.342 ms 2024-09-05T12:44:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:42 smithi174 ceph-mon[34313]: pgmap v534: 129 pgs: 129 active+clean; 759 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 827 op/s 2024-09-05T12:44:42.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 74635 18.73 MB/sec execute 96 sec latency 205.340 ms 2024-09-05T12:44:42.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:42 smithi071 ceph-mon[29845]: pgmap v534: 129 pgs: 129 active+clean; 759 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 827 op/s 2024-09-05T12:44:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:42 smithi123 ceph-mon[34321]: pgmap v534: 129 pgs: 129 active+clean; 759 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 9.8 MiB/s wr, 827 op/s 2024-09-05T12:44:43.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75400 19.00 MB/sec execute 97 sec latency 404.177 ms 2024-09-05T12:44:44.638 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:44 smithi071 ceph-mon[29845]: pgmap v535: 129 pgs: 129 active+clean; 761 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 684 op/s 2024-09-05T12:44:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:44 smithi174 ceph-mon[34313]: pgmap v535: 129 pgs: 129 active+clean; 761 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 684 op/s 2024-09-05T12:44:44.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75811 19.10 MB/sec execute 98 sec latency 472.458 ms 2024-09-05T12:44:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:44 smithi123 ceph-mon[34321]: pgmap v535: 129 pgs: 129 active+clean; 761 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 684 op/s 2024-09-05T12:44:45.813 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75814 18.91 MB/sec execute 99 sec latency 1472.516 ms 2024-09-05T12:44:46.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:46 smithi071 ceph-mon[29845]: pgmap v536: 129 pgs: 129 active+clean; 761 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 623 op/s 2024-09-05T12:44:46.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75815 18.72 MB/sec execute 100 sec latency 2472.578 ms 2024-09-05T12:44:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:46 smithi123 ceph-mon[34321]: pgmap v536: 129 pgs: 129 active+clean; 761 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 623 op/s 2024-09-05T12:44:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:46 smithi174 ceph-mon[34313]: pgmap v536: 129 pgs: 129 active+clean; 761 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 623 op/s 2024-09-05T12:44:47.809 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75815 18.54 MB/sec execute 101 sec latency 3472.630 ms 2024-09-05T12:44:48.671 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:48 smithi071 ceph-mon[29845]: pgmap v537: 129 pgs: 129 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 904 op/s 2024-09-05T12:44:48.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75815 18.36 MB/sec execute 102 sec latency 4472.681 ms 2024-09-05T12:44:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:48 smithi123 ceph-mon[34321]: pgmap v537: 129 pgs: 129 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 904 op/s 2024-09-05T12:44:49.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:48 smithi174 ceph-mon[34313]: pgmap v537: 129 pgs: 129 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 904 op/s 2024-09-05T12:44:49.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75845 18.18 MB/sec execute 103 sec latency 5472.760 ms 2024-09-05T12:44:50.620 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:50 smithi071 ceph-mon[29845]: pgmap v538: 129 pgs: 129 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-09-05T12:44:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- 172.21.15.71:0/2591237771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074620 msgr2=0x7f6ab8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- 172.21.15.71:0/2591237771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074620 0x7f6ab8074a00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009900 tx=0x7f6aa002a4a0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- 172.21.15.71:0/2591237771 shutdown_connections 2024-09-05T12:44:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- 172.21.15.71:0/2591237771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ab810e320 0x7f6ab8110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- 172.21.15.71:0/2591237771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab8074f40 0x7f6ab810dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- 172.21.15.71:0/2591237771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074620 0x7f6ab8074a00 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009900 tx=0x7f6aa002a4a0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- 172.21.15.71:0/2591237771 >> 172.21.15.71:0/2591237771 conn(0x7f6ab80fd7e0 msgr2=0x7f6ab80ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- 172.21.15.71:0/2591237771 shutdown_connections 2024-09-05T12:44:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- 172.21.15.71:0/2591237771 wait complete. 2024-09-05T12:44:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 Processor -- start 2024-09-05T12:44:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- start start 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 0x7f6ab810af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab810e320 0x7f6ab810b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ab810b9a0 0x7f6ab81aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8114190 con 0x7f6ab810e320 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8114010 con 0x7f6ab810b9a0 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6abde00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab8114310 con 0x7f6ab8074f40 2024-09-05T12:44:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6ab6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 0x7f6ab810af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6ab6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 0x7f6ab810af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50846/0 (socket says 172.21.15.71:50846) 2024-09-05T12:44:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6ab6a00640 1 -- 172.21.15.71:0/3282935959 learned_addr learned my addr 172.21.15.71:0/3282935959 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.693+0000 7f6ab6000640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab810e320 0x7f6ab810b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.698+0000 7f6ab6a00640 1 -- 172.21.15.71:0/3282935959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ab810b9a0 msgr2=0x7f6ab81aaed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.698+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ab810b9a0 0x7f6ab81aaed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.698+0000 7f6ab6a00640 1 -- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab810e320 msgr2=0x7f6ab810b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.698+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab810e320 0x7f6ab810b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.698+0000 7f6ab6a00640 1 -- 172.21.15.71:0/3282935959 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab81ab410 con 0x7f6ab8074f40 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.698+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 0x7f6ab810af20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6aa002aa90 tx=0x7f6aa0003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.700+0000 7f6ab6000640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab810e320 0x7f6ab810b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.700+0000 7f6abca00640 1 -- 172.21.15.71:0/3282935959 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa003f020 con 0x7f6ab8074f40 2024-09-05T12:44:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.700+0000 7f6abca00640 1 -- 172.21.15.71:0/3282935959 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa003b460 con 0x7f6ab8074f40 2024-09-05T12:44:50.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.700+0000 7f6abca00640 1 -- 172.21.15.71:0/3282935959 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0035070 con 0x7f6ab8074f40 2024-09-05T12:44:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.700+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab81ab5e0 con 0x7f6ab8074f40 2024-09-05T12:44:50.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.700+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81aba50 con 0x7f6ab8074f40 2024-09-05T12:44:50.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.705+0000 7f6ac0dce740 1 Processor -- start 2024-09-05T12:44:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.710+0000 7f6abca00640 1 -- 172.21.15.71:0/3282935959 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6aa0042020 con 0x7f6ab8074f40 2024-09-05T12:44:50.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.710+0000 7f6ac0dce740 1 -- start start 2024-09-05T12:44:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acc25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107accce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56107ac28100 con 0x56107acc21e0 2024-09-05T12:44:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56107abd01b0 con 0x56107acc2b00 2024-09-05T12:44:50.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56107ac240c0 con 0x56107accd3a0 2024-09-05T12:44:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107accce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accf790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50852/0 (socket says 172.21.15.71:50852) 2024-09-05T12:44:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107accce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49602/0 (socket says 172.21.15.71:49602) 2024-09-05T12:44:50.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6000640 1 -- 172.21.15.71:0/73232626 learned_addr learned my addr 172.21.15.71:0/73232626 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab7400640 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acc25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 -- 172.21.15.71:0/73232626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 msgr2=0x56107accf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 -- 172.21.15.71:0/73232626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 msgr2=0x56107acc25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acc25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab6a00640 1 -- 172.21.15.71:0/73232626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107a93b210 con 0x56107acc2b00 2024-09-05T12:44:50.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6ab7400640 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.712+0000 7f6abca00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a9c0a2ef0 0x7f6a9c0a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ab7400640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a9c0a2ef0 0x7f6a9c0a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107accce60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00ecc50 tx=0x7f6aa00f5b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8b976af6d5b7b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:50.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6aa7e00640 1 -- 172.21.15.71:0/73232626 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00fe070 con 0x56107acc2b00 2024-09-05T12:44:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6abca00640 1 -- 172.21.15.71:0/3282935959 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f6aa00ef8f0 con 0x7f6ab8074f40 2024-09-05T12:44:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ab7400640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a9c0a2ef0 0x7f6a9c0a53b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0007db0 tx=0x7f6ab0007340 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:50.721 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6aa7e00640 1 -- 172.21.15.71:0/73232626 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa00f2e50 con 0x56107acc2b00 2024-09-05T12:44:50.722 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6aa7e00640 1 -- 172.21.15.71:0/73232626 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00b63a0 con 0x56107acc2b00 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 -- 172.21.15.71:0/73232626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 msgr2=0x56107accce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107accce60 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6aa00ecc50 tx=0x7f6aa00f5b20 comp rx=0 tx=0).stop 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 -- 172.21.15.71:0/73232626 shutdown_connections 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107accce60 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/73232626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acc25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.723 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.717+0000 7f6ac0dce740 1 -- 172.21.15.71:0/73232626 >> 172.21.15.71:0/73232626 conn(0x56107acbe750 msgr2=0x56107acbeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.722+0000 7f6ac0dce740 1 -- 172.21.15.71:0/73232626 shutdown_connections 2024-09-05T12:44:50.724 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.722+0000 7f6ac0dce740 1 -- 172.21.15.71:0/73232626 wait complete. 2024-09-05T12:44:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 -- start start 2024-09-05T12:44:50.727 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 Processor -- start 2024-09-05T12:44:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.728 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107acca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56107accfd90 con 0x56107acc21e0 2024-09-05T12:44:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56107abb1090 con 0x56107acc2b00 2024-09-05T12:44:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56107accff10 con 0x56107accd3a0 2024-09-05T12:44:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ac0dce740 10 client.0 _ll_register_callbacks cb 0x56107ac05f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:50.729 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107acca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107acca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49604/0 (socket says 172.21.15.71:49604) 2024-09-05T12:44:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ab6000640 1 -- 172.21.15.71:0/4115619160 learned_addr learned my addr 172.21.15.71:0/4115619160 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.724+0000 7f6ab7400640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ab6000640 1 -- 172.21.15.71:0/4115619160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 msgr2=0x56107accad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ab6000640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ab6000640 1 -- 172.21.15.71:0/4115619160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 msgr2=0x56107acca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ab6000640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ab6000640 1 -- 172.21.15.71:0/4115619160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107aa07620 con 0x56107acc2b00 2024-09-05T12:44:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ab6000640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107acca840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8010810 tx=0x7f6aa8010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b26b6ca88bbc9b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:44:50.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6aa4c00640 1 -- 172.21.15.71:0/4115619160 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa801d060 con 0x56107acc2b00 2024-09-05T12:44:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6aa4c00640 1 -- 172.21.15.71:0/4115619160 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f6aa800eda0 con 0x56107acc2b00 2024-09-05T12:44:50.733 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aa00f2e50 con 0x56107acc2b00 2024-09-05T12:44:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.729+0000 7f6aa4c00640 1 -- 172.21.15.71:0/4115619160 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f6aa8018430 con 0x56107acc2b00 2024-09-05T12:44:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.734+0000 7f6ac0dce740 10 client.27317 fetch_fsmap learned FSMap version 73 2024-09-05T12:44:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.734+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x56107acdb8e0 con 0x56107acc2b00 2024-09-05T12:44:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6aa4c00640 1 -- 172.21.15.71:0/4115619160 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa00f2e50 con 0x56107acc2b00 2024-09-05T12:44:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6aa4c00640 1 -- 172.21.15.71:0/4115619160 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 73) ==== 8382+0+0 (secure 0 0 0) 0x7f6aa800d560 con 0x56107acc2b00 2024-09-05T12:44:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 10 client.27317 fetch_fsmap finished waiting for FSMap version 73 2024-09-05T12:44:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 10 client.27317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 20 client.27317 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107acdcda0 0x56107acdf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 4 client.27317 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:44:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56107ac35830 con 0x56107acdcda0 2024-09-05T12:44:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107acdcda0 0x56107acdf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107acdcda0 0x56107acdf190 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7228c3bf5bf55133 server_cookie=10e8d59c089eb1b in_seq=0 out_seq=0 2024-09-05T12:44:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6aa4c00640 10 client.27317 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6aa4c00640 1 -- 172.21.15.71:0/4115619160 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56107ac35830 con 0x56107acdcda0 2024-09-05T12:44:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6aa4c00640 10 client.27317 handle_command_reply: tid=0 2024-09-05T12:44:50.741 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 client.27317 shutdown 2024-09-05T12:44:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 msgr2=0x56107acca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107acca840 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8010810 tx=0x7f6aa8010cd0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 shutdown_connections 2024-09-05T12:44:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107acdcda0 0x56107acdf190 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.742 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107accd3a0 0x56107accad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6aa4c00640 0 client.27317 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc2b00 0x56107acca840 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/4115619160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acc21e0 0x56107acca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.735+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 >> 172.21.15.71:0/4115619160 conn(0x56107acbe750 msgr2=0x56107acc44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.741+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 shutdown_connections 2024-09-05T12:44:50.743 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.741+0000 7f6ac0dce740 1 -- 172.21.15.71:0/4115619160 wait complete. 2024-09-05T12:44:50.744 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.741+0000 7f6ac0dce740 20 client.27317 trim_cache size 0 max 16384 2024-09-05T12:44:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 Processor -- start 2024-09-05T12:44:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 -- start start 2024-09-05T12:44:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc21e0 0x56107acc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 0x56107ace4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acdcda0 0x56107ace4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b5b00 con 0x56107accd3a0 2024-09-05T12:44:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b76a0 con 0x56107acc21e0 2024-09-05T12:44:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab7400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acdcda0 0x56107ace4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ac0dce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b6200 con 0x56107acdcda0 2024-09-05T12:44:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 0x56107ace4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 0x56107ace4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:58482/0 (socket says 172.21.15.71:58482) 2024-09-05T12:44:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 -- 172.21.15.71:0/262602689 learned_addr learned my addr 172.21.15.71:0/262602689 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 -- 172.21.15.71:0/262602689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acdcda0 msgr2=0x56107ace4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acdcda0 0x56107ace4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 -- 172.21.15.71:0/262602689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc21e0 msgr2=0x56107acc43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc21e0 0x56107acc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc21e0 0x56107acc43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.795 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.788+0000 7f6ab6000640 1 -- 172.21.15.71:0/262602689 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56107acca0d0 con 0x56107accd3a0 2024-09-05T12:44:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.793+0000 7f6ab7400640 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acdcda0 0x56107ace4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.793+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc21e0 0x56107acc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:44:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.793+0000 7f6ab6000640 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 0x56107ace4a60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8013040 tx=0x7f6aa80158a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65287301708cb2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.793+0000 7f6aa7400640 1 -- 172.21.15.71:0/262602689 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8027080 con 0x56107accd3a0 2024-09-05T12:44:50.796 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6aa7400640 1 -- 172.21.15.71:0/262602689 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aa800d3a0 con 0x56107accd3a0 2024-09-05T12:44:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/262602689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 msgr2=0x56107ace4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6aa7400640 1 -- 172.21.15.71:0/262602689 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8027080 con 0x56107accd3a0 2024-09-05T12:44:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 0x56107ace4a60 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6aa8013040 tx=0x7f6aa80158a0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/262602689 shutdown_connections 2024-09-05T12:44:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acdcda0 0x56107ace4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107accd3a0 0x56107ace4a60 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/262602689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107acc21e0 0x56107acc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/262602689 >> 172.21.15.71:0/262602689 conn(0x56107acbe750 msgr2=0x56107acc3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/262602689 shutdown_connections 2024-09-05T12:44:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/262602689 wait complete. 2024-09-05T12:44:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- start start 2024-09-05T12:44:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 Processor -- start 2024-09-05T12:44:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acc21e0 0x56107acc7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 0x56107acc78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acc21e0 0x56107acc7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 0x56107acc78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 0x56107acc78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49622/0 (socket says 172.21.15.71:49622) 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 -- 172.21.15.71:0/1306223942 learned_addr learned my addr 172.21.15.71:0/1306223942 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acdcda0 0x56107acc7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b6fd0 con 0x56107acdcda0 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b68d0 con 0x56107accd3a0 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab81b3ac0 con 0x56107acc21e0 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ac0dce740 10 client.0 _ll_register_callbacks cb 0x7f6aa800b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 -- 172.21.15.71:0/1306223942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acc21e0 msgr2=0x56107acc7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acc21e0 0x56107acc7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 -- 172.21.15.71:0/1306223942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acdcda0 msgr2=0x56107acc7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:44:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acdcda0 0x56107acc7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.795+0000 7f6ab6000640 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56107ad46a80 con 0x56107accd3a0 2024-09-05T12:44:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ab6000640 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 0x56107acc78c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x56107acdb880 tx=0x7f6aa8017e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71c987120460fe2a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:44:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:44:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 1 -- 172.21.15.71:0/1306223942 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8027070 con 0x56107accd3a0 2024-09-05T12:44:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 1 -- 172.21.15.71:0/1306223942 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7f6aa80092a0 con 0x56107accd3a0 2024-09-05T12:44:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56107ac2fb80 con 0x56107accd3a0 2024-09-05T12:44:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 1 -- 172.21.15.71:0/1306223942 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f6aa803b050 con 0x56107accd3a0 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 10 client.27320 fetch_fsmap learned FSMap version 73 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x56107ad47ca0 con 0x56107accd3a0 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 1 -- 172.21.15.71:0/1306223942 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56107ac2fb80 con 0x56107accd3a0 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 1 -- 172.21.15.71:0/1306223942 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 73) ==== 8382+0+0 (secure 0 0 0) 0x7f6aa800d560 con 0x56107accd3a0 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 10 client.27320 fetch_fsmap finished waiting for FSMap version 73 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 10 client.27320 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 20 client.27320 populate_metadata read hostname 'smithi071' 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107ad48380 0x56107ad4a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:44:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 4 client.27320 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:44:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56107abac7c0 con 0x56107ad48380 2024-09-05T12:44:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107ad48380 0x56107ad4a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:44:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6aa4c00640 10 client.27320 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:50.800+0000 7f6ab6a00640 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107ad48380 0x56107ad4a7a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e172b68edea9e0d server_cookie=7870b7857e8020e0 in_seq=0 out_seq=0 2024-09-05T12:44:50.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75845 18.01 MB/sec execute 104 sec latency 6472.829 ms 2024-09-05T12:44:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:50 smithi123 ceph-mon[34321]: pgmap v538: 129 pgs: 129 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-09-05T12:44:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:50 smithi174 ceph-mon[34313]: pgmap v538: 129 pgs: 129 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-09-05T12:44:51.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75845 17.83 MB/sec execute 105 sec latency 7472.886 ms 2024-09-05T12:44:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6aa4c00640 1 -- 172.21.15.71:0/1306223942 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56107abac7c0 con 0x56107ad48380 2024-09-05T12:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6aa4c00640 10 client.27320 handle_command_reply: tid=0 2024-09-05T12:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 client.27320 shutdown 2024-09-05T12:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 msgr2=0x56107acc78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 0x56107acc78c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x56107acdb880 tx=0x7f6aa8017e90 comp rx=0 tx=0).stop 2024-09-05T12:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:44:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 shutdown_connections 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56107ad48380 0x56107ad4a7a0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56107acdcda0 0x56107acc7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56107accd3a0 0x56107acc78c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 --2- 172.21.15.71:0/1306223942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56107acc21e0 0x56107acc7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 >> 172.21.15.71:0/1306223942 conn(0x56107acbe750 msgr2=0x56107acc3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6aa4c00640 0 client.27320 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 shutdown_connections 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 1 -- 172.21.15.71:0/1306223942 wait complete. 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6ac0dce740 20 client.27320 trim_cache size 0 max 16384 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a9c0a2ef0 msgr2=0x7f6a9c0a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a9c0a2ef0 0x7f6a9c0a53b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0007db0 tx=0x7f6ab0007340 comp rx=0 tx=0).stop 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 msgr2=0x7f6ab810af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 0x7f6ab810af20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6aa002aa90 tx=0x7f6aa0003cf0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 shutdown_connections 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ab810b9a0 0x7f6ab81aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a9c0a2ef0 0x7f6a9c0a53b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6ab810e320 0x7f6ab810b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 --2- 172.21.15.71:0/3282935959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8074f40 0x7f6ab810af20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:44:52.246 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 >> 172.21.15.71:0/3282935959 conn(0x7f6ab80fd7e0 msgr2=0x7f6ab80fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:44:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 shutdown_connections 2024-09-05T12:44:52.247 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:44:52.240+0000 7f6abde00640 1 -- 172.21.15.71:0/3282935959 wait complete. 2024-09-05T12:44:52.629 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:52 smithi071 ceph-mon[29845]: pgmap v539: 129 pgs: 129 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 15 MiB/s wr, 778 op/s 2024-09-05T12:44:52.629 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:52 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:52.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75846 17.67 MB/sec execute 106 sec latency 8245.457 ms 2024-09-05T12:44:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:52 smithi123 ceph-mon[34321]: pgmap v539: 129 pgs: 129 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 15 MiB/s wr, 778 op/s 2024-09-05T12:44:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:52 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:52 smithi174 ceph-mon[34313]: pgmap v539: 129 pgs: 129 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 15 MiB/s wr, 778 op/s 2024-09-05T12:44:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:52 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:44:53.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75846 17.50 MB/sec execute 107 sec latency 9245.523 ms 2024-09-05T12:44:54.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 17.34 MB/sec execute 108 sec latency 10245.566 ms 2024-09-05T12:44:54.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:54 smithi071 ceph-mon[29845]: pgmap v540: 129 pgs: 129 active+clean; 861 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 528 op/s 2024-09-05T12:44:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:54 smithi123 ceph-mon[34321]: pgmap v540: 129 pgs: 129 active+clean; 861 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 528 op/s 2024-09-05T12:44:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:54 smithi174 ceph-mon[34313]: pgmap v540: 129 pgs: 129 active+clean; 861 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 13 MiB/s wr, 528 op/s 2024-09-05T12:44:55.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 17.18 MB/sec execute 109 sec latency 11245.654 ms 2024-09-05T12:44:56.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 17.02 MB/sec execute 110 sec latency 12245.720 ms 2024-09-05T12:44:56.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:56 smithi071 ceph-mon[29845]: pgmap v541: 129 pgs: 129 active+clean; 861 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 458 op/s 2024-09-05T12:44:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:56 smithi123 ceph-mon[34321]: pgmap v541: 129 pgs: 129 active+clean; 861 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 458 op/s 2024-09-05T12:44:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:56 smithi174 ceph-mon[34313]: pgmap v541: 129 pgs: 129 active+clean; 861 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 13 MiB/s wr, 458 op/s 2024-09-05T12:44:57.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 16.87 MB/sec execute 111 sec latency 13245.769 ms 2024-09-05T12:44:58.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 16.72 MB/sec execute 112 sec latency 14245.842 ms 2024-09-05T12:44:58.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:44:58 smithi071 ceph-mon[29845]: pgmap v542: 129 pgs: 129 active+clean; 837 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 620 op/s 2024-09-05T12:44:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:44:58 smithi123 ceph-mon[34321]: pgmap v542: 129 pgs: 129 active+clean; 837 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 620 op/s 2024-09-05T12:44:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:44:58 smithi174 ceph-mon[34313]: pgmap v542: 129 pgs: 129 active+clean; 837 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 15 MiB/s wr, 620 op/s 2024-09-05T12:44:59.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 16.57 MB/sec execute 113 sec latency 15245.950 ms 2024-09-05T12:45:00.768 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:00 smithi071 ceph-mon[29845]: pgmap v543: 129 pgs: 129 active+clean; 837 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 7.8 MiB/s wr, 339 op/s 2024-09-05T12:45:00.810 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75847 16.43 MB/sec execute 114 sec latency 16245.990 ms 2024-09-05T12:45:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:00 smithi123 ceph-mon[34321]: pgmap v543: 129 pgs: 129 active+clean; 837 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 7.8 MiB/s wr, 339 op/s 2024-09-05T12:45:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:00 smithi174 ceph-mon[34313]: pgmap v543: 129 pgs: 129 active+clean; 837 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 7.8 MiB/s wr, 339 op/s 2024-09-05T12:45:01.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75848 16.28 MB/sec execute 115 sec latency 17246.621 ms 2024-09-05T12:45:02.730 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:02 smithi071 ceph-mon[29845]: pgmap v544: 129 pgs: 129 active+clean; 811 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 8.0 MiB/s wr, 381 op/s 2024-09-05T12:45:02.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75848 16.14 MB/sec execute 116 sec latency 18246.659 ms 2024-09-05T12:45:02.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:02 smithi123 ceph-mon[34321]: pgmap v544: 129 pgs: 129 active+clean; 811 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 8.0 MiB/s wr, 381 op/s 2024-09-05T12:45:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:02 smithi174 ceph-mon[34313]: pgmap v544: 129 pgs: 129 active+clean; 811 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 812 KiB/s rd, 8.0 MiB/s wr, 381 op/s 2024-09-05T12:45:03.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75848 16.01 MB/sec execute 117 sec latency 19246.724 ms 2024-09-05T12:45:04.710 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:04 smithi071 ceph-mon[29845]: pgmap v545: 129 pgs: 129 active+clean; 801 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-09-05T12:45:04.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75848 15.87 MB/sec execute 118 sec latency 20246.797 ms 2024-09-05T12:45:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:04 smithi123 ceph-mon[34321]: pgmap v545: 129 pgs: 129 active+clean; 801 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-09-05T12:45:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:04 smithi174 ceph-mon[34313]: pgmap v545: 129 pgs: 129 active+clean; 801 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 880 KiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-09-05T12:45:05.363 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 -- 172.21.15.71:0/4058414134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc94103f50 msgr2=0x7fcc94104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 --2- 172.21.15.71:0/4058414134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc94103f50 0x7fcc94104330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009900 tx=0x7fcc8802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 -- 172.21.15.71:0/4058414134 shutdown_connections 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 --2- 172.21.15.71:0/4058414134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc941090a0 0x7fcc94110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 --2- 172.21.15.71:0/4058414134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc94104900 0x7fcc94108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 --2- 172.21.15.71:0/4058414134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc94103f50 0x7fcc94104330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 -- 172.21.15.71:0/4058414134 >> 172.21.15.71:0/4058414134 conn(0x7fcc940ff9e0 msgr2=0x7fcc94101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 -- 172.21.15.71:0/4058414134 shutdown_connections 2024-09-05T12:45:05.364 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.362+0000 7fcc9c200640 1 -- 172.21.15.71:0/4058414134 wait complete. 2024-09-05T12:45:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.363+0000 7fcc9c200640 1 Processor -- start 2024-09-05T12:45:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.363+0000 7fcc9c200640 1 -- start start 2024-09-05T12:45:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9c200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc94103f50 0x7fcc9419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9c200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 0x7fcc9419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9c200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc9419ee80 0x7fcc9419f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9c200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941140f0 con 0x7fcc941090a0 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9c200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94113f70 con 0x7fcc9419ee80 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9c200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc94114270 con 0x7fcc94103f50 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 0x7fcc9419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 0x7fcc9419ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43968/0 (socket says 172.21.15.71:43968) 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 -- 172.21.15.71:0/2694410990 learned_addr learned my addr 172.21.15.71:0/2694410990 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc9a200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc9419ee80 0x7fcc9419f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 -- 172.21.15.71:0/2694410990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc94103f50 msgr2=0x7fcc9419a5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc94103f50 0x7fcc9419a5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 -- 172.21.15.71:0/2694410990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc9419ee80 msgr2=0x7fcc9419f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc9419ee80 0x7fcc9419f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.364+0000 7fcc98e00640 1 -- 172.21.15.71:0/2694410990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9419f970 con 0x7fcc941090a0 2024-09-05T12:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.365+0000 7fcc98e00640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 0x7fcc9419ab00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000e9f0 tx=0x7fcc8000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.365+0000 7fcc92000640 1 -- 172.21.15.71:0/2694410990 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8000ccc0 con 0x7fcc941090a0 2024-09-05T12:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.365+0000 7fcc92000640 1 -- 172.21.15.71:0/2694410990 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc80005500 con 0x7fcc941090a0 2024-09-05T12:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.365+0000 7fcc92000640 1 -- 172.21.15.71:0/2694410990 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80010690 con 0x7fcc941090a0 2024-09-05T12:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.365+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9419fc60 con 0x7fcc941090a0 2024-09-05T12:45:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.365+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc941a7400 con 0x7fcc941090a0 2024-09-05T12:45:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 Processor -- start 2024-09-05T12:45:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 -- start start 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x5581368895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136893e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x558136896790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5581367ef100 con 0x558136889b00 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581367971b0 con 0x5581368943a0 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.369+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581367eb0c0 con 0x5581368891e0 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136893e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc9a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x5581368895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136893e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43980/0 (socket says 172.21.15.71:43980) 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc9a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x5581368895c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52306/0 (socket says 172.21.15.71:52306) 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 -- 172.21.15.71:0/3592313192 learned_addr learned my addr 172.21.15.71:0/3592313192 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc92000640 1 -- 172.21.15.71:0/2694410990 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fcc800046c0 con 0x7fcc941090a0 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 -- 172.21.15.71:0/3592313192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 msgr2=0x5581368895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x5581368895c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 -- 172.21.15.71:0/3592313192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 msgr2=0x558136896790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x558136896790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.370+0000 7fcc99800640 1 -- 172.21.15.71:0/3592313192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558136502210 con 0x558136889b00 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.371+0000 7fcc99800640 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136893e60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802a910 tx=0x7fcc88031510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce34fef739cd7c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.371+0000 7fcc86a00640 1 -- 172.21.15.71:0/3592313192 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8803e070 con 0x558136889b00 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.371+0000 7fcc86a00640 1 -- 172.21.15.71:0/3592313192 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc88031870 con 0x558136889b00 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.371+0000 7fcc86a00640 1 -- 172.21.15.71:0/3592313192 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88035040 con 0x558136889b00 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.371+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/3592313192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 msgr2=0x558136893e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.371+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136893e60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcc8802a910 tx=0x7fcc88031510 comp rx=0 tx=0).stop 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/3592313192 shutdown_connections 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x558136896790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136893e60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/3592313192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x5581368895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/3592313192 >> 172.21.15.71:0/3592313192 conn(0x558136885750 msgr2=0x558136885b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/3592313192 shutdown_connections 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/3592313192 wait complete. 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc92000640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcc7c0a2cb0 0x7fcc7c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 -- start start 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc92000640 1 -- 172.21.15.71:0/2694410990 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7fcc800cfd20 con 0x7fcc941090a0 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9a200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcc7c0a2cb0 0x7fcc7c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.372+0000 7fcc9f1c1740 1 Processor -- start 2024-09-05T12:45:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x558136891380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558136889b00 0x5581368918c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368943a0 0x558136891e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558136896f70 con 0x5581368943a0 2024-09-05T12:45:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558136778090 con 0x558136889b00 2024-09-05T12:45:05.377 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581368970f0 con 0x5581368891e0 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9f1c1740 10 client.0 _ll_register_callbacks cb 0x5581367ccf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc99800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x558136891380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9a200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcc7c0a2cb0 0x7fcc7c0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005fd0 tx=0x7fcc8c009760 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc99800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x558136891380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:52320/0 (socket says 172.21.15.71:52320) 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc99800640 1 -- 172.21.15.71:0/1239937741 learned_addr learned my addr 172.21.15.71:0/1239937741 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc9a200640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368943a0 0x558136891e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.373+0000 7fcc98e00640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558136889b00 0x5581368918c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.374+0000 7fcc99800640 1 -- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558136889b00 msgr2=0x5581368918c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.374+0000 7fcc99800640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558136889b00 0x5581368918c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.374+0000 7fcc99800640 1 -- 172.21.15.71:0/1239937741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368943a0 msgr2=0x558136891e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.374+0000 7fcc99800640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368943a0 0x558136891e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.378 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.374+0000 7fcc99800640 1 -- 172.21.15.71:0/1239937741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581365ce620 con 0x5581368891e0 2024-09-05T12:45:05.381 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.380+0000 7fcc99800640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x558136891380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcc880096c0 tx=0x7fcc88005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31986f96572afb2d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc7b400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc7b400640 1 -- 172.21.15.71:0/1239937741 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8803e070 con 0x5581368891e0 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc7b400640 1 -- 172.21.15.71:0/1239937741 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7fcc88004250 con 0x5581368891e0 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc88031870 con 0x5581368891e0 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc7b400640 1 -- 172.21.15.71:0/1239937741 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fcc88055050 con 0x5581368891e0 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc9f1c1740 10 client.27382 fetch_fsmap learned FSMap version 74 2024-09-05T12:45:05.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x558136896f70 con 0x5581368891e0 2024-09-05T12:45:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.381+0000 7fcc7b400640 1 -- 172.21.15.71:0/1239937741 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc88031870 con 0x5581368891e0 2024-09-05T12:45:05.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc7b400640 1 -- 172.21.15.71:0/1239937741 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 74) ==== 8378+0+0 (secure 0 0 0) 0x7fcc880038c0 con 0x5581368891e0 2024-09-05T12:45:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc9f1c1740 10 client.27382 fetch_fsmap finished waiting for FSMap version 74 2024-09-05T12:45:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc9f1c1740 10 client.27382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc9f1c1740 20 client.27382 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5581368a3d80 0x5581368a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc9f1c1740 4 client.27382 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:45:05.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.382+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581367fc830 con 0x5581368a3d80 2024-09-05T12:45:05.428 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.427+0000 7fcc98e00640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5581368a3d80 0x5581368a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.545+0000 7fcc98e00640 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5581368a3d80 0x5581368a6170 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7318debeffbcb5 server_cookie=e0144fb1aaaa95cb in_seq=0 out_seq=0 2024-09-05T12:45:05.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.545+0000 7fcc7b400640 10 client.27382 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.569+0000 7fcc7b400640 1 -- 172.21.15.71:0/1239937741 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581367fc830 con 0x5581368a3d80 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.569+0000 7fcc7b400640 10 client.27382 handle_command_reply: tid=0 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.569+0000 7fcc9f1c1740 1 client.27382 shutdown 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 msgr2=0x558136891380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x558136891380 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fcc880096c0 tx=0x7fcc88005280 comp rx=0 tx=0).stop 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 shutdown_connections 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5581368a3d80 0x5581368a6170 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368943a0 0x558136891e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558136889b00 0x5581368918c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/1239937741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368891e0 0x558136891380 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc7b400640 0 client.27382 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 >> 172.21.15.71:0/1239937741 conn(0x558136885750 msgr2=0x55813688b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 shutdown_connections 2024-09-05T12:45:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/1239937741 wait complete. 2024-09-05T12:45:05.572 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.570+0000 7fcc9f1c1740 20 client.27382 trim_cache size 0 max 16384 2024-09-05T12:45:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 Processor -- start 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 -- start start 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 0x55813688b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558136889b00 0x5581368ab950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x5581368abe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b1020 con 0x5581368891e0 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b2b90 con 0x5581368943a0 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.581+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b16f0 con 0x558136889b00 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 0x55813688b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 0x55813688b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43998/0 (socket says 172.21.15.71:43998) 2024-09-05T12:45:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 -- 172.21.15.71:0/49973446 learned_addr learned my addr 172.21.15.71:0/49973446 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9a200640 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x5581368abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 -- 172.21.15.71:0/49973446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558136889b00 msgr2=0x5581368ab950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558136889b00 0x5581368ab950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 -- 172.21.15.71:0/49973446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 msgr2=0x5581368abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x5581368abe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 -- 172.21.15.71:0/49973446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558136893a00 con 0x5581368891e0 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 0x55813688b6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005570 tx=0x7fcc88031490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888731ab25813404 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc86000640 1 -- 172.21.15.71:0/49973446 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8803e070 con 0x5581368891e0 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc86000640 1 -- 172.21.15.71:0/49973446 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc8804c070 con 0x5581368891e0 2024-09-05T12:45:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc86000640 1 -- 172.21.15.71:0/49973446 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc88035040 con 0x5581368891e0 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/49973446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 msgr2=0x55813688b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 0x55813688b6d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005570 tx=0x7fcc88031490 comp rx=0 tx=0).stop 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/49973446 shutdown_connections 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368943a0 0x5581368abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558136889b00 0x5581368ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/49973446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581368891e0 0x55813688b6d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/49973446 >> 172.21.15.71:0/49973446 conn(0x558136885750 msgr2=0x5581368932f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/49973446 shutdown_connections 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/49973446 wait complete. 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- start start 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 Processor -- start 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368891e0 0x55813689c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136897a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368943a0 0x558136897fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b24c0 con 0x558136889b00 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941b1dc0 con 0x5581368891e0 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc941af020 con 0x5581368943a0 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 10 client.0 _ll_register_callbacks cb 0x7fcc8c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136897a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.594 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136897a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44002/0 (socket says 172.21.15.71:44002) 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 -- 172.21.15.71:0/2598396960 learned_addr learned my addr 172.21.15.71:0/2598396960 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc99800640 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368891e0 0x55813689c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 -- 172.21.15.71:0/2598396960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368943a0 msgr2=0x558136897fa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368943a0 0x558136897fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 -- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368891e0 msgr2=0x55813689c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368891e0 0x55813689c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 -- 172.21.15.71:0/2598396960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581368984e0 con 0x558136889b00 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc98e00640 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136897a60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000cfd0 tx=0x7fcc80005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683bef4cccc904c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc7b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc7b400640 1 -- 172.21.15.71:0/2598396960 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800a0200 con 0x558136889b00 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc7b400640 1 -- 172.21.15.71:0/2598396960 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9438+0+0 (secure 0 0 0) 0x7fcc800984c0 con 0x558136889b00 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8804c070 con 0x558136889b00 2024-09-05T12:45:05.595 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc7b400640 1 -- 172.21.15.71:0/2598396960 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fcc800a4ac0 con 0x558136889b00 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 10 client.17328 fetch_fsmap learned FSMap version 74 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55813690e640 con 0x558136889b00 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc7b400640 1 -- 172.21.15.71:0/2598396960 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc8804c070 con 0x558136889b00 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc7b400640 1 -- 172.21.15.71:0/2598396960 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 74) ==== 8378+0+0 (secure 0 0 0) 0x7fcc8009b2a0 con 0x558136889b00 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 10 client.17328 fetch_fsmap finished waiting for FSMap version 74 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.586+0000 7fcc9f1c1740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc9f1c1740 20 client.17328 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55813690edf0 0x558136911210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc9f1c1740 4 client.17328 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581367f6b80 con 0x55813690edf0 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc99800640 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55813690edf0 0x558136911210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc99800640 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55813690edf0 0x558136911210 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e94bdeeb45c4e3 server_cookie=e4b51f484b609f8b in_seq=0 out_seq=0 2024-09-05T12:45:05.596 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.591+0000 7fcc7b400640 10 client.17328 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc7b400640 1 -- 172.21.15.71:0/2598396960 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581367f6b80 con 0x55813690edf0 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc7b400640 10 client.17328 handle_command_reply: tid=0 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 client.17328 shutdown 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 msgr2=0x558136897a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136897a60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000cfd0 tx=0x7fcc80005770 comp rx=0 tx=0).stop 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 shutdown_connections 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55813690edf0 0x558136911210 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581368943a0 0x558136897fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558136889b00 0x558136897a60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 --2- 172.21.15.71:0/2598396960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581368891e0 0x55813689c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 >> 172.21.15.71:0/2598396960 conn(0x558136885750 msgr2=0x558136884830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc7b400640 0 client.17328 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 shutdown_connections 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.651+0000 7fcc9f1c1740 1 -- 172.21.15.71:0/2598396960 wait complete. 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9f1c1740 20 client.17328 trim_cache size 0 max 16384 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:45:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcc7c0a2cb0 msgr2=0x7fcc7c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcc7c0a2cb0 0x7fcc7c0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c005fd0 tx=0x7fcc8c009760 comp rx=0 tx=0).stop 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 msgr2=0x7fcc9419ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 0x7fcc9419ab00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000e9f0 tx=0x7fcc8000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 shutdown_connections 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc9419ee80 0x7fcc9419f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fcc7c0a2cb0 0x7fcc7c0a5170 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc941090a0 0x7fcc9419ab00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 --2- 172.21.15.71:0/2694410990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc94103f50 0x7fcc9419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 >> 172.21.15.71:0/2694410990 conn(0x7fcc940ff9e0 msgr2=0x7fcc94100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 shutdown_connections 2024-09-05T12:45:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:05.657+0000 7fcc9c200640 1 -- 172.21.15.71:0/2694410990 wait complete. 2024-09-05T12:45:05.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75848 15.74 MB/sec execute 119 sec latency 21246.845 ms 2024-09-05T12:45:06.692 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:06 smithi071 ceph-mon[29845]: pgmap v546: 129 pgs: 129 active+clean; 801 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-09-05T12:45:06.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75850 15.61 MB/sec execute 120 sec latency 22246.894 ms 2024-09-05T12:45:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:06 smithi123 ceph-mon[34321]: pgmap v546: 129 pgs: 129 active+clean; 801 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-09-05T12:45:07.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:06 smithi174 ceph-mon[34313]: pgmap v546: 129 pgs: 129 active+clean; 801 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 691 KiB/s rd, 2.8 MiB/s wr, 227 op/s 2024-09-05T12:45:07.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75851 15.48 MB/sec execute 121 sec latency 23246.959 ms 2024-09-05T12:45:08.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75851 15.35 MB/sec execute 122 sec latency 24246.998 ms 2024-09-05T12:45:08.961 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:08 smithi071 ceph-mon[29845]: pgmap v547: 129 pgs: 129 active+clean; 802 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-05T12:45:08.961 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:08 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:45:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:08 smithi174 ceph-mon[34313]: pgmap v547: 129 pgs: 129 active+clean; 802 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-05T12:45:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:08 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:45:09.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:08 smithi123 ceph-mon[34321]: pgmap v547: 129 pgs: 129 active+clean; 802 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 KiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-09-05T12:45:09.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:08 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:45:09.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75851 15.22 MB/sec execute 123 sec latency 25247.043 ms 2024-09-05T12:45:10.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75851 15.10 MB/sec execute 124 sec latency 26247.091 ms 2024-09-05T12:45:10.948 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:10 smithi071 ceph-mon[29845]: pgmap v548: 129 pgs: 129 active+clean; 802 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 617 KiB/s wr, 87 op/s 2024-09-05T12:45:11.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:10 smithi174 ceph-mon[34313]: pgmap v548: 129 pgs: 129 active+clean; 802 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 617 KiB/s wr, 87 op/s 2024-09-05T12:45:11.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:10 smithi123 ceph-mon[34321]: pgmap v548: 129 pgs: 129 active+clean; 802 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 617 KiB/s wr, 87 op/s 2024-09-05T12:45:11.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75851 14.98 MB/sec execute 125 sec latency 27247.144 ms 2024-09-05T12:45:12.811 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75851 14.86 MB/sec execute 126 sec latency 28247.187 ms 2024-09-05T12:45:12.813 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:12 smithi071 ceph-mon[29845]: pgmap v549: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 662 KiB/s wr, 120 op/s 2024-09-05T12:45:13.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:12 smithi174 ceph-mon[34313]: pgmap v549: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 662 KiB/s wr, 120 op/s 2024-09-05T12:45:13.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:12 smithi123 ceph-mon[34321]: pgmap v549: 129 pgs: 129 active+clean; 784 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 662 KiB/s wr, 120 op/s 2024-09-05T12:45:13.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 75852 14.75 MB/sec execute 127 sec latency 29247.230 ms 2024-09-05T12:45:14.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 76531 14.85 MB/sec execute 128 sec latency 29504.548 ms 2024-09-05T12:45:14.966 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:14 smithi071 ceph-mon[29845]: pgmap v550: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 521 KiB/s wr, 107 op/s 2024-09-05T12:45:15.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:14 smithi174 ceph-mon[34313]: pgmap v550: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 521 KiB/s wr, 107 op/s 2024-09-05T12:45:15.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:14 smithi123 ceph-mon[34321]: pgmap v550: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 521 KiB/s wr, 107 op/s 2024-09-05T12:45:15.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 77624 15.24 MB/sec execute 129 sec latency 81.774 ms 2024-09-05T12:45:16.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 78566 15.50 MB/sec execute 130 sec latency 63.848 ms 2024-09-05T12:45:17.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:16 smithi174 ceph-mon[34313]: pgmap v551: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 279 KiB/s wr, 82 op/s 2024-09-05T12:45:17.070 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:16 smithi071 ceph-mon[29845]: pgmap v551: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 279 KiB/s wr, 82 op/s 2024-09-05T12:45:17.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:16 smithi123 ceph-mon[34321]: pgmap v551: 129 pgs: 129 active+clean; 774 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 279 KiB/s wr, 82 op/s 2024-09-05T12:45:17.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 79308 15.70 MB/sec execute 131 sec latency 95.305 ms 2024-09-05T12:45:18.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 79683 15.73 MB/sec execute 132 sec latency 160.150 ms 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 -- 172.21.15.71:0/3197603371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 msgr2=0x7f80a4107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 --2- 172.21.15.71:0/3197603371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a4107f50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8094009900 tx=0x7f809402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 -- 172.21.15.71:0/3197603371 shutdown_connections 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 --2- 172.21.15.71:0/3197603371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80a4102470 0x7f80a410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 --2- 172.21.15.71:0/3197603371 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 0x7f80a4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 --2- 172.21.15.71:0/3197603371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a4107f50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.854 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 -- 172.21.15.71:0/3197603371 >> 172.21.15.71:0/3197603371 conn(0x7f80a40fd7d0 msgr2=0x7f80a40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 -- 172.21.15.71:0/3197603371 shutdown_connections 2024-09-05T12:45:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.853+0000 7f80aa400640 1 -- 172.21.15.71:0/3197603371 wait complete. 2024-09-05T12:45:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.855+0000 7f80aa400640 1 Processor -- start 2024-09-05T12:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 -- start start 2024-09-05T12:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 0x7f80a41960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80a4102470 0x7f80a4196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f80a410fd80 con 0x7f80a4101ad0 2024-09-05T12:45:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80a410fc00 con 0x7f80a4107b70 2024-09-05T12:45:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.856+0000 7f80aa400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80a410ff00 con 0x7f80a4102470 2024-09-05T12:45:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.857+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.857+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33734/0 (socket says 172.21.15.71:33734) 2024-09-05T12:45:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.857+0000 7f80a3e00640 1 -- 172.21.15.71:0/1852736015 learned_addr learned my addr 172.21.15.71:0/1852736015 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.858+0000 7f80a3e00640 1 -- 172.21.15.71:0/1852736015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80a4102470 msgr2=0x7f80a4196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.858+0000 7f80a2a00640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80a4102470 0x7f80a4196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.858+0000 7f80a3400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 0x7f80a41960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.859+0000 7f80a3e00640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80a4102470 0x7f80a4196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.859+0000 7f80a3e00640 1 -- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 msgr2=0x7f80a41960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.859+0000 7f80a3e00640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 0x7f80a41960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.859+0000 7f80a3e00640 1 -- 172.21.15.71:0/1852736015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a419b090 con 0x7f80a4107b70 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.859+0000 7f80a3400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 0x7f80a41960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.859+0000 7f80a3e00640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a419a990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f809c00eb40 tx=0x7f809c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.860+0000 7f809be00640 1 -- 172.21.15.71:0/1852736015 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c004b20 con 0x7f80a4107b70 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.860+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a419b380 con 0x7f80a4107b70 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.860+0000 7f809be00640 1 -- 172.21.15.71:0/1852736015 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809c004cc0 con 0x7f80a4107b70 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.860+0000 7f809be00640 1 -- 172.21.15.71:0/1852736015 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c010760 con 0x7f80a4107b70 2024-09-05T12:45:18.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.860+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a406bc10 con 0x7f80a4107b70 2024-09-05T12:45:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.866+0000 7f809be00640 1 -- 172.21.15.71:0/1852736015 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f809c005020 con 0x7f80a4107b70 2024-09-05T12:45:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.868+0000 7f809be00640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f80880a2df0 0x7f80880a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.868+0000 7f809be00640 1 -- 172.21.15.71:0/1852736015 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9526+0+0 (secure 0 0 0) 0x7f809c0cfea0 con 0x7f80a4107b70 2024-09-05T12:45:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.871+0000 7f80a3400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f80880a2df0 0x7f80880a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.871+0000 7f80ad27d740 1 Processor -- start 2024-09-05T12:45:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.872+0000 7f80ad27d740 1 -- start start 2024-09-05T12:45:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.873+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 0x55a2dbe605c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.874 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.873+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.873+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.873+0000 7f80ad27d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a2dbdc6100 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.873+0000 7f80ad27d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a2dbd6e1b0 con 0x55a2dbe601e0 2024-09-05T12:45:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.873+0000 7f80ad27d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a2dbdc20c0 con 0x55a2dbe60b00 2024-09-05T12:45:18.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.874+0000 7f80a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 0x55a2dbe605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.874+0000 7f80a3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 0x55a2dbe605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33750/0 (socket says 172.21.15.71:33750) 2024-09-05T12:45:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.874+0000 7f80a3400640 1 -- 172.21.15.71:0/2598155253 learned_addr learned my addr 172.21.15.71:0/2598155253 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.874+0000 7f80a3400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f80880a2df0 0x7f80880a52b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f80940096c0 tx=0x7f8094000bc0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3400640 1 -- 172.21.15.71:0/2598155253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe60b00 msgr2=0x55a2dbe6ae60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:45:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3400640 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6ae60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3e00640 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3400640 1 -- 172.21.15.71:0/2598155253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe6b3a0 msgr2=0x55a2dbe6d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3400640 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3400640 1 -- 172.21.15.71:0/2598155253 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2dbad9210 con 0x55a2dbe601e0 2024-09-05T12:45:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3e00640 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:45:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.875+0000 7f80a3400640 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 0x55a2dbe605c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f80940043d0 tx=0x7f8094043000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3abbb8c489d0b5d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.877+0000 7f8098c00640 1 -- 172.21.15.71:0/2598155253 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094035050 con 0x55a2dbe601e0 2024-09-05T12:45:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.877+0000 7f8098c00640 1 -- 172.21.15.71:0/2598155253 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8094004ba0 con 0x55a2dbe601e0 2024-09-05T12:45:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.877+0000 7f8098c00640 1 -- 172.21.15.71:0/2598155253 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094032550 con 0x55a2dbe601e0 2024-09-05T12:45:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 -- 172.21.15.71:0/2598155253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 msgr2=0x55a2dbe605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 0x55a2dbe605c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f80940043d0 tx=0x7f8094043000 comp rx=0 tx=0).stop 2024-09-05T12:45:18.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 -- 172.21.15.71:0/2598155253 shutdown_connections 2024-09-05T12:45:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 --2- 172.21.15.71:0/2598155253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe601e0 0x55a2dbe605c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.878+0000 7f80ad27d740 1 -- 172.21.15.71:0/2598155253 >> 172.21.15.71:0/2598155253 conn(0x55a2dbe5c750 msgr2=0x55a2dbe5cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:18.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.879+0000 7f80ad27d740 1 -- 172.21.15.71:0/2598155253 shutdown_connections 2024-09-05T12:45:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.879+0000 7f80ad27d740 1 -- 172.21.15.71:0/2598155253 wait complete. 2024-09-05T12:45:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.879+0000 7f80ad27d740 1 -- start start 2024-09-05T12:45:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.880+0000 7f80ad27d740 1 Processor -- start 2024-09-05T12:45:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.881+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe601e0 0x55a2dbe68350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.881+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 0x55a2dbe68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.881+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe68dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.882+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe68dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.882+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe68dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33752/0 (socket says 172.21.15.71:33752) 2024-09-05T12:45:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.882+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 0x55a2dbe68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.882+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 0x55a2dbe68890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40578/0 (socket says 172.21.15.71:40578) 2024-09-05T12:45:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe601e0 0x55a2dbe68350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe601e0 0x55a2dbe68350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33660/0 (socket says 172.21.15.71:33660) 2024-09-05T12:45:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3e00640 1 -- 172.21.15.71:0/80244721 learned_addr learned my addr 172.21.15.71:0/80244721 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a2dbe6df70 con 0x55a2dbe60b00 2024-09-05T12:45:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a2dbd4f090 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a2dbe6e0f0 con 0x55a2dbe601e0 2024-09-05T12:45:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80ad27d740 10 client.0 _ll_register_callbacks cb 0x55a2dbda3f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3e00640 1 -- 172.21.15.71:0/80244721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe601e0 msgr2=0x55a2dbe68350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3e00640 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe601e0 0x55a2dbe68350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3e00640 1 -- 172.21.15.71:0/80244721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 msgr2=0x55a2dbe68890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3e00640 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 0x55a2dbe68890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.883+0000 7f80a3e00640 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2dbba5620 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8094004ba0 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f80a2a00640 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 0x55a2dbe68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:45:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f80a3e00640 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe68dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c7d0 tx=0x7f809c09c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ccdf1bed642b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f808d600640 10 client.27347 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:45:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f808d600640 1 -- 172.21.15.71:0/80244721 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c066050 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f808d600640 1 -- 172.21.15.71:0/80244721 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9526+0+0 (secure 0 0 0) 0x7f809c016df0 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.884+0000 7f808d600640 1 -- 172.21.15.71:0/80244721 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f809c0b4070 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.885+0000 7f80ad27d740 10 client.27347 fetch_fsmap learned FSMap version 75 2024-09-05T12:45:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.885+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=75}) -- 0x55a2dbe79ac0 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f808d600640 1 -- 172.21.15.71:0/80244721 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 75) ==== 8374+0+0 (secure 0 0 0) 0x7f809c098640 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80ad27d740 10 client.27347 fetch_fsmap finished waiting for FSMap version 75 2024-09-05T12:45:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80ad27d740 10 client.27347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80ad27d740 20 client.27347 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80ad27d740 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbe7ae70 0x55a2dbe7d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80ad27d740 4 client.27347 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2dbdd3830 con 0x55a2dbe7ae70 2024-09-05T12:45:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.886+0000 7f80a3400640 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbe7ae70 0x55a2dbe7d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.887+0000 7f80a3400640 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbe7ae70 0x55a2dbe7d260 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e94afe014fabb0b9 server_cookie=ab08d9b0f7f73ccb in_seq=0 out_seq=0 2024-09-05T12:45:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.887+0000 7f808d600640 10 client.27347 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.888+0000 7f808d600640 1 -- 172.21.15.71:0/80244721 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2dbdd3830 con 0x55a2dbe7ae70 2024-09-05T12:45:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.888+0000 7f808d600640 10 client.27347 handle_command_reply: tid=0 2024-09-05T12:45:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.889+0000 7f80ad27d740 1 client.27347 shutdown 2024-09-05T12:45:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.890+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 msgr2=0x55a2dbe68dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.890+0000 7f80ad27d740 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe68dd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f809c00c7d0 tx=0x7f809c09c790 comp rx=0 tx=0).stop 2024-09-05T12:45:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 shutdown_connections 2024-09-05T12:45:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f80ad27d740 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbe7ae70 0x55a2dbe7d260 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f80ad27d740 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe68dd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f80ad27d740 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe60b00 0x55a2dbe68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f80ad27d740 1 --2- 172.21.15.71:0/80244721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe601e0 0x55a2dbe68350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f808d600640 0 client.27347 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.891+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 >> 172.21.15.71:0/80244721 conn(0x55a2dbe5c750 msgr2=0x55a2dbe62460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.892+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 shutdown_connections 2024-09-05T12:45:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.892+0000 7f80ad27d740 1 -- 172.21.15.71:0/80244721 wait complete. 2024-09-05T12:45:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.892+0000 7f80ad27d740 20 client.27347 trim_cache size 0 max 16384 2024-09-05T12:45:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.927+0000 7f80ad27d740 1 Processor -- start 2024-09-05T12:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.928+0000 7f80ad27d740 1 -- start start 2024-09-05T12:45:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.928+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe82b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe83060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80ad27d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41b5630 con 0x55a2dbe601e0 2024-09-05T12:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80ad27d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41b7170 con 0x55a2dbe60b00 2024-09-05T12:45:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80ad27d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41b5d00 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe82b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe83060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80a3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe82b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33758/0 (socket says 172.21.15.71:33758) 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.929+0000 7f80a2a00640 1 -- 172.21.15.71:0/3242808171 learned_addr learned my addr 172.21.15.71:0/3242808171 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.930+0000 7f80a2a00640 1 -- 172.21.15.71:0/3242808171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 msgr2=0x55a2dbe83060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.930+0000 7f80a2a00640 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe83060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.930+0000 7f80a2a00640 1 -- 172.21.15.71:0/3242808171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 msgr2=0x55a2dbe6a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.930+0000 7f80a2a00640 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.930+0000 7f80a2a00640 1 -- 172.21.15.71:0/3242808171 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2dbe771d0 con 0x55a2dbe60b00 2024-09-05T12:45:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.931+0000 7f80a2a00640 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe82b20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f809000dcc0 tx=0x7f8090017c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b917abf81ddaaa62 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.931+0000 7f808fe00640 1 -- 172.21.15.71:0/3242808171 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809000ca20 con 0x55a2dbe60b00 2024-09-05T12:45:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.931+0000 7f808fe00640 1 -- 172.21.15.71:0/3242808171 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809000cbc0 con 0x55a2dbe60b00 2024-09-05T12:45:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.931+0000 7f808fe00640 1 -- 172.21.15.71:0/3242808171 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809001f390 con 0x55a2dbe60b00 2024-09-05T12:45:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.932+0000 7f80ad27d740 1 -- 172.21.15.71:0/3242808171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 msgr2=0x55a2dbe82b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:18.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.932+0000 7f80ad27d740 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe82b20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f809000dcc0 tx=0x7f8090017c30 comp rx=0 tx=0).stop 2024-09-05T12:45:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.934+0000 7f80ad27d740 1 -- 172.21.15.71:0/3242808171 shutdown_connections 2024-09-05T12:45:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.934+0000 7f80ad27d740 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe83060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.934+0000 7f80ad27d740 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe82b20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.934+0000 7f80ad27d740 1 --2- 172.21.15.71:0/3242808171 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.934+0000 7f80ad27d740 1 -- 172.21.15.71:0/3242808171 >> 172.21.15.71:0/3242808171 conn(0x55a2dbe5c750 msgr2=0x55a2dbe69aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.936+0000 7f80ad27d740 1 -- 172.21.15.71:0/3242808171 shutdown_connections 2024-09-05T12:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.936+0000 7f80ad27d740 1 -- 172.21.15.71:0/3242808171 wait complete. 2024-09-05T12:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.936+0000 7f80ad27d740 1 -- start start 2024-09-05T12:45:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.937+0000 7f80ad27d740 1 Processor -- start 2024-09-05T12:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.937+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80ad27d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80a2a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:33760/0 (socket says 172.21.15.71:33760) 2024-09-05T12:45:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80ad27d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41b6aa0 con 0x55a2dbe601e0 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80ad27d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80a41b63d0 con 0x55a2dbe60b00 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80ad27d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80a4103210 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80ad27d740 10 client.0 _ll_register_callbacks cb 0x7f809c004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80a3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:33690/0 (socket says 172.21.15.71:33690) 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.938+0000 7f80a3e00640 1 -- 172.21.15.71:0/1284894107 learned_addr learned my addr 172.21.15.71:0/1284894107 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a3400640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a3e00640 1 -- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 msgr2=0x55a2dbe6f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a3e00640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a3e00640 1 -- 172.21.15.71:0/1284894107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 msgr2=0x55a2dbe6f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a3e00640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a3e00640 1 -- 172.21.15.71:0/1284894107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2dbee4d40 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.939+0000 7f80a2a00640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.940+0000 7f80a3e00640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6fd70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f809c009710 tx=0x7f809c00cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f1f1735a7de1ba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.940+0000 7f808d600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.941+0000 7f808d600640 1 -- 172.21.15.71:0/1284894107 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809c066040 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.941+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809000cbc0 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.941+0000 7f808d600640 1 -- 172.21.15.71:0/1284894107 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9526+0+0 (secure 0 0 0) 0x7f809c0a4910 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.941+0000 7f808d600640 1 -- 172.21.15.71:0/1284894107 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f809c0a4c40 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.941+0000 7f80ad27d740 10 client.27394 fetch_fsmap learned FSMap version 75 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.941+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x55a2dbee6050 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f808d600640 1 -- 172.21.15.71:0/1284894107 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809000cbc0 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f808d600640 1 -- 172.21.15.71:0/1284894107 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 75) ==== 8374+0+0 (secure 0 0 0) 0x7f809c081070 con 0x55a2dbe6b3a0 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f80ad27d740 10 client.27394 fetch_fsmap finished waiting for FSMap version 75 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f80ad27d740 10 client.27394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f80ad27d740 20 client.27394 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f80ad27d740 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbee6770 0x55a2dbee8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f80ad27d740 4 client.27394 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.942+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2dbdcdb80 con 0x55a2dbee6770 2024-09-05T12:45:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.943+0000 7f80a3400640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbee6770 0x55a2dbee8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.944+0000 7f80a3400640 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbee6770 0x55a2dbee8b60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=648699fdefb72cc8 server_cookie=ee65374bb49164fe in_seq=0 out_seq=0 2024-09-05T12:45:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:18.944+0000 7f808d600640 10 client.27394 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:18 smithi174 ceph-mon[34313]: pgmap v552: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-05T12:45:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:18 smithi174 ceph-mon[34313]: osdmap e119: 12 total, 12 up, 12 in 2024-09-05T12:45:19.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:18 smithi071 ceph-mon[29845]: pgmap v552: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-05T12:45:19.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:18 smithi071 ceph-mon[29845]: osdmap e119: 12 total, 12 up, 12 in 2024-09-05T12:45:19.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:18 smithi123 ceph-mon[34321]: pgmap v552: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-05T12:45:19.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:18 smithi123 ceph-mon[34321]: osdmap e119: 12 total, 12 up, 12 in 2024-09-05T12:45:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.352+0000 7f808d600640 1 -- 172.21.15.71:0/1284894107 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2dbdcdb80 con 0x55a2dbee6770 2024-09-05T12:45:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.352+0000 7f808d600640 10 client.27394 handle_command_reply: tid=0 2024-09-05T12:45:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.353+0000 7f80ad27d740 1 client.27394 shutdown 2024-09-05T12:45:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.353+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 msgr2=0x55a2dbe6fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.353+0000 7f80ad27d740 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6fd70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f809c009710 tx=0x7f809c00cf10 comp rx=0 tx=0).stop 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 shutdown_connections 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55a2dbee6770 0x55a2dbee8b60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a2dbe6b3a0 0x55a2dbe6fd70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f808d600640 0 client.27394 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a2dbe60b00 0x55a2dbe6f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 --2- 172.21.15.71:0/1284894107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a2dbe601e0 0x55a2dbe6f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 >> 172.21.15.71:0/1284894107 conn(0x55a2dbe5c750 msgr2=0x55a2dbe769e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 shutdown_connections 2024-09-05T12:45:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 1 -- 172.21.15.71:0/1284894107 wait complete. 2024-09-05T12:45:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.354+0000 7f80ad27d740 20 client.27394 trim_cache size 0 max 16384 2024-09-05T12:45:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:45:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:45:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:45:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:45:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f80880a2df0 msgr2=0x7f80880a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f80880a2df0 0x7f80880a52b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f80940096c0 tx=0x7f8094000bc0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 msgr2=0x7f80a419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a419a990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f809c00eb40 tx=0x7f809c00c2d0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 shutdown_connections 2024-09-05T12:45:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80a4107b70 0x7f80a419a990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80a4102470 0x7f80a4196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f80880a2df0 0x7f80880a52b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 --2- 172.21.15.71:0/1852736015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80a4101ad0 0x7f80a41960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.356+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 >> 172.21.15.71:0/1852736015 conn(0x7f80a40fd7d0 msgr2=0x7f80a40fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.357+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 shutdown_connections 2024-09-05T12:45:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:19.357+0000 7f80aa400640 1 -- 172.21.15.71:0/1852736015 wait complete. 2024-09-05T12:45:19.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 80083 15.71 MB/sec execute 133 sec latency 186.128 ms 2024-09-05T12:45:20.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 80713 15.84 MB/sec execute 134 sec latency 1063.167 ms 2024-09-05T12:45:21.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:20 smithi174 ceph-mon[34313]: pgmap v554: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-09-05T12:45:21.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:20 smithi071 ceph-mon[29845]: pgmap v554: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-09-05T12:45:21.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:20 smithi123 ceph-mon[34321]: pgmap v554: 129 pgs: 129 active+clean; 789 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-09-05T12:45:21.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 81549 16.14 MB/sec execute 135 sec latency 2063.266 ms 2024-09-05T12:45:22.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:21 smithi174 ceph-mon[34313]: pgmap v555: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 812 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.7 MiB/s wr, 422 op/s 2024-09-05T12:45:22.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:21 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:45:22.080 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:21 smithi123 ceph-mon[34321]: pgmap v555: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 812 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.7 MiB/s wr, 422 op/s 2024-09-05T12:45:22.080 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:21 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:45:22.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:21 smithi071 ceph-mon[29845]: pgmap v555: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 812 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.7 MiB/s wr, 422 op/s 2024-09-05T12:45:22.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:21 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:45:22.813 INFO:tasks.workunit.client.0.smithi071.stdout: 10 82018 16.17 MB/sec execute 136 sec latency 3063.407 ms 2024-09-05T12:45:23.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:22 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:45:23.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:22 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:45:23.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:22 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:45:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:22 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:45:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:22 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:45:23.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:22 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:45:23.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:22 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:45:23.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:22 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:45:23.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:22 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:45:23.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 82575 16.29 MB/sec execute 137 sec latency 4063.465 ms 2024-09-05T12:45:24.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:23 smithi174 ceph-mon[34313]: pgmap v556: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 816 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 443 op/s 2024-09-05T12:45:24.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:23 smithi071 ceph-mon[29845]: pgmap v556: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 816 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 443 op/s 2024-09-05T12:45:24.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:23 smithi123 ceph-mon[34321]: pgmap v556: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 816 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 443 op/s 2024-09-05T12:45:24.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83180 16.43 MB/sec execute 138 sec latency 5063.512 ms 2024-09-05T12:45:25.812 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83714 16.47 MB/sec execute 139 sec latency 6063.560 ms 2024-09-05T12:45:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:25 smithi174 ceph-mon[34313]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:45:26.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:25 smithi174 ceph-mon[34313]: pgmap v557: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 816 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 443 op/s 2024-09-05T12:45:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:25 smithi071 ceph-mon[29845]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:45:26.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:25 smithi071 ceph-mon[29845]: pgmap v557: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 816 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 443 op/s 2024-09-05T12:45:26.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:25 smithi123 ceph-mon[34321]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-05T12:45:26.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:25 smithi123 ceph-mon[34321]: pgmap v557: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 816 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 5.2 MiB/s wr, 443 op/s 2024-09-05T12:45:26.833 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83896 16.43 MB/sec execute 140 sec latency 7083.819 ms 2024-09-05T12:45:27.835 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83897 16.32 MB/sec execute 141 sec latency 8086.606 ms 2024-09-05T12:45:28.192 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:27 smithi071 ceph-mon[29845]: pgmap v558: 129 pgs: 129 active+clean; 840 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.3 MiB/s wr, 531 op/s 2024-09-05T12:45:28.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:27 smithi174 ceph-mon[34313]: pgmap v558: 129 pgs: 129 active+clean; 840 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.3 MiB/s wr, 531 op/s 2024-09-05T12:45:28.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:27 smithi123 ceph-mon[34321]: pgmap v558: 129 pgs: 129 active+clean; 840 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 6.3 MiB/s wr, 531 op/s 2024-09-05T12:45:28.835 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83897 16.20 MB/sec execute 142 sec latency 9086.681 ms 2024-09-05T12:45:29.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83898 16.09 MB/sec execute 143 sec latency 10086.734 ms 2024-09-05T12:45:30.186 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:29 smithi071 ceph-mon[29845]: pgmap v559: 129 pgs: 129 active+clean; 846 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.3 MiB/s wr, 512 op/s 2024-09-05T12:45:30.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:29 smithi174 ceph-mon[34313]: pgmap v559: 129 pgs: 129 active+clean; 846 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.3 MiB/s wr, 512 op/s 2024-09-05T12:45:30.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:29 smithi123 ceph-mon[34321]: pgmap v559: 129 pgs: 129 active+clean; 846 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 6.3 MiB/s wr, 512 op/s 2024-09-05T12:45:30.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 83988 16.04 MB/sec execute 144 sec latency 11086.788 ms 2024-09-05T12:45:31.563 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:31 smithi071 ceph-mon[29845]: osdmap e120: 12 total, 12 up, 12 in 2024-09-05T12:45:31.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:31 smithi123 ceph-mon[34321]: osdmap e120: 12 total, 12 up, 12 in 2024-09-05T12:45:31.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:31 smithi174 ceph-mon[34313]: osdmap e120: 12 total, 12 up, 12 in 2024-09-05T12:45:31.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84029 15.94 MB/sec execute 145 sec latency 12086.942 ms 2024-09-05T12:45:32.401 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:32 smithi071 ceph-mon[29845]: pgmap v561: 129 pgs: 129 active+clean; 859 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 419 op/s 2024-09-05T12:45:32.485 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/142934014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 msgr2=0x7f1028104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- 172.21.15.71:0/142934014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f1028104330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f101c009900 tx=0x7f101c02a410 comp rx=0 tx=0).stop 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/142934014 shutdown_connections 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- 172.21.15.71:0/142934014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10281090a0 0x7f1028110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- 172.21.15.71:0/142934014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1028104900 0x7f1028108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- 172.21.15.71:0/142934014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f1028104330 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/142934014 >> 172.21.15.71:0/142934014 conn(0x7f10280ff9e0 msgr2=0x7f1028101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/142934014 shutdown_connections 2024-09-05T12:45:32.486 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/142934014 wait complete. 2024-09-05T12:45:32.488 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 Processor -- start 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- start start 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f102819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10281090a0 0x7f102819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f102819ef00 0x7f102819f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1028114190 con 0x7f1028103f50 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1028114010 con 0x7f102819ef00 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1028114310 con 0x7f10281090a0 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f102819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f102819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59644/0 (socket says 172.21.15.71:59644) 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 -- 172.21.15.71:0/407782986 learned_addr learned my addr 172.21.15.71:0/407782986 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102e800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f102819ef00 0x7f102819f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.489 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 -- 172.21.15.71:0/407782986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10281090a0 msgr2=0x7f102819ab80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10281090a0 0x7f102819ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 -- 172.21.15.71:0/407782986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f102819ef00 msgr2=0x7f102819f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f102819ef00 0x7f102819f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 -- 172.21.15.71:0/407782986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f102819f9f0 con 0x7f1028103f50 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102e800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f102819ef00 0x7f102819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f102de00640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f102819a640 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f101c004770 tx=0x7f101c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1022a00640 1 -- 172.21.15.71:0/407782986 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c03f070 con 0x7f1028103f50 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1022a00640 1 -- 172.21.15.71:0/407782986 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f101c031ca0 con 0x7f1028103f50 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f102819fc80 con 0x7f1028103f50 2024-09-05T12:45:32.490 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1022a00640 1 -- 172.21.15.71:0/407782986 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101c035030 con 0x7f1028103f50 2024-09-05T12:45:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.483+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10281a7440 con 0x7f1028103f50 2024-09-05T12:45:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 Processor -- start 2024-09-05T12:45:32.491 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 -- start start 2024-09-05T12:45:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3f9b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.492 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3fa63a0 0x560fa3fa8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560fa3f01100 con 0x560fa3f9b1e0 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560fa3ea91b0 con 0x560fa3fa63a0 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f103368d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560fa3efd0c0 con 0x560fa3f9bb00 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3f9b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3f9b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59650/0 (socket says 172.21.15.71:59650) 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 -- 172.21.15.71:0/409532994 learned_addr learned my addr 172.21.15.71:0/409532994 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 -- 172.21.15.71:0/409532994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 msgr2=0x560fa3fa5e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa5e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 -- 172.21.15.71:0/409532994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3fa63a0 msgr2=0x560fa3fa8790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3fa63a0 0x560fa3fa8790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 -- 172.21.15.71:0/409532994 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa3c14210 con 0x560fa3f9b1e0 2024-09-05T12:45:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f1022a00640 1 -- 172.21.15.71:0/407782986 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f101c042070 con 0x7f1028103f50 2024-09-05T12:45:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102e800640 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3f9b5c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1024010750 tx=0x7f1024010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd6e7226e6628c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f1017000640 1 -- 172.21.15.71:0/409532994 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400ead0 con 0x560fa3f9b1e0 2024-09-05T12:45:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f1017000640 1 -- 172.21.15.71:0/409532994 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024010e60 con 0x560fa3f9b1e0 2024-09-05T12:45:32.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f1017000640 1 -- 172.21.15.71:0/409532994 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024011600 con 0x560fa3f9b1e0 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f1022a00640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f10100a2da0 0x7f10100a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f1022a00640 1 -- 172.21.15.71:0/407782986 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f101c0f0060 con 0x7f1028103f50 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.489+0000 7f102de00640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f10100a2da0 0x7f10100a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/409532994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 msgr2=0x560fa3f9b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3f9b5c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f1024010750 tx=0x7f1024010c10 comp rx=0 tx=0).stop 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102de00640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f10100a2da0 0x7f10100a5260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f101c0b8b60 tx=0x7f101c009260 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/409532994 shutdown_connections 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3fa63a0 0x560fa3fa8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- 172.21.15.71:0/409532994 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3f9b5c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/409532994 >> 172.21.15.71:0/409532994 conn(0x560fa3f97750 msgr2=0x560fa3f97b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/409532994 shutdown_connections 2024-09-05T12:45:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/409532994 wait complete. 2024-09-05T12:45:32.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- start start 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 Processor -- start 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 0x560fa3fa3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560fa3fa8fd0 con 0x560fa3fa63a0 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560fa3e8a090 con 0x560fa3f9b1e0 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560fa3fa9150 con 0x560fa3f9bb00 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 10 client.0 _ll_register_callbacks cb 0x560fa3edef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 0x560fa3fa3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 0x560fa3fa3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59664/0 (socket says 172.21.15.71:59664) 2024-09-05T12:45:32.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 -- 172.21.15.71:0/220847217 learned_addr learned my addr 172.21.15.71:0/220847217 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102d400640 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 -- 172.21.15.71:0/220847217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 msgr2=0x560fa3fa37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 -- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 msgr2=0x560fa3fa32a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa32a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 -- 172.21.15.71:0/220847217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa3ce0620 con 0x560fa3fa63a0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f102e800640 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 0x560fa3fa3d20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f10240098b0 tx=0x7f10240103b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5f37fbcb2f9875 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f100b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f100b400640 1 -- 172.21.15.71:0/220847217 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400eba0 con 0x560fa3fa63a0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1024010e60 con 0x560fa3fa63a0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f100b400640 1 -- 172.21.15.71:0/220847217 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f102400ed40 con 0x560fa3fa63a0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f100b400640 1 -- 172.21.15.71:0/220847217 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1024004a40 con 0x560fa3fa63a0 2024-09-05T12:45:32.500 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 10 client.17352 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560fa3fb4b80 con 0x560fa3fa63a0 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f100b400640 1 -- 172.21.15.71:0/220847217 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024010e60 con 0x560fa3fa63a0 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f100b400640 1 -- 172.21.15.71:0/220847217 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f102400f040 con 0x560fa3fa63a0 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 10 client.17352 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 20 client.17352 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa3fb5fe0 0x560fa3fb83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.502 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 4 client.17352 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:45:32.503 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.494+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560fa3f0e830 con 0x560fa3fb5fe0 2024-09-05T12:45:32.530 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.530+0000 7f102d400640 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa3fb5fe0 0x560fa3fb83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.601+0000 7f100b400640 10 client.17352 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.601+0000 7f102d400640 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa3fb5fe0 0x560fa3fb83d0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1b370bf70bcc39 server_cookie=33ff4fc72da09d28 in_seq=0 out_seq=0 2024-09-05T12:45:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f100b400640 1 -- 172.21.15.71:0/220847217 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560fa3f0e830 con 0x560fa3fb5fe0 2024-09-05T12:45:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f100b400640 10 client.17352 handle_command_reply: tid=0 2024-09-05T12:45:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 client.17352 shutdown 2024-09-05T12:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 msgr2=0x560fa3fa3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 0x560fa3fa3d20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f10240098b0 tx=0x7f10240103b0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 shutdown_connections 2024-09-05T12:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa3fb5fe0 0x560fa3fb83d0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3fa63a0 0x560fa3fa3d20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3f9bb00 0x560fa3fa37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 --2- 172.21.15.71:0/220847217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f100b400640 0 client.17352 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.602+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 >> 172.21.15.71:0/220847217 conn(0x560fa3f97750 msgr2=0x560fa3f9d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.603+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 shutdown_connections 2024-09-05T12:45:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.603+0000 7f103368d740 1 -- 172.21.15.71:0/220847217 wait complete. 2024-09-05T12:45:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.603+0000 7f103368d740 20 client.17352 trim_cache size 0 max 16384 2024-09-05T12:45:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 Processor -- start 2024-09-05T12:45:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- start start 2024-09-05T12:45:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9bb00 0x560fa3fbdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fbe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f10281b1100 con 0x560fa3f9b1e0 2024-09-05T12:45:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10281b2c40 con 0x560fa3f9bb00 2024-09-05T12:45:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10281b17d0 con 0x560fa3fa63a0 2024-09-05T12:45:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59666/0 (socket says 172.21.15.71:59666) 2024-09-05T12:45:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 -- 172.21.15.71:0/98678539 learned_addr learned my addr 172.21.15.71:0/98678539 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fbe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 -- 172.21.15.71:0/98678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 msgr2=0x560fa3fbe120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fbe120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 -- 172.21.15.71:0/98678539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9bb00 msgr2=0x560fa3fbdbe0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9bb00 0x560fa3fbdbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 -- 172.21.15.71:0/98678539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa3f97580 con 0x560fa3f9b1e0 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa5100 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1024015040 tx=0x7f1024010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50365430fd05bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f1016600640 1 -- 172.21.15.71:0/98678539 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024005b00 con 0x560fa3f9b1e0 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f1016600640 1 -- 172.21.15.71:0/98678539 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024012c90 con 0x560fa3f9b1e0 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f1016600640 1 -- 172.21.15.71:0/98678539 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102400edc0 con 0x560fa3f9b1e0 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- 172.21.15.71:0/98678539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 msgr2=0x560fa3fa5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa5100 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f1024015040 tx=0x7f1024010420 comp rx=0 tx=0).stop 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- 172.21.15.71:0/98678539 shutdown_connections 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fbe120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9bb00 0x560fa3fbdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- 172.21.15.71:0/98678539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa5100 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- 172.21.15.71:0/98678539 >> 172.21.15.71:0/98678539 conn(0x560fa3f97750 msgr2=0x560fa3fa49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- 172.21.15.71:0/98678539 shutdown_connections 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- 172.21.15.71:0/98678539 wait complete. 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- start start 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 Processor -- start 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 0x560fa3fa0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fa0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f10281b2570 con 0x560fa3f9bb00 2024-09-05T12:45:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10281b1ea0 con 0x560fa3f9b1e0 2024-09-05T12:45:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10281af100 con 0x560fa3fa63a0 2024-09-05T12:45:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f103368d740 10 client.0 _ll_register_callbacks cb 0x7f1024016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 0x560fa3fa0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.618+0000 7f102e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 0x560fa3fa0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59670/0 (socket says 172.21.15.71:59670) 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 -- 172.21.15.71:0/3219560816 learned_addr learned my addr 172.21.15.71:0/3219560816 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102e800640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fa0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 -- 172.21.15.71:0/3219560816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 msgr2=0x560fa3fa0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fa0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 -- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 msgr2=0x560fa3fa0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:32.639 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 -- 172.21.15.71:0/3219560816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa3f98800 con 0x560fa3f9bb00 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102e800640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f102d400640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 0x560fa3fa0940 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x560fa3f98ad0 tx=0x7f1018014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc43b0f1af41b45 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f100b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f100b400640 1 -- 172.21.15.71:0/3219560816 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018022060 con 0x560fa3f9bb00 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f100b400640 1 -- 172.21.15.71:0/3219560816 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f101800f040 con 0x560fa3f9bb00 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1024012c90 con 0x560fa3f9bb00 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f100b400640 1 -- 172.21.15.71:0/3219560816 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1018029050 con 0x560fa3f9bb00 2024-09-05T12:45:32.640 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f103368d740 10 client.17358 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.623+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560fa4020e90 con 0x560fa3f9bb00 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f100b400640 1 -- 172.21.15.71:0/3219560816 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1024012c90 con 0x560fa3f9bb00 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f100b400640 1 -- 172.21.15.71:0/3219560816 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f10180154b0 con 0x560fa3f9bb00 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f103368d740 10 client.17358 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f103368d740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f103368d740 20 client.17358 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f103368d740 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa40215f0 0x560fa4023a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f103368d740 4 client.17358 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560fa3f08b80 con 0x560fa40215f0 2024-09-05T12:45:32.641 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f102e800640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa40215f0 0x560fa4023a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f102e800640 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa40215f0 0x560fa4023a10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4089baaea2588abd server_cookie=c471998eea6ff79 in_seq=0 out_seq=0 2024-09-05T12:45:32.642 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:32.625+0000 7f100b400640 10 client.17358 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:32.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:32 smithi123 ceph-mon[34321]: pgmap v561: 129 pgs: 129 active+clean; 859 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 419 op/s 2024-09-05T12:45:32.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:32 smithi174 ceph-mon[34313]: pgmap v561: 129 pgs: 129 active+clean; 859 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 419 op/s 2024-09-05T12:45:32.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84029 15.83 MB/sec execute 146 sec latency 13087.027 ms 2024-09-05T12:45:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f100b400640 1 -- 172.21.15.71:0/3219560816 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560fa3f08b80 con 0x560fa40215f0 2024-09-05T12:45:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f100b400640 10 client.17358 handle_command_reply: tid=0 2024-09-05T12:45:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 client.17358 shutdown 2024-09-05T12:45:33.293 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 msgr2=0x560fa3fa0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:33.294 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 0x560fa3fa0940 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x560fa3f98ad0 tx=0x7f1018014780 comp rx=0 tx=0).stop 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 shutdown_connections 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560fa40215f0 0x560fa4023a10 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f100b400640 0 client.17358 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560fa3fa63a0 0x560fa3fa0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fa3f9bb00 0x560fa3fa0940 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 --2- 172.21.15.71:0/3219560816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3f9b1e0 0x560fa3fa0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 >> 172.21.15.71:0/3219560816 conn(0x560fa3f97750 msgr2=0x560fa3fa39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 shutdown_connections 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 1 -- 172.21.15.71:0/3219560816 wait complete. 2024-09-05T12:45:33.295 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.290+0000 7f103368d740 20 client.17358 trim_cache size 0 max 16384 2024-09-05T12:45:33.297 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.296+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f10100a2da0 msgr2=0x7f10100a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.296+0000 7f1030800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f10100a2da0 0x7f10100a5260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f101c0b8b60 tx=0x7f101c009260 comp rx=0 tx=0).stop 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.296+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 msgr2=0x7f102819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.296+0000 7f1030800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f102819a640 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f101c004770 tx=0x7f101c003c10 comp rx=0 tx=0).stop 2024-09-05T12:45:33.298 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 shutdown_connections 2024-09-05T12:45:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f102819ef00 0x7f102819f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10281090a0 0x7f102819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f10100a2da0 0x7f10100a5260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 --2- 172.21.15.71:0/407782986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1028103f50 0x7f102819a640 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:33.299 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 >> 172.21.15.71:0/407782986 conn(0x7f10280ff9e0 msgr2=0x7f1028100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:33.309 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 shutdown_connections 2024-09-05T12:45:33.310 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:33.297+0000 7f1030800640 1 -- 172.21.15.71:0/407782986 wait complete. 2024-09-05T12:45:33.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84029 15.73 MB/sec execute 147 sec latency 14087.075 ms 2024-09-05T12:45:34.359 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:34 smithi071 ceph-mon[29845]: pgmap v562: 129 pgs: 129 active+clean; 862 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-09-05T12:45:34.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:34 smithi123 ceph-mon[34321]: pgmap v562: 129 pgs: 129 active+clean; 862 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-09-05T12:45:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:34 smithi174 ceph-mon[34313]: pgmap v562: 129 pgs: 129 active+clean; 862 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-09-05T12:45:34.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84030 15.62 MB/sec execute 148 sec latency 15087.152 ms 2024-09-05T12:45:35.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84030 15.52 MB/sec execute 149 sec latency 16087.225 ms 2024-09-05T12:45:36.623 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:36 smithi071 ceph-mon[29845]: pgmap v563: 129 pgs: 129 active+clean; 862 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-09-05T12:45:36.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:36 smithi123 ceph-mon[34321]: pgmap v563: 129 pgs: 129 active+clean; 862 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-09-05T12:45:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:36 smithi174 ceph-mon[34313]: pgmap v563: 129 pgs: 129 active+clean; 862 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 7.8 MiB/s wr, 414 op/s 2024-09-05T12:45:36.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84030 15.41 MB/sec execute 150 sec latency 17087.294 ms 2024-09-05T12:45:37.008 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.8 2024-09-05T12:45:37.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:45:37.482 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:45:37.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84047 15.31 MB/sec execute 151 sec latency 18087.331 ms 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 -- 172.21.15.71:0/2172901631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260101df0 msgr2=0x7f92601021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 --2- 172.21.15.71:0/2172901631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260101df0 0x7f92601021d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9250009900 tx=0x7f925002a420 comp rx=0 tx=0).stop 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 -- 172.21.15.71:0/2172901631 shutdown_connections 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 --2- 172.21.15.71:0/2172901631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9260106f20 0x7f926010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 --2- 172.21.15.71:0/2172901631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 0x7f92601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 --2- 172.21.15.71:0/2172901631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260101df0 0x7f92601021d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.325 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.319+0000 7f9266800640 1 -- 172.21.15.71:0/2172901631 >> 172.21.15.71:0/2172901631 conn(0x7f92600fd8b0 msgr2=0x7f92600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:38.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- 172.21.15.71:0/2172901631 shutdown_connections 2024-09-05T12:45:38.326 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- 172.21.15.71:0/2172901631 wait complete. 2024-09-05T12:45:38.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 Processor -- start 2024-09-05T12:45:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- start start 2024-09-05T12:45:38.328 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9260101df0 0x7f926010e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:38.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 0x7f926010b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:38.329 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 0x7f926010bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9260114250 con 0x7f92601027a0 2024-09-05T12:45:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92601140d0 con 0x7f9260101df0 2024-09-05T12:45:38.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92601143d0 con 0x7f9260106f20 2024-09-05T12:45:38.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 0x7f926010bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:38.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9260101df0 0x7f926010e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:38.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 0x7f926010bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37730/0 (socket says 172.21.15.71:37730) 2024-09-05T12:45:38.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 -- 172.21.15.71:0/931610576 learned_addr learned my addr 172.21.15.71:0/931610576 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:38.332 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925ea00640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 0x7f926010b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 -- 172.21.15.71:0/931610576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9260101df0 msgr2=0x7f926010e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9260101df0 0x7f926010e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 -- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 msgr2=0x7f926010b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 0x7f926010b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 -- 172.21.15.71:0/931610576 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926010c750 con 0x7f9260106f20 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925ea00640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 0x7f926010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f925fe00640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 0x7f926010bee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f925800b990 tx=0x7f925800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925802a070 con 0x7f9260106f20 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f925801dbb0 con 0x7f9260106f20 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92580076e0 con 0x7f9260106f20 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92601a07f0 con 0x7f9260106f20 2024-09-05T12:45:38.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.324+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a0cb0 con 0x7f9260106f20 2024-09-05T12:45:38.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.329+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92601032d0 con 0x7f9260106f20 2024-09-05T12:45:38.339 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.336+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f925802d070 con 0x7f9260106f20 2024-09-05T12:45:38.341 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.336+0000 7f9257e00640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92440a2dc0 0x7f92440a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:38.342 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.336+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f92580dfef0 con 0x7f9260106f20 2024-09-05T12:45:38.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.336+0000 7f925f400640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92440a2dc0 0x7f92440a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:38.343 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.341+0000 7f925f400640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92440a2dc0 0x7f92440a5280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9250000c00 tx=0x7f9250031400 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:38.348 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.343+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92580ac830 con 0x7f9260106f20 2024-09-05T12:45:38.583 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:38 smithi071 ceph-mon[29845]: pgmap v564: 129 pgs: 129 active+clean; 853 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.2 MiB/s wr, 369 op/s 2024-09-05T12:45:38.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:38 smithi123 ceph-mon[34321]: pgmap v564: 129 pgs: 129 active+clean; 853 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.2 MiB/s wr, 369 op/s 2024-09-05T12:45:38.734 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.731+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92601021d0 con 0x7f9260106f20 2024-09-05T12:45:38.736 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.735+0000 7f9257e00640 1 -- 172.21.15.71:0/931610576 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7f92580ac130 con 0x7f9260106f20 2024-09-05T12:45:38.741 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:45:38.741 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92440a2dc0 msgr2=0x7f92440a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92440a2dc0 0x7f92440a5280 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9250000c00 tx=0x7f9250031400 comp rx=0 tx=0).stop 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 msgr2=0x7f926010bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 0x7f926010bee0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f925800b990 tx=0x7f925800be50 comp rx=0 tx=0).stop 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 shutdown_connections 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9260106f20 0x7f926010bee0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.742 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f92440a2dc0 0x7f92440a5280 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f92601027a0 0x7f926010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 --2- 172.21.15.71:0/931610576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9260101df0 0x7f926010e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 >> 172.21.15.71:0/931610576 conn(0x7f92600fd8b0 msgr2=0x7f92600feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 shutdown_connections 2024-09-05T12:45:38.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:38.736+0000 7f9266800640 1 -- 172.21.15.71:0/931610576 wait complete. 2024-09-05T12:45:38.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:38 smithi174 ceph-mon[34313]: pgmap v564: 129 pgs: 129 active+clean; 853 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.2 MiB/s wr, 369 op/s 2024-09-05T12:45:38.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 84405 15.31 MB/sec execute 152 sec latency 18221.991 ms 2024-09-05T12:45:39.410 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:39 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/931610576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:45:39.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:39 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/931610576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:45:39.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:39 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/931610576' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:45:39.769 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 152.8 --await-for 45.0' 2024-09-05T12:45:39.769 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 152.8 --await-for 45.0 2024-09-05T12:45:39.836 INFO:tasks.workunit.client.0.smithi071.stdout: 10 85140 15.49 MB/sec execute 153 sec latency 12667.835 ms 2024-09-05T12:45:40.369 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:45:40.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:40 smithi071 ceph-mon[29845]: pgmap v565: 129 pgs: 129 active+clean; 852 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 343 op/s 2024-09-05T12:45:40.681 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:40 smithi123 ceph-mon[34321]: pgmap v565: 129 pgs: 129 active+clean; 852 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 343 op/s 2024-09-05T12:45:40.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:40 smithi174 ceph-mon[34313]: pgmap v565: 129 pgs: 129 active+clean; 852 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.7 MiB/s wr, 343 op/s 2024-09-05T12:45:40.837 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86030 15.71 MB/sec execute 154 sec latency 58.303 ms 2024-09-05T12:45:41.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.436+0000 7f30b9600640 1 -- 172.21.15.71:0/2126299659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109830 msgr2=0x7f30b4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:41.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.436+0000 7f30b9600640 1 --2- 172.21.15.71:0/2126299659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109830 0x7f30b4109c90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009900 tx=0x7f30a002a420 comp rx=0 tx=0).stop 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.436+0000 7f30b9600640 1 -- 172.21.15.71:0/2126299659 shutdown_connections 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.436+0000 7f30b9600640 1 --2- 172.21.15.71:0/2126299659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b411b140 0x7f30b411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.436+0000 7f30b9600640 1 --2- 172.21.15.71:0/2126299659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109830 0x7f30b4109c90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.436+0000 7f30b9600640 1 --2- 172.21.15.71:0/2126299659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b4109450 0x7f30b4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.437+0000 7f30b9600640 1 -- 172.21.15.71:0/2126299659 >> 172.21.15.71:0/2126299659 conn(0x7f30b406e140 msgr2=0x7f30b406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.437+0000 7f30b9600640 1 -- 172.21.15.71:0/2126299659 shutdown_connections 2024-09-05T12:45:41.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.438+0000 7f30b9600640 1 -- 172.21.15.71:0/2126299659 wait complete. 2024-09-05T12:45:41.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.440+0000 7f30b9600640 1 Processor -- start 2024-09-05T12:45:41.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.440+0000 7f30b9600640 1 -- start start 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109450 0x7f30b41a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b9600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 0x7f30b41a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b9600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b411b140 0x7f30b41abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b9600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4120e10 con 0x7f30b4109450 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b9600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4120c90 con 0x7f30b411b140 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b9600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4120f90 con 0x7f30b4109830 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.441+0000 7f30b2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 0x7f30b41a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.442+0000 7f30b2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 0x7f30b41a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37740/0 (socket says 172.21.15.71:37740) 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.442+0000 7f30b2000640 1 -- 172.21.15.71:0/833765577 learned_addr learned my addr 172.21.15.71:0/833765577 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:41.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.442+0000 7f30b3400640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b411b140 0x7f30b41abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.442+0000 7f30b2000640 1 -- 172.21.15.71:0/833765577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b411b140 msgr2=0x7f30b41abb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:41.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.443+0000 7f30b2000640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b411b140 0x7f30b41abb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.443+0000 7f30b2000640 1 -- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109450 msgr2=0x7f30b41a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:41.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.443+0000 7f30b2000640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109450 0x7f30b41a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.443+0000 7f30b2000640 1 -- 172.21.15.71:0/833765577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b41ac260 con 0x7f30b4109830 2024-09-05T12:45:41.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.443+0000 7f30b3400640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b411b140 0x7f30b41abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:45:41.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.444+0000 7f30b2000640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 0x7f30b41a77f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30a002aa10 tx=0x7f30a0005230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:41.446 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.444+0000 7f30ab400640 1 -- 172.21.15.71:0/833765577 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a0040030 con 0x7f30b4109830 2024-09-05T12:45:41.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.444+0000 7f30ab400640 1 -- 172.21.15.71:0/833765577 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a0003e30 con 0x7f30b4109830 2024-09-05T12:45:41.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.444+0000 7f30ab400640 1 -- 172.21.15.71:0/833765577 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a003ca70 con 0x7f30b4109830 2024-09-05T12:45:41.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.444+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b41ac4f0 con 0x7f30b4109830 2024-09-05T12:45:41.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.445+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b41ac8c0 con 0x7f30b4109830 2024-09-05T12:45:41.458 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.454+0000 7f30bc244740 1 Processor -- start 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 -- start start 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556b7771f530 con 0x556b777b93b0 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b776c6860 con 0x556b777b9cd0 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.455+0000 7f30bc244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b7771b4f0 con 0x556b777c4590 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777b9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53274/0 (socket says 172.21.15.71:53274) 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 -- 172.21.15.71:0/4113248570 learned_addr learned my addr 172.21.15.71:0/4113248570 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b2a00640 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 -- 172.21.15.71:0/4113248570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 msgr2=0x556b777c6980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:45:41.459 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c6980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 -- 172.21.15.71:0/4113248570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 msgr2=0x556b777c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.456+0000 7f30b3400640 1 -- 172.21.15.71:0/4113248570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b77490b90 con 0x556b777b93b0 2024-09-05T12:45:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.458+0000 7f30b3400640 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777b9790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00f5c0 tx=0x7f30ac00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1288ea4feb7affca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:41.460 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.459+0000 7f309f400640 1 -- 172.21.15.71:0/4113248570 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac018020 con 0x556b777b93b0 2024-09-05T12:45:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.459+0000 7f309f400640 1 -- 172.21.15.71:0/4113248570 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30ac014b60 con 0x556b777b93b0 2024-09-05T12:45:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.459+0000 7f309f400640 1 -- 172.21.15.71:0/4113248570 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac00dc70 con 0x556b777b93b0 2024-09-05T12:45:41.461 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.460+0000 7f30ab400640 1 -- 172.21.15.71:0/833765577 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f30a0043030 con 0x7f30b4109830 2024-09-05T12:45:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.460+0000 7f30bc244740 1 -- 172.21.15.71:0/4113248570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 msgr2=0x556b777b9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:41.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.460+0000 7f30bc244740 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777b9790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00f5c0 tx=0x7f30ac00fa80 comp rx=0 tx=0).stop 2024-09-05T12:45:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.461+0000 7f30bc244740 1 -- 172.21.15.71:0/4113248570 shutdown_connections 2024-09-05T12:45:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.461+0000 7f30bc244740 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.461+0000 7f30bc244740 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.461+0000 7f30bc244740 1 --2- 172.21.15.71:0/4113248570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777b9790 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.461+0000 7f30bc244740 1 -- 172.21.15.71:0/4113248570 >> 172.21.15.71:0/4113248570 conn(0x556b777b5940 msgr2=0x556b777b5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:41.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.463+0000 7f30bc244740 1 -- 172.21.15.71:0/4113248570 shutdown_connections 2024-09-05T12:45:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.463+0000 7f30bc244740 1 -- 172.21.15.71:0/4113248570 wait complete. 2024-09-05T12:45:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.463+0000 7f30ab400640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30980a2cd0 0x7f30980a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.463+0000 7f30bc244740 1 -- start start 2024-09-05T12:45:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.464+0000 7f30ab400640 1 -- 172.21.15.71:0/833765577 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f30a00f0080 con 0x7f30b4109830 2024-09-05T12:45:41.466 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.464+0000 7f30bc244740 1 Processor -- start 2024-09-05T12:45:41.467 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.465+0000 7f30b3400640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30980a2cd0 0x7f30980a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.465+0000 7f30bc244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777c1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.466+0000 7f30bc244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.466+0000 7f30bc244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.466+0000 7f30b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777c1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.466+0000 7f30b2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777c1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:53278/0 (socket says 172.21.15.71:53278) 2024-09-05T12:45:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30b2a00640 1 -- 172.21.15.71:0/1961564247 learned_addr learned my addr 172.21.15.71:0/1961564247 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:41.469 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556b777c6fb0 con 0x556b777b93b0 2024-09-05T12:45:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b776d9e60 con 0x556b777b9cd0 2024-09-05T12:45:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b777c7130 con 0x556b777c4590 2024-09-05T12:45:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30bc244740 10 client.0 _ll_register_callbacks cb 0x556b776decc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:41.470 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30b3400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.471 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.467+0000 7f30b2000640 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.468+0000 7f30b2a00640 1 -- 172.21.15.71:0/1961564247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 msgr2=0x556b777c21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.468+0000 7f30b2a00640 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.468+0000 7f30b2a00640 1 -- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 msgr2=0x556b777c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.468+0000 7f30b2a00640 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.468+0000 7f30b2a00640 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b776e5990 con 0x556b777b93b0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.468+0000 7f30b2a00640 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777c1740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30a4003e90 tx=0x7f30a402d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9549903a4c140516 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.469+0000 7f3093e00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.469+0000 7f3093e00640 1 -- 172.21.15.71:0/1961564247 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4033860 con 0x556b777b93b0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.469+0000 7f3093e00640 1 -- 172.21.15.71:0/1961564247 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f30a4033a00 con 0x556b777b93b0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.470+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30ac014b60 con 0x556b777b93b0 2024-09-05T12:45:41.472 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.470+0000 7f30b3400640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30980a2cd0 0x7f30980a5190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00f590 tx=0x7f30ac004a40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:41.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.470+0000 7f3093e00640 1 -- 172.21.15.71:0/1961564247 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f30a4004490 con 0x556b777b93b0 2024-09-05T12:45:41.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.470+0000 7f30bc244740 10 client.17370 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:41.473 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.471+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x556b777d2f70 con 0x556b777b93b0 2024-09-05T12:45:41.474 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.471+0000 7f3093e00640 1 -- 172.21.15.71:0/1961564247 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30ac014b60 con 0x556b777b93b0 2024-09-05T12:45:41.476 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f3093e00640 1 -- 172.21.15.71:0/1961564247 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f30a4030040 con 0x556b777b93b0 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30bc244740 10 client.17370 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30bc244740 10 client.17370 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30bc244740 20 client.17370 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30bc244740 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b777d4380 0x556b777d6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30bc244740 4 client.17370 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b776c2310 con 0x556b777d4380 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.472+0000 7f30b2000640 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b777d4380 0x556b777d6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.474+0000 7f30b2000640 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b777d4380 0x556b777d6770 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a5466772d2284 server_cookie=d37946853027dada in_seq=0 out_seq=0 2024-09-05T12:45:41.477 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.474+0000 7f3093e00640 10 client.17370 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:41.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.475+0000 7f3093e00640 1 -- 172.21.15.71:0/1961564247 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b776c2310 con 0x556b777d4380 2024-09-05T12:45:41.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.475+0000 7f3093e00640 10 client.17370 handle_command_reply: tid=0 2024-09-05T12:45:41.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 client.17370 shutdown 2024-09-05T12:45:41.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 msgr2=0x556b777c1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:41.478 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777c1740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f30a4003e90 tx=0x7f30a402d970 comp rx=0 tx=0).stop 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 shutdown_connections 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b777d4380 0x556b777d6770 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777c4590 0x556b777c21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777b9cd0 0x556b777c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 --2- 172.21.15.71:0/1961564247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777b93b0 0x556b777c1740 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 >> 172.21.15.71:0/1961564247 conn(0x556b777b5940 msgr2=0x556b777b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.476+0000 7f3093e00640 0 client.17370 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.477+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 shutdown_connections 2024-09-05T12:45:41.479 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.477+0000 7f30bc244740 1 -- 172.21.15.71:0/1961564247 wait complete. 2024-09-05T12:45:41.480 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.477+0000 7f30bc244740 20 client.17370 trim_cache size 0 max 16384 2024-09-05T12:45:41.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.516+0000 7f30bc244740 1 Processor -- start 2024-09-05T12:45:41.518 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.516+0000 7f30bc244740 1 -- start start 2024-09-05T12:45:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.516+0000 7f30bc244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777bbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.516+0000 7f30bc244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30bc244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b777dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.519 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30bc244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407e460 con 0x556b777dbfd0 2024-09-05T12:45:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30bc244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407ffa0 con 0x556b777c4590 2024-09-05T12:45:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30bc244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407eb30 con 0x556b777b9cd0 2024-09-05T12:45:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30b2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777bbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30b2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777bbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37752/0 (socket says 172.21.15.71:37752) 2024-09-05T12:45:41.520 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.517+0000 7f30b2a00640 1 -- 172.21.15.71:0/1179041970 learned_addr learned my addr 172.21.15.71:0/1179041970 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2000640 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2a00640 1 -- 172.21.15.71:0/1179041970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 msgr2=0x556b777c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2a00640 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2a00640 1 -- 172.21.15.71:0/1179041970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 msgr2=0x556b777dc430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2a00640 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b777dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2a00640 1 -- 172.21.15.71:0/1179041970 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b777c4010 con 0x556b777b9cd0 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2000640 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:45:41.521 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.518+0000 7f30b2a00640 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777bbe60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f30a4046590 tx=0x7f30a4004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ce409462ac6a89 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.519+0000 7f309ea00640 1 -- 172.21.15.71:0/1179041970 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4046ab0 con 0x556b777b9cd0 2024-09-05T12:45:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.519+0000 7f309ea00640 1 -- 172.21.15.71:0/1179041970 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a4037070 con 0x556b777b9cd0 2024-09-05T12:45:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.519+0000 7f309ea00640 1 -- 172.21.15.71:0/1179041970 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4033da0 con 0x556b777b9cd0 2024-09-05T12:45:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.520+0000 7f30bc244740 1 -- 172.21.15.71:0/1179041970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 msgr2=0x556b777bbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.520+0000 7f30bc244740 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777bbe60 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f30a4046590 tx=0x7f30a4004630 comp rx=0 tx=0).stop 2024-09-05T12:45:41.522 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.520+0000 7f30bc244740 1 -- 172.21.15.71:0/1179041970 shutdown_connections 2024-09-05T12:45:41.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.520+0000 7f30bc244740 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b777dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.520+0000 7f30bc244740 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.523 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.521+0000 7f30bc244740 1 --2- 172.21.15.71:0/1179041970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777bbe60 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.521+0000 7f30bc244740 1 -- 172.21.15.71:0/1179041970 >> 172.21.15.71:0/1179041970 conn(0x556b777b5940 msgr2=0x556b777c2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.521+0000 7f30bc244740 1 -- 172.21.15.71:0/1179041970 shutdown_connections 2024-09-05T12:45:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.521+0000 7f30bc244740 1 -- 172.21.15.71:0/1179041970 wait complete. 2024-09-05T12:45:41.524 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.523+0000 7f30bc244740 1 -- start start 2024-09-05T12:45:41.526 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.523+0000 7f30bc244740 1 Processor -- start 2024-09-05T12:45:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.524+0000 7f30bc244740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.524+0000 7f30bc244740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.524+0000 7f30bc244740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b77836f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.525+0000 7f30b2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.525+0000 7f30b2a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:37756/0 (socket says 172.21.15.71:37756) 2024-09-05T12:45:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30bc244740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407f8d0 con 0x556b777dbfd0 2024-09-05T12:45:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30bc244740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407f200 con 0x556b777c4590 2024-09-05T12:45:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30bc244740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30b407c460 con 0x556b777b9cd0 2024-09-05T12:45:41.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30bc244740 10 client.0 _ll_register_callbacks cb 0x7f30a4004180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:41.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30b2a00640 1 -- 172.21.15.71:0/2243781523 learned_addr learned my addr 172.21.15.71:0/2243781523 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:41.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30b3400640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b77836f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.526+0000 7f30b2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.527+0000 7f30b2000640 1 -- 172.21.15.71:0/2243781523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 msgr2=0x556b777c8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.528+0000 7f30b2000640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777c8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.528+0000 7f30b2000640 1 -- 172.21.15.71:0/2243781523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 msgr2=0x556b77836f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.528+0000 7f30b2000640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b77836f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.528+0000 7f30b2000640 1 -- 172.21.15.71:0/2243781523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b77837600 con 0x556b777c4590 2024-09-05T12:45:41.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.528+0000 7f30b2a00640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:45:41.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.529+0000 7f30b2000640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c9260 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f30a00ed440 tx=0x7f30a002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f778cbf1594b584a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:41.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.530+0000 7f3093e00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:45:41.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.531+0000 7f3093e00640 1 -- 172.21.15.71:0/2243781523 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a004a040 con 0x556b777c4590 2024-09-05T12:45:41.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.531+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30a4037070 con 0x556b777c4590 2024-09-05T12:45:41.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.532+0000 7f3093e00640 1 -- 172.21.15.71:0/2243781523 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f30a00ba510 con 0x556b777c4590 2024-09-05T12:45:41.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.532+0000 7f3093e00640 1 -- 172.21.15.71:0/2243781523 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f30a004a040 con 0x556b777c4590 2024-09-05T12:45:41.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.533+0000 7f30bc244740 10 client.27380 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:41.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.533+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x556b7783fa70 con 0x556b777c4590 2024-09-05T12:45:41.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.534+0000 7f3093e00640 1 -- 172.21.15.71:0/2243781523 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30a4037070 con 0x556b777c4590 2024-09-05T12:45:41.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f3093e00640 1 -- 172.21.15.71:0/2243781523 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f30a00b87f0 con 0x556b777c4590 2024-09-05T12:45:41.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f30bc244740 10 client.27380 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:41.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f30bc244740 10 client.27380 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:45:41.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f30bc244740 20 client.27380 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:41.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f30bc244740 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b77840210 0x556b77842630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:41.537 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f30bc244740 4 client.27380 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.8, "await_for": 45.0}] 2024-09-05T12:45:41.538 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.535+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 152.8, "await_for": 45.0}) -- 0x556b7772cc60 con 0x556b77840210 2024-09-05T12:45:41.539 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.537+0000 7f30b2a00640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b77840210 0x556b77842630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:41.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.539+0000 7f30b2a00640 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b77840210 0x556b77842630 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d3c5b56d64c812 server_cookie=9a1ecd6a5008509 in_seq=0 out_seq=0 2024-09-05T12:45:41.541 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:41.539+0000 7f3093e00640 10 client.27380 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:41.837 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86433 15.74 MB/sec execute 155 sec latency 347.550 ms 2024-09-05T12:45:42.085 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.083+0000 7f3093e00640 1 -- 172.21.15.71:0/2243781523 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556b7772cc60 con 0x556b77840210 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.083+0000 7f3093e00640 10 client.27380 handle_command_reply: tid=0 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.083+0000 7f30bc244740 1 client.27380 shutdown 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.083+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 msgr2=0x556b777c9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.083+0000 7f30bc244740 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c9260 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f30a00ed440 tx=0x7f30a002a420 comp rx=0 tx=0).stop 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 shutdown_connections 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556b77840210 0x556b77842630 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b777dbfd0 0x556b77836f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b777c4590 0x556b777c9260 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f30a00ed440 tx=0x7f30a002a420 comp rx=0 tx=0).stop 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 --2- 172.21.15.71:0/2243781523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b777b9cd0 0x556b777c8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:42.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 >> 172.21.15.71:0/2243781523 conn(0x556b777b5940 msgr2=0x556b777c1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:42.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f3093e00640 0 client.27380 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:42.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 shutdown_connections 2024-09-05T12:45:42.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 1 -- 172.21.15.71:0/2243781523 wait complete. 2024-09-05T12:45:42.087 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.084+0000 7f30bc244740 20 client.27380 trim_cache size 0 max 16384 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 76, 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 58, 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "25f6eb5a": { 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 58, 2024-09-05T12:45:42.088 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 152.8, 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:45:42.089 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30980a2cd0 msgr2=0x7f30980a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30980a2cd0 0x7f30980a5190 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00f590 tx=0x7f30ac004a40 comp rx=0 tx=0).stop 2024-09-05T12:45:42.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 msgr2=0x7f30b41a77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 0x7f30b41a77f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30a002aa10 tx=0x7f30a0005230 comp rx=0 tx=0).stop 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 shutdown_connections 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30b411b140 0x7f30b41abb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30b4109830 0x7f30b41a77f0 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f30a002aa10 tx=0x7f30a0005230 comp rx=0 tx=0).stop 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f30980a2cd0 0x7f30980a5190 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 --2- 172.21.15.71:0/833765577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30b4109450 0x7f30b41a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.086+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 >> 172.21.15.71:0/833765577 conn(0x7f30b406e140 msgr2=0x7f30b411be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.087+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 shutdown_connections 2024-09-05T12:45:42.091 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:45:42.087+0000 7f30b9600640 1 -- 172.21.15.71:0/833765577 wait complete. 2024-09-05T12:45:42.547 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:42 smithi071 ceph-mon[29845]: pgmap v566: 129 pgs: 129 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.6 MiB/s wr, 428 op/s 2024-09-05T12:45:42.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:42 smithi123 ceph-mon[34321]: pgmap v566: 129 pgs: 129 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.6 MiB/s wr, 428 op/s 2024-09-05T12:45:42.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:42 smithi174 ceph-mon[34313]: pgmap v566: 129 pgs: 129 active+clean; 861 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.6 MiB/s wr, 428 op/s 2024-09-05T12:45:42.837 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.64 MB/sec execute 156 sec latency 1280.776 ms 2024-09-05T12:45:43.391 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '25f6eb5a', quiesce took 6.4 seconds. Will release after: 26.4 2024-09-05T12:45:43.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.54 MB/sec execute 157 sec latency 2280.905 ms 2024-09-05T12:45:44.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:44 smithi071 ceph-mon[29845]: pgmap v567: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-05T12:45:44.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:44 smithi123 ceph-mon[34321]: pgmap v567: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-05T12:45:44.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:44 smithi174 ceph-mon[34313]: pgmap v567: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.8 MiB/s wr, 371 op/s 2024-09-05T12:45:44.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.44 MB/sec execute 158 sec latency 3281.009 ms 2024-09-05T12:45:45.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.35 MB/sec execute 159 sec latency 4281.155 ms 2024-09-05T12:45:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 -- 172.21.15.71:0/3136446901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42241090a0 msgr2=0x7f4224110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 --2- 172.21.15.71:0/3136446901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42241090a0 0x7f4224110bf0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 -- 172.21.15.71:0/3136446901 shutdown_connections 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 --2- 172.21.15.71:0/3136446901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f42241090a0 0x7f4224110bf0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 --2- 172.21.15.71:0/3136446901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4224104900 0x7f4224108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 --2- 172.21.15.71:0/3136446901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4224103f50 0x7f4224104330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 -- 172.21.15.71:0/3136446901 >> 172.21.15.71:0/3136446901 conn(0x7f42240ff9e0 msgr2=0x7f4224101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.432+0000 7f422a200640 1 -- 172.21.15.71:0/3136446901 shutdown_connections 2024-09-05T12:45:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.433+0000 7f422a200640 1 -- 172.21.15.71:0/3136446901 wait complete. 2024-09-05T12:45:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.433+0000 7f422a200640 1 Processor -- start 2024-09-05T12:45:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 -- start start 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 0x7f422419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42241090a0 0x7f422419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f422419ee00 0x7f422419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42241140f0 con 0x7f4224103f50 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4224113f70 con 0x7f422419ee00 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f422a200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4224114270 con 0x7f42241090a0 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f4223400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 0x7f422419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f4223400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 0x7f422419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56188/0 (socket says 172.21.15.71:56188) 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f4223400640 1 -- 172.21.15.71:0/1335015985 learned_addr learned my addr 172.21.15.71:0/1335015985 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f421be00640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42241090a0 0x7f422419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.434+0000 7f4223e00640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f422419ee00 0x7f422419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223400640 1 -- 172.21.15.71:0/1335015985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42241090a0 msgr2=0x7f422419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223400640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42241090a0 0x7f422419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223400640 1 -- 172.21.15.71:0/1335015985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f422419ee00 msgr2=0x7f422419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223400640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f422419ee00 0x7f422419f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223400640 1 -- 172.21.15.71:0/1335015985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422419f8f0 con 0x7f4224103f50 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223e00640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f422419ee00 0x7f422419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4223400640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 0x7f422419a540 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9f0 tx=0x7f421400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4220c00640 1 -- 172.21.15.71:0/1335015985 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421400cd80 con 0x7f4224103f50 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4220c00640 1 -- 172.21.15.71:0/1335015985 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4214005500 con 0x7f4224103f50 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422419fbe0 con 0x7f4224103f50 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f4220c00640 1 -- 172.21.15.71:0/1335015985 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214010690 con 0x7f4224103f50 2024-09-05T12:45:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.435+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42241a7380 con 0x7f4224103f50 2024-09-05T12:45:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.438+0000 7f4220c00640 1 -- 172.21.15.71:0/1335015985 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f42140046c0 con 0x7f4224103f50 2024-09-05T12:45:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.438+0000 7f422d0d0740 1 Processor -- start 2024-09-05T12:45:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 -- start start 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c321e0 0x563ff8c325c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c3ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563ff8b98100 con 0x563ff8c321e0 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ff8b401b0 con 0x563ff8c3d3a0 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f422d0d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ff8b940c0 con 0x563ff8c32b00 2024-09-05T12:45:46.442 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f421be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f4223e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c321e0 0x563ff8c325c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f421be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55866/0 (socket says 172.21.15.71:55866) 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f421be00640 1 -- 172.21.15.71:0/1619592957 learned_addr learned my addr 172.21.15.71:0/1619592957 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.439+0000 7f4223400640 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c3ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f421be00640 1 -- 172.21.15.71:0/1619592957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 msgr2=0x563ff8c3ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f421be00640 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c3ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f421be00640 1 -- 172.21.15.71:0/1619592957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c321e0 msgr2=0x563ff8c325c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f421be00640 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c321e0 0x563ff8c325c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.443 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f421be00640 1 -- 172.21.15.71:0/1619592957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff88ab210 con 0x563ff8c3d3a0 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f4220c00640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f42080a2da0 0x7f42080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f421be00640 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3f790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f420c009c20 tx=0x7f420c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a462fa282aa4913 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f4220c00640 1 -- 172.21.15.71:0/1335015985 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f42140cfcd0 con 0x7f4224103f50 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f4223e00640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f42080a2da0 0x7f42080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f4213e00640 1 -- 172.21.15.71:0/1619592957 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c01e070 con 0x563ff8c3d3a0 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f4213e00640 1 -- 172.21.15.71:0/1619592957 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c009dd0 con 0x563ff8c3d3a0 2024-09-05T12:45:46.444 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.440+0000 7f4213e00640 1 -- 172.21.15.71:0/1619592957 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c018730 con 0x563ff8c3d3a0 2024-09-05T12:45:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 -- 172.21.15.71:0/1619592957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 msgr2=0x563ff8c3f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f4223e00640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f42080a2da0 0x7f42080a5260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f421c02f540 tx=0x7f421c02f310 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3f790 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f420c009c20 tx=0x7f420c011810 comp rx=0 tx=0).stop 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 -- 172.21.15.71:0/1619592957 shutdown_connections 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3f790 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c3ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1619592957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c321e0 0x563ff8c325c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 -- 172.21.15.71:0/1619592957 >> 172.21.15.71:0/1619592957 conn(0x563ff8c2e750 msgr2=0x563ff8c2eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 -- 172.21.15.71:0/1619592957 shutdown_connections 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.441+0000 7f422d0d0740 1 -- 172.21.15.71:0/1619592957 wait complete. 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 -- start start 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 Processor -- start 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c3a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c3a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563ff8c3ffd0 con 0x563ff8c32b00 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ff8b21090 con 0x563ff8c321e0 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.442+0000 7f422d0d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ff8c40150 con 0x563ff8c3d3a0 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f422d0d0740 10 client.0 _ll_register_callbacks cb 0x563ff8b75f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c3a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c3a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:55882/0 (socket says 172.21.15.71:55882) 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f421be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c3a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 -- 172.21.15.71:0/1786654071 learned_addr learned my addr 172.21.15.71:0/1786654071 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223e00640 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 -- 172.21.15.71:0/1786654071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 msgr2=0x563ff8c3ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 -- 172.21.15.71:0/1786654071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 msgr2=0x563ff8c3a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c3a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 -- 172.21.15.71:0/1786654071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff8977620 con 0x563ff8c321e0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4223400640 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c3a400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4214098260 tx=0x7f4214004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb3aa822a301505c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.443+0000 7f4210c00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f4210c00640 1 -- 172.21.15.71:0/1786654071 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421409b760 con 0x563ff8c321e0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f4210c00640 1 -- 172.21.15.71:0/1786654071 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f42140a45e0 con 0x563ff8c321e0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420c009dd0 con 0x563ff8c321e0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f4210c00640 1 -- 172.21.15.71:0/1786654071 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f42140cd4c0 con 0x563ff8c321e0 2024-09-05T12:45:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f422d0d0740 10 client.27392 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563ff8c4bd30 con 0x563ff8c321e0 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f4210c00640 1 -- 172.21.15.71:0/1786654071 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c009dd0 con 0x563ff8c321e0 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f4210c00640 1 -- 172.21.15.71:0/1786654071 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f421409b250 con 0x563ff8c321e0 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f422d0d0740 10 client.27392 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f422d0d0740 10 client.27392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.444+0000 7f422d0d0740 20 client.27392 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.445+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8c4d1a0 0x563ff8c4f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.445+0000 7f422d0d0740 4 client.27392 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.445+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ff8ba5830 con 0x563ff8c4d1a0 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.445+0000 7f421be00640 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8c4d1a0 0x563ff8c4f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.445+0000 7f421be00640 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8c4d1a0 0x563ff8c4f5b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f52a54358a45dd server_cookie=ea5a7a5b8ac7f379 in_seq=0 out_seq=0 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.445+0000 7f4210c00640 10 client.27392 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f4210c00640 1 -- 172.21.15.71:0/1786654071 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ff8ba5830 con 0x563ff8c4d1a0 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f4210c00640 10 client.27392 handle_command_reply: tid=0 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 client.27392 shutdown 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 msgr2=0x563ff8c3a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c3a400 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4214098260 tx=0x7f4214004ba0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 shutdown_connections 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8c4d1a0 0x563ff8c4f5b0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c3ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c3a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1786654071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c3a400 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f4210c00640 0 client.27392 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.446+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 >> 172.21.15.71:0/1786654071 conn(0x563ff8c2e750 msgr2=0x563ff8c344b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.447+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 shutdown_connections 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.447+0000 7f422d0d0740 1 -- 172.21.15.71:0/1786654071 wait complete. 2024-09-05T12:45:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.447+0000 7f422d0d0740 20 client.27392 trim_cache size 0 max 16384 2024-09-05T12:45:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.461+0000 7f422d0d0740 1 Processor -- start 2024-09-05T12:45:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 -- start start 2024-09-05T12:45:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c54ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c55420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c37640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42241b10d0 con 0x563ff8c32b00 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42241b2c10 con 0x563ff8c321e0 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f422d0d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42241b17a0 con 0x563ff8c3d3a0 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c55420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c55420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56220/0 (socket says 172.21.15.71:56220) 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 -- 172.21.15.71:0/2489222009 learned_addr learned my addr 172.21.15.71:0/2489222009 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f4223400640 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c54ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f4223e00640 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 -- 172.21.15.71:0/2489222009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 msgr2=0x563ff8c37640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c37640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 -- 172.21.15.71:0/2489222009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 msgr2=0x563ff8c54ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c54ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.462+0000 7f421be00640 1 -- 172.21.15.71:0/2489222009 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff8c29d60 con 0x563ff8c32b00 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.463+0000 7f421be00640 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c55420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f420c014040 tx=0x7f420c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26308b69e19a29b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.463+0000 7f4213400640 1 -- 172.21.15.71:0/2489222009 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c029070 con 0x563ff8c32b00 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.463+0000 7f4213400640 1 -- 172.21.15.71:0/2489222009 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c005370 con 0x563ff8c32b00 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.463+0000 7f4213400640 1 -- 172.21.15.71:0/2489222009 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c01e040 con 0x563ff8c32b00 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 -- 172.21.15.71:0/2489222009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 msgr2=0x563ff8c55420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c55420 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f420c014040 tx=0x7f420c005f90 comp rx=0 tx=0).stop 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 -- 172.21.15.71:0/2489222009 shutdown_connections 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c3d3a0 0x563ff8c37640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c32b00 0x563ff8c55420 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 --2- 172.21.15.71:0/2489222009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c54ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 -- 172.21.15.71:0/2489222009 >> 172.21.15.71:0/2489222009 conn(0x563ff8c2e750 msgr2=0x563ff8c356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 -- 172.21.15.71:0/2489222009 shutdown_connections 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 -- 172.21.15.71:0/2489222009 wait complete. 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 -- start start 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.464+0000 7f422d0d0740 1 Processor -- start 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c416e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 0x563ff8c41c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f42241b2540 con 0x563ff8c3d3a0 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42241b1e70 con 0x563ff8c321e0 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42241af0d0 con 0x563ff8c32b00 2024-09-05T12:45:46.471 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f422d0d0740 10 client.0 _ll_register_callbacks cb 0x7f42140d3470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 0x563ff8c41c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 0x563ff8c41c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56226/0 (socket says 172.21.15.71:56226) 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 -- 172.21.15.71:0/1249316890 learned_addr learned my addr 172.21.15.71:0/1249316890 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f421be00640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223400640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 -- 172.21.15.71:0/1249316890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 msgr2=0x563ff8c416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 -- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 msgr2=0x563ff8c411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f4223e00640 1 -- 172.21.15.71:0/1249316890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff8cb6f60 con 0x563ff8c3d3a0 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.465+0000 7f421be00640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f4223e00640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 0x563ff8c41c20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937c1c85247ab045 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f4210c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f4210c00640 1 -- 172.21.15.71:0/1249316890 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c0318e0 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f4210c00640 1 -- 172.21.15.71:0/1249316890 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f421c041030 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420c005370 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f4210c00640 1 -- 172.21.15.71:0/1249316890 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f421c004410 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f422d0d0740 10 client.17391 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563ff8cb8250 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.466+0000 7f4210c00640 1 -- 172.21.15.71:0/1249316890 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f420c005370 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f4210c00640 1 -- 172.21.15.71:0/1249316890 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f421c0305c0 con 0x563ff8c3d3a0 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f422d0d0740 10 client.17391 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f422d0d0740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f422d0d0740 20 client.17391 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8cb89d0 0x563ff8cbadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f422d0d0740 4 client.17391 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:45:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ff8b9fb80 con 0x563ff8cb89d0 2024-09-05T12:45:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f4223400640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8cb89d0 0x563ff8cbadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f4210c00640 10 client.17391 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.467+0000 7f4223400640 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8cb89d0 0x563ff8cbadf0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b6d4aa91ace1859 server_cookie=bd560dcf43f3fbe2 in_seq=0 out_seq=0 2024-09-05T12:45:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f4210c00640 1 -- 172.21.15.71:0/1249316890 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ff8b9fb80 con 0x563ff8cb89d0 2024-09-05T12:45:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f4210c00640 10 client.17391 handle_command_reply: tid=0 2024-09-05T12:45:46.575 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 client.17391 shutdown 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 msgr2=0x563ff8c41c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 0x563ff8c41c20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f421c009ca0 tx=0x7f421c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 shutdown_connections 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563ff8cb89d0 0x563ff8cbadf0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563ff8c3d3a0 0x563ff8c41c20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ff8c32b00 0x563ff8c416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f4210c00640 0 client.17391 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 --2- 172.21.15.71:0/1249316890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ff8c321e0 0x563ff8c411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 >> 172.21.15.71:0/1249316890 conn(0x563ff8c2e750 msgr2=0x563ff8c3a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 shutdown_connections 2024-09-05T12:45:46.576 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 1 -- 172.21.15.71:0/1249316890 wait complete. 2024-09-05T12:45:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.573+0000 7f422d0d0740 20 client.17391 trim_cache size 0 max 16384 2024-09-05T12:45:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:45:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:45:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:45:46.577 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f42080a2da0 msgr2=0x7f42080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f42080a2da0 0x7f42080a5260 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f421c02f540 tx=0x7f421c02f310 comp rx=0 tx=0).stop 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 msgr2=0x7f422419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 0x7f422419a540 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f421400e9f0 tx=0x7f421400edf0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 shutdown_connections 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f422419ee00 0x7f422419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42241090a0 0x7f422419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f42080a2da0 0x7f42080a5260 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 --2- 172.21.15.71:0/1335015985 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4224103f50 0x7f422419a540 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:46.578 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 >> 172.21.15.71:0/1335015985 conn(0x7f42240ff9e0 msgr2=0x7f4224100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 shutdown_connections 2024-09-05T12:45:46.579 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:46.574+0000 7f422a200640 1 -- 172.21.15.71:0/1335015985 wait complete. 2024-09-05T12:45:46.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:46 smithi071 ceph-mon[29845]: pgmap v568: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-09-05T12:45:46.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:46 smithi123 ceph-mon[34321]: pgmap v568: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-09-05T12:45:46.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:46 smithi174 ceph-mon[34313]: pgmap v568: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 329 op/s 2024-09-05T12:45:46.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.25 MB/sec execute 160 sec latency 5281.259 ms 2024-09-05T12:45:47.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.15 MB/sec execute 161 sec latency 6281.403 ms 2024-09-05T12:45:48.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:48 smithi071 ceph-mon[29845]: pgmap v569: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.5 MiB/s wr, 390 op/s 2024-09-05T12:45:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:48 smithi123 ceph-mon[34321]: pgmap v569: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.5 MiB/s wr, 390 op/s 2024-09-05T12:45:48.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:48 smithi174 ceph-mon[34313]: pgmap v569: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.5 MiB/s wr, 390 op/s 2024-09-05T12:45:48.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 15.06 MB/sec execute 162 sec latency 7281.574 ms 2024-09-05T12:45:49.838 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.97 MB/sec execute 163 sec latency 8281.674 ms 2024-09-05T12:45:50.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:50 smithi071 ceph-mon[29845]: pgmap v570: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-09-05T12:45:50.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:50 smithi123 ceph-mon[34321]: pgmap v570: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-09-05T12:45:50.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:50 smithi174 ceph-mon[34313]: pgmap v570: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.8 MiB/s wr, 252 op/s 2024-09-05T12:45:50.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.88 MB/sec execute 164 sec latency 9281.776 ms 2024-09-05T12:45:51.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.79 MB/sec execute 165 sec latency 10281.903 ms 2024-09-05T12:45:52.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:52 smithi071 ceph-mon[29845]: pgmap v571: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.7 MiB/s wr, 263 op/s 2024-09-05T12:45:52.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:52 smithi123 ceph-mon[34321]: pgmap v571: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.7 MiB/s wr, 263 op/s 2024-09-05T12:45:52.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:52 smithi174 ceph-mon[34313]: pgmap v571: 129 pgs: 129 active+clean; 867 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.7 MiB/s wr, 263 op/s 2024-09-05T12:45:52.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.70 MB/sec execute 166 sec latency 11281.974 ms 2024-09-05T12:45:53.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.61 MB/sec execute 167 sec latency 12282.032 ms 2024-09-05T12:45:54.678 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:54 smithi071 ceph-mon[29845]: pgmap v572: 129 pgs: 129 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-09-05T12:45:54.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:54 smithi123 ceph-mon[34321]: pgmap v572: 129 pgs: 129 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-09-05T12:45:54.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:54 smithi174 ceph-mon[34313]: pgmap v572: 129 pgs: 129 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-09-05T12:45:54.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.52 MB/sec execute 168 sec latency 13282.122 ms 2024-09-05T12:45:55.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.44 MB/sec execute 169 sec latency 14282.198 ms 2024-09-05T12:45:56.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:56 smithi123 ceph-mon[34321]: pgmap v573: 129 pgs: 129 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 785 KiB/s wr, 100 op/s 2024-09-05T12:45:56.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:56 smithi174 ceph-mon[34313]: pgmap v573: 129 pgs: 129 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 785 KiB/s wr, 100 op/s 2024-09-05T12:45:56.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.35 MB/sec execute 170 sec latency 15282.349 ms 2024-09-05T12:45:56.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:56 smithi071 ceph-mon[29845]: pgmap v573: 129 pgs: 129 active+clean; 868 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 785 KiB/s wr, 100 op/s 2024-09-05T12:45:57.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.27 MB/sec execute 171 sec latency 16282.455 ms 2024-09-05T12:45:58.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:45:58 smithi123 ceph-mon[34321]: pgmap v574: 129 pgs: 129 active+clean; 886 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-09-05T12:45:58.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:45:58 smithi174 ceph-mon[34313]: pgmap v574: 129 pgs: 129 active+clean; 886 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-09-05T12:45:58.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.19 MB/sec execute 172 sec latency 17282.606 ms 2024-09-05T12:45:58.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:45:58 smithi071 ceph-mon[29845]: pgmap v574: 129 pgs: 129 active+clean; 886 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.1 MiB/s wr, 150 op/s 2024-09-05T12:45:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 -- 172.21.15.71:0/3391545289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 msgr2=0x7f8f3c103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 --2- 172.21.15.71:0/3391545289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c103f40 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009930 tx=0x7f8f2402a420 comp rx=0 tx=0).stop 2024-09-05T12:45:59.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 -- 172.21.15.71:0/3391545289 shutdown_connections 2024-09-05T12:45:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 --2- 172.21.15.71:0/3391545289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f3c104480 0x7f8f3c1109b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 --2- 172.21.15.71:0/3391545289 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c103f40 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 --2- 172.21.15.71:0/3391545289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c109b80 0x7f8f3c109f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 -- 172.21.15.71:0/3391545289 >> 172.21.15.71:0/3391545289 conn(0x7f8f3c0ff7c0 msgr2=0x7f8f3c101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 -- 172.21.15.71:0/3391545289 shutdown_connections 2024-09-05T12:45:59.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.699+0000 7f8f41000640 1 -- 172.21.15.71:0/3391545289 wait complete. 2024-09-05T12:45:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 Processor -- start 2024-09-05T12:45:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 -- start start 2024-09-05T12:45:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f3c104480 0x7f8f3c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c109b80 0x7f8f3c19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.703 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c113f80 con 0x7f8f3c103ae0 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c113e00 con 0x7f8f3c109b80 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.700+0000 7f8f41000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c114100 con 0x7f8f3c104480 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46590/0 (socket says 172.21.15.71:46590) 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 -- 172.21.15.71:0/644803760 learned_addr learned my addr 172.21.15.71:0/644803760 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f32a00640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f3c104480 0x7f8f3c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3be00640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c109b80 0x7f8f3c19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 -- 172.21.15.71:0/644803760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f3c104480 msgr2=0x7f8f3c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f3c104480 0x7f8f3c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 -- 172.21.15.71:0/644803760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c109b80 msgr2=0x7f8f3c19ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c109b80 0x7f8f3c19ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.704 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 -- 172.21.15.71:0/644803760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c19f460 con 0x7f8f3c103ae0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.701+0000 7f8f3b400640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c19a470 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00e9f0 tx=0x7f8f2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.702+0000 7f8f38c00640 1 -- 172.21.15.71:0/644803760 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00cd80 con 0x7f8f3c103ae0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.702+0000 7f8f38c00640 1 -- 172.21.15.71:0/644803760 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f2c010910 con 0x7f8f3c103ae0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.702+0000 7f8f38c00640 1 -- 172.21.15.71:0/644803760 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c018990 con 0x7f8f3c103ae0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.702+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c19f750 con 0x7f8f3c103ae0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.702+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c1a6fe0 con 0x7f8f3c103ae0 2024-09-05T12:45:59.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f38c00640 1 -- 172.21.15.71:0/644803760 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8f2c010ac0 con 0x7f8f3c103ae0 2024-09-05T12:45:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 Processor -- start 2024-09-05T12:45:59.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 -- start start 2024-09-05T12:45:59.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adaa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560e1ad10100 con 0x560e1adaacd0 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e1acb81b0 con 0x560e1adaa3b0 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f45651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e1ad0c0c0 con 0x560e1adb5570 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.705+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adaa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46592/0 (socket says 172.21.15.71:46592) 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adaa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48696/0 (socket says 172.21.15.71:48696) 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3b400640 1 -- 172.21.15.71:0/1798949341 learned_addr learned my addr 172.21.15.71:0/1798949341 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f32a00640 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 -- 172.21.15.71:0/1798949341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 msgr2=0x560e1adb7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adb7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 -- 172.21.15.71:0/1798949341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 msgr2=0x560e1adb5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 -- 172.21.15.71:0/1798949341 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1aa23210 con 0x560e1adaa3b0 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3b400640 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f3be00640 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adaa790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009f00 tx=0x7f8f34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faef0efeddcaa63e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.706+0000 7f8f38c00640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8f200a2d70 0x7f8f200a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f30c00640 1 -- 172.21.15.71:0/1798949341 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34004d20 con 0x560e1adaa3b0 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f38c00640 1 -- 172.21.15.71:0/644803760 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f8f2c0cfa10 con 0x7f8f3c103ae0 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f3b400640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8f200a2d70 0x7f8f200a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f30c00640 1 -- 172.21.15.71:0/1798949341 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f34004030 con 0x560e1adaa3b0 2024-09-05T12:45:59.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f30c00640 1 -- 172.21.15.71:0/1798949341 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f34005260 con 0x560e1adaa3b0 2024-09-05T12:45:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 -- 172.21.15.71:0/1798949341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 msgr2=0x560e1adaa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f3b400640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8f200a2d70 0x7f8f200a5230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c6a0 tx=0x7f8f2c0d3410 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adaa790 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009f00 tx=0x7f8f34011810 comp rx=0 tx=0).stop 2024-09-05T12:45:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 -- 172.21.15.71:0/1798949341 shutdown_connections 2024-09-05T12:45:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adb7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 --2- 172.21.15.71:0/1798949341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adaa790 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.707+0000 7f8f45651740 1 -- 172.21.15.71:0/1798949341 >> 172.21.15.71:0/1798949341 conn(0x560e1ada6920 msgr2=0x560e1ada6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.708+0000 7f8f45651740 1 -- 172.21.15.71:0/1798949341 shutdown_connections 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.708+0000 7f8f45651740 1 -- 172.21.15.71:0/1798949341 wait complete. 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.708+0000 7f8f45651740 1 -- start start 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.708+0000 7f8f45651740 1 Processor -- start 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adb30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560e1adb7ff0 con 0x560e1adaacd0 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e1ac99090 con 0x560e1adb5570 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e1adb8170 con 0x560e1adaa3b0 2024-09-05T12:45:59.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f45651740 10 client.0 _ll_register_callbacks cb 0x560e1acedf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46606/0 (socket says 172.21.15.71:46606) 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 -- 172.21.15.71:0/3952171334 learned_addr learned my addr 172.21.15.71:0/3952171334 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f3be00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f3be00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adb30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 -- 172.21.15.71:0/3952171334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 msgr2=0x560e1adb2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adb2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 -- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 msgr2=0x560e1adb30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adb30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.709+0000 7f8f32a00640 1 -- 172.21.15.71:0/3952171334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1aaef620 con 0x560e1adaacd0 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f32a00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb2ba0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009930 tx=0x7f8f240043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a41677c4ee4ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f29600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f29600640 1 -- 172.21.15.71:0/3952171334 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24035040 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f29600640 1 -- 172.21.15.71:0/3952171334 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f8f24005450 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f34004030 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f29600640 1 -- 172.21.15.71:0/3952171334 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8f2404f050 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f45651740 10 client.17403 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.710+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560e1adc3d80 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f29600640 1 -- 172.21.15.71:0/3952171334 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f34004030 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f29600640 1 -- 172.21.15.71:0/3952171334 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f8f24003850 con 0x560e1adaacd0 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f45651740 10 client.17403 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f45651740 10 client.17403 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f45651740 20 client.17403 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f45651740 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1adc5240 0x560e1adc7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f45651740 4 client.17403 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e1ad1d830 con 0x560e1adc5240 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.711+0000 7f8f3be00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1adc5240 0x560e1adc7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.712+0000 7f8f3be00640 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1adc5240 0x560e1adc7630 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc8da0b1fa2e580d server_cookie=907c8d570056bfe6 in_seq=0 out_seq=0 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.712+0000 7f8f29600640 10 client.17403 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f29600640 1 -- 172.21.15.71:0/3952171334 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e1ad1d830 con 0x560e1adc5240 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f29600640 10 client.17403 handle_command_reply: tid=0 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 client.17403 shutdown 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 msgr2=0x560e1adb2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb2ba0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009930 tx=0x7f8f240043b0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 shutdown_connections 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1adc5240 0x560e1adc7630 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adb30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb2ba0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 --2- 172.21.15.71:0/3952171334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f29600640 0 client.17403 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 >> 172.21.15.71:0/3952171334 conn(0x560e1ada6920 msgr2=0x560e1adac680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 shutdown_connections 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 1 -- 172.21.15.71:0/3952171334 wait complete. 2024-09-05T12:45:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.713+0000 7f8f45651740 20 client.17403 trim_cache size 0 max 16384 2024-09-05T12:45:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.728+0000 7f8f45651740 1 Processor -- start 2024-09-05T12:45:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.728+0000 7f8f45651740 1 -- start start 2024-09-05T12:45:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f45651740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f45651740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f45651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adccf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f45651740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b1010 con 0x560e1adaacd0 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f45651740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b2b50 con 0x560e1adaa3b0 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f45651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b16e0 con 0x560e1adb5570 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46614/0 (socket says 172.21.15.71:46614) 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 -- 172.21.15.71:0/4048835034 learned_addr learned my addr 172.21.15.71:0/4048835034 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f3be00640 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f3be00640 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 -- 172.21.15.71:0/4048835034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 msgr2=0x560e1adccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adccf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 -- 172.21.15.71:0/4048835034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 msgr2=0x560e1adb2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adb2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 -- 172.21.15.71:0/4048835034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1ada61b0 con 0x560e1adaacd0 2024-09-05T12:45:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.729+0000 7f8f32a00640 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb44c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8f2402a940 tx=0x7f8f24003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4f8a9f80f3129 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f2be00640 1 -- 172.21.15.71:0/4048835034 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24059070 con 0x560e1adaacd0 2024-09-05T12:45:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f2be00640 1 -- 172.21.15.71:0/4048835034 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f24041070 con 0x560e1adaacd0 2024-09-05T12:45:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f2be00640 1 -- 172.21.15.71:0/4048835034 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24035030 con 0x560e1adaacd0 2024-09-05T12:45:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 -- 172.21.15.71:0/4048835034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 msgr2=0x560e1adb44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb44c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8f2402a940 tx=0x7f8f24003fe0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 -- 172.21.15.71:0/4048835034 shutdown_connections 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adb5570 0x560e1adccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adb44c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 --2- 172.21.15.71:0/4048835034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adaa3b0 0x560e1adb2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.730+0000 7f8f45651740 1 -- 172.21.15.71:0/4048835034 >> 172.21.15.71:0/4048835034 conn(0x560e1ada6920 msgr2=0x560e1adb3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 -- 172.21.15.71:0/4048835034 shutdown_connections 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 -- 172.21.15.71:0/4048835034 wait complete. 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 -- start start 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 Processor -- start 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adbdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.736 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adaf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adafa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b2480 con 0x560e1adaacd0 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1b1db0 con 0x560e1adb5570 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.731+0000 7f8f45651740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c1af010 con 0x560e1adaa3b0 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f45651740 10 client.0 _ll_register_callbacks cb 0x7f8f24009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adaf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adaf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46630/0 (socket says 172.21.15.71:46630) 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 -- 172.21.15.71:0/243293467 learned_addr learned my addr 172.21.15.71:0/243293467 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f3be00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adafa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f3be00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adbdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 -- 172.21.15.71:0/243293467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 msgr2=0x560e1adbdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adbdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.737 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 -- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 msgr2=0x560e1adafa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adafa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 -- 172.21.15.71:0/243293467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1adb0270 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f32a00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adaf460 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f24031550 tx=0x7f8f24009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a0e756a4e68266 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.732+0000 7f8f29600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f29600640 1 -- 172.21.15.71:0/243293467 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24035070 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f29600640 1 -- 172.21.15.71:0/243293467 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9438+0+0 (secure 0 0 0) 0x7f8f240315b0 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f24041070 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f29600640 1 -- 172.21.15.71:0/243293467 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8f24059040 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 10 client.17409 fetch_fsmap learned FSMap version 76 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560e1ae387c0 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f29600640 1 -- 172.21.15.71:0/243293467 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f24041070 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f29600640 1 -- 172.21.15.71:0/243293467 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f8f24004a00 con 0x560e1adaacd0 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 10 client.17409 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:45:59.738 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 10 client.17409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 20 client.17409 populate_metadata read hostname 'smithi071' 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1ae38e10 0x560e1ae3b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 4 client.17409 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.733+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e1ad17b80 con 0x560e1ae38e10 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.734+0000 7f8f3be00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1ae38e10 0x560e1ae3b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.734+0000 7f8f3be00640 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1ae38e10 0x560e1ae3b200 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d05b122ad7e7ad server_cookie=916adee538f5fde6 in_seq=0 out_seq=0 2024-09-05T12:45:59.739 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.734+0000 7f8f29600640 10 client.17409 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f29600640 1 -- 172.21.15.71:0/243293467 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e1ad17b80 con 0x560e1ae38e10 2024-09-05T12:45:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f29600640 10 client.17409 handle_command_reply: tid=0 2024-09-05T12:45:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 client.17409 shutdown 2024-09-05T12:45:59.771 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 msgr2=0x560e1adaf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adaf460 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8f24031550 tx=0x7f8f24009290 comp rx=0 tx=0).stop 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 shutdown_connections 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560e1ae38e10 0x560e1ae3b200 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e1adb5570 0x560e1adafa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e1adaacd0 0x560e1adaf460 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 --2- 172.21.15.71:0/243293467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e1adaa3b0 0x560e1adbdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 >> 172.21.15.71:0/243293467 conn(0x560e1ada6920 msgr2=0x560e1ada59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f29600640 0 client.17409 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 shutdown_connections 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.770+0000 7f8f45651740 1 -- 172.21.15.71:0/243293467 wait complete. 2024-09-05T12:45:59.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f45651740 20 client.17409 trim_cache size 0 max 16384 2024-09-05T12:45:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:45:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:45:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:45:59.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8f200a2d70 msgr2=0x7f8f200a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8f200a2d70 0x7f8f200a5230 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c6a0 tx=0x7f8f2c0d3410 comp rx=0 tx=0).stop 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 msgr2=0x7f8f3c19a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c19a470 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00e9f0 tx=0x7f8f2c00eeb0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 shutdown_connections 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f3c109b80 0x7f8f3c19ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f3c104480 0x7f8f3c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8f200a2d70 0x7f8f200a5230 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 --2- 172.21.15.71:0/644803760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f3c103ae0 0x7f8f3c19a470 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.771+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 >> 172.21.15.71:0/644803760 conn(0x7f8f3c0ff7c0 msgr2=0x7f8f3c0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.772+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 shutdown_connections 2024-09-05T12:45:59.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:45:59.772+0000 7f8f41000640 1 -- 172.21.15.71:0/644803760 wait complete. 2024-09-05T12:45:59.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.10 MB/sec execute 173 sec latency 18282.745 ms 2024-09-05T12:46:00.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:00 smithi123 ceph-mon[34321]: pgmap v575: 129 pgs: 129 active+clean; 887 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.6 MiB/s wr, 94 op/s 2024-09-05T12:46:00.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:00 smithi174 ceph-mon[34313]: pgmap v575: 129 pgs: 129 active+clean; 887 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.6 MiB/s wr, 94 op/s 2024-09-05T12:46:00.839 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 14.02 MB/sec execute 174 sec latency 19282.852 ms 2024-09-05T12:46:00.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:00 smithi071 ceph-mon[29845]: pgmap v575: 129 pgs: 129 active+clean; 887 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.6 MiB/s wr, 94 op/s 2024-09-05T12:46:01.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.94 MB/sec execute 175 sec latency 20282.927 ms 2024-09-05T12:46:02.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:02 smithi174 ceph-mon[34313]: pgmap v576: 129 pgs: 129 active+clean; 879 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.7 MiB/s wr, 109 op/s 2024-09-05T12:46:02.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.86 MB/sec execute 176 sec latency 21283.010 ms 2024-09-05T12:46:02.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:02 smithi071 ceph-mon[29845]: pgmap v576: 129 pgs: 129 active+clean; 879 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.7 MiB/s wr, 109 op/s 2024-09-05T12:46:02.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:02 smithi123 ceph-mon[34321]: pgmap v576: 129 pgs: 129 active+clean; 879 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.7 MiB/s wr, 109 op/s 2024-09-05T12:46:03.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:03 smithi174 ceph-mon[34313]: osdmap e121: 12 total, 12 up, 12 in 2024-09-05T12:46:03.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.79 MB/sec execute 177 sec latency 22283.493 ms 2024-09-05T12:46:03.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:03 smithi071 ceph-mon[29845]: osdmap e121: 12 total, 12 up, 12 in 2024-09-05T12:46:03.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:03 smithi123 ceph-mon[34321]: osdmap e121: 12 total, 12 up, 12 in 2024-09-05T12:46:04.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:04 smithi174 ceph-mon[34313]: pgmap v578: 129 pgs: 129 active+clean; 872 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-05T12:46:04.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.71 MB/sec execute 178 sec latency 23283.577 ms 2024-09-05T12:46:04.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:04 smithi071 ceph-mon[29845]: pgmap v578: 129 pgs: 129 active+clean; 872 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-05T12:46:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:04 smithi123 ceph-mon[34321]: pgmap v578: 129 pgs: 129 active+clean; 872 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-05T12:46:05.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.63 MB/sec execute 179 sec latency 24283.656 ms 2024-09-05T12:46:06.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:06 smithi174 ceph-mon[34313]: pgmap v579: 129 pgs: 129 active+clean; 872 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-05T12:46:06.840 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.56 MB/sec execute 180 sec latency 25283.759 ms 2024-09-05T12:46:06.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:06 smithi071 ceph-mon[29845]: pgmap v579: 129 pgs: 129 active+clean; 872 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-05T12:46:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:06 smithi123 ceph-mon[34321]: pgmap v579: 129 pgs: 129 active+clean; 872 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 3.2 MiB/s wr, 122 op/s 2024-09-05T12:46:07.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.48 MB/sec execute 181 sec latency 26283.883 ms 2024-09-05T12:46:08.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:08 smithi174 ceph-mon[34313]: pgmap v580: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 423 KiB/s wr, 69 op/s 2024-09-05T12:46:08.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.41 MB/sec execute 182 sec latency 27283.985 ms 2024-09-05T12:46:08.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:08 smithi071 ceph-mon[29845]: pgmap v580: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 423 KiB/s wr, 69 op/s 2024-09-05T12:46:08.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:08 smithi123 ceph-mon[34321]: pgmap v580: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 423 KiB/s wr, 69 op/s 2024-09-05T12:46:09.792 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:46:09.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.33 MB/sec execute 183 sec latency 28284.079 ms 2024-09-05T12:46:10.170 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:46:10.576 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:10 smithi071 ceph-mon[29845]: pgmap v581: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 65 op/s 2024-09-05T12:46:10.577 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:10 smithi071 ceph-mon[29845]: osdmap e122: 12 total, 12 up, 12 in 2024-09-05T12:46:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:10 smithi174 ceph-mon[34313]: pgmap v581: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 65 op/s 2024-09-05T12:46:10.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:10 smithi174 ceph-mon[34313]: osdmap e122: 12 total, 12 up, 12 in 2024-09-05T12:46:10.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.26 MB/sec execute 184 sec latency 29284.141 ms 2024-09-05T12:46:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:10 smithi123 ceph-mon[34321]: pgmap v581: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 195 KiB/s wr, 65 op/s 2024-09-05T12:46:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:10 smithi123 ceph-mon[34321]: osdmap e122: 12 total, 12 up, 12 in 2024-09-05T12:46:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.206+0000 7fc7f0c00640 1 -- 172.21.15.71:0/2904307580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec101de0 msgr2=0x7fc7ec1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:11.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/2904307580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec101de0 0x7fc7ec1021c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009900 tx=0x7fc7dc02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 -- 172.21.15.71:0/2904307580 shutdown_connections 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/2904307580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/2904307580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7ec102790 0x7fc7ec1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/2904307580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec101de0 0x7fc7ec1021c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 -- 172.21.15.71:0/2904307580 >> 172.21.15.71:0/2904307580 conn(0x7fc7ec0fd8c0 msgr2=0x7fc7ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 -- 172.21.15.71:0/2904307580 shutdown_connections 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.207+0000 7fc7f0c00640 1 -- 172.21.15.71:0/2904307580 wait complete. 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 Processor -- start 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 -- start start 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7ec101de0 0x7fc7ec1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:11.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec102790 0x7fc7ec1a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec111f70 con 0x7fc7ec102790 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec111df0 con 0x7fc7ec106f10 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.208+0000 7fc7f0c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec1120f0 con 0x7fc7ec101de0 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7e1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec102790 0x7fc7ec1a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:50216/0 (socket says 172.21.15.71:50216) 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 -- 172.21.15.71:0/1395203135 learned_addr learned my addr 172.21.15.71:0/1395203135 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7e9e00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7ec101de0 0x7fc7ec1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 -- 172.21.15.71:0/1395203135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7ec101de0 msgr2=0x7fc7ec1a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7ec101de0 0x7fc7ec1a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 -- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec102790 msgr2=0x7fc7ec1a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec102790 0x7fc7ec1a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 -- 172.21.15.71:0/1395203135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec10e450 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7e1400640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec102790 0x7fc7ec1a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.209+0000 7fc7ea800640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10db90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400eaa0 tx=0x7fc7e400ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.210+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400cdb0 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.210+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec10e740 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.210+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7e4005020 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.210+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40116b0 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.210+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec1ab6f0 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.211+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec1032c0 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.213+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc7e4005560 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.214+0000 7fc7e3400640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d00a2cf0 0x7fc7d00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.214+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fc7e40cf8d0 con 0x7fc7ec106f10 2024-09-05T12:46:11.234 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.214+0000 7fc7e9e00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d00a2cf0 0x7fc7d00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.215+0000 7fc7e9e00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d00a2cf0 0x7fc7d00a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009230 tx=0x7fc7dc0058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:11.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.216+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc7e4016030 con 0x7fc7ec106f10 2024-09-05T12:46:11.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.428+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7ec105ed0 con 0x7fc7ec106f10 2024-09-05T12:46:11.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.429+0000 7fc7e3400640 1 -- 172.21.15.71:0/1395203135 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7fc7e409bab0 con 0x7fc7ec106f10 2024-09-05T12:46:11.430 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:46:11.430 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:46:11.431 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:46:11.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d00a2cf0 msgr2=0x7fc7d00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:11.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d00a2cf0 0x7fc7d00a51b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc009230 tx=0x7fc7dc0058d0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 msgr2=0x7fc7ec10db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:11.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10db90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400eaa0 tx=0x7fc7e400ef60 comp rx=0 tx=0).stop 2024-09-05T12:46:11.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 shutdown_connections 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7ec106f10 0x7fc7ec10db90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc7d00a2cf0 0x7fc7d00a51b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7ec102790 0x7fc7ec1a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 --2- 172.21.15.71:0/1395203135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc7ec101de0 0x7fc7ec1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 >> 172.21.15.71:0/1395203135 conn(0x7fc7ec0fd8c0 msgr2=0x7fc7ec0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 shutdown_connections 2024-09-05T12:46:11.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:11.431+0000 7fc7f0c00640 1 -- 172.21.15.71:0/1395203135 wait complete. 2024-09-05T12:46:11.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:11 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1395203135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:46:11.799 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:11 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1395203135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:46:11.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.19 MB/sec execute 185 sec latency 30284.281 ms 2024-09-05T12:46:11.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:11 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1395203135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:46:12.615 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 25f6eb5a --release --await' 2024-09-05T12:46:12.615 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id 25f6eb5a --release --await 2024-09-05T12:46:12.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:12 smithi174 ceph-mon[34313]: pgmap v583: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T12:46:12.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86440 13.12 MB/sec execute 186 sec latency 31284.360 ms 2024-09-05T12:46:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 -- 172.21.15.71:0/1890463128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac102ae0 msgr2=0x7f37ac102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 --2- 172.21.15.71:0/1890463128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac102ae0 0x7f37ac102ec0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f379c009900 tx=0x7f379c02a480 comp rx=0 tx=0).stop 2024-09-05T12:46:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 -- 172.21.15.71:0/1890463128 shutdown_connections 2024-09-05T12:46:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 --2- 172.21.15.71:0/1890463128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac10bf00 0x7f37ac10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 --2- 172.21.15.71:0/1890463128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac103400 0x7f37ac10b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 --2- 172.21.15.71:0/1890463128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac102ae0 0x7f37ac102ec0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 -- 172.21.15.71:0/1890463128 >> 172.21.15.71:0/1890463128 conn(0x7f37ac0777c0 msgr2=0x7f37ac0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 -- 172.21.15.71:0/1890463128 shutdown_connections 2024-09-05T12:46:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.896+0000 7f37b0c00640 1 -- 172.21.15.71:0/1890463128 wait complete. 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 Processor -- start 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 -- start start 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac102ae0 0x7f37ac198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 0x7f37ac1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac10bf00 0x7f37ac19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac111d20 con 0x7f37ac103400 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac111ba0 con 0x7f37ac10bf00 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.897+0000 7f37b0c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac111ea0 con 0x7f37ac102ae0 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 0x7f37ac1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 0x7f37ac1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45600/0 (socket says 172.21.15.71:45600) 2024-09-05T12:46:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 -- 172.21.15.71:0/2999258147 learned_addr learned my addr 172.21.15.71:0/2999258147 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37ab400640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac102ae0 0x7f37ac198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37abe00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac10bf00 0x7f37ac19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 -- 172.21.15.71:0/2999258147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac102ae0 msgr2=0x7f37ac198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac102ae0 0x7f37ac198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 -- 172.21.15.71:0/2999258147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac10bf00 msgr2=0x7f37ac19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac10bf00 0x7f37ac19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.898+0000 7f37aaa00640 1 -- 172.21.15.71:0/2999258147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac19d260 con 0x7f37ac103400 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.899+0000 7f37aaa00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 0x7f37ac1987b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f379800d770 tx=0x7f379800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.899+0000 7f37a3e00640 1 -- 172.21.15.71:0/2999258147 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798017070 con 0x7f37ac103400 2024-09-05T12:46:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.899+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac19d550 con 0x7f37ac103400 2024-09-05T12:46:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.899+0000 7f37a3e00640 1 -- 172.21.15.71:0/2999258147 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3798005500 con 0x7f37ac103400 2024-09-05T12:46:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.899+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac1a4de0 con 0x7f37ac103400 2024-09-05T12:46:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.899+0000 7f37a3e00640 1 -- 172.21.15.71:0/2999258147 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798020060 con 0x7f37ac103400 2024-09-05T12:46:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 Processor -- start 2024-09-05T12:46:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 -- start start 2024-09-05T12:46:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac416d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558ac40c8100 con 0x558ac41623b0 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ac40701b0 con 0x558ac416d570 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.903+0000 7f37b521a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ac40c40c0 con 0x558ac4162cd0 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45614/0 (socket says 172.21.15.71:45614) 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37abe00640 1 -- 172.21.15.71:0/1289296271 learned_addr learned my addr 172.21.15.71:0/1289296271 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37a3e00640 1 -- 172.21.15.71:0/2999258147 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3798004da0 con 0x7f37ac103400 2024-09-05T12:46:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac416d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 -- 172.21.15.71:0/1289296271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 msgr2=0x558ac416f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 -- 172.21.15.71:0/1289296271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 msgr2=0x558ac4162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4162790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 -- 172.21.15.71:0/1289296271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac3ddb210 con 0x558ac4162cd0 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.904+0000 7f37ab400640 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac416d030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f379c02aa70 tx=0x7f379c0040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86abfd10f07053 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.905+0000 7f3797e00640 1 -- 172.21.15.71:0/1289296271 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c035040 con 0x558ac4162cd0 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.905+0000 7f3797e00640 1 -- 172.21.15.71:0/1289296271 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c005520 con 0x558ac4162cd0 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.905+0000 7f3797e00640 1 -- 172.21.15.71:0/1289296271 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c004e60 con 0x558ac4162cd0 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.905+0000 7f37a3e00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37900a2cf0 0x7f37900a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37a3e00640 1 -- 172.21.15.71:0/2999258147 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f37980cf260 con 0x7f37ac103400 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 -- 172.21.15.71:0/1289296271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 msgr2=0x558ac416d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac416d030 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f379c02aa70 tx=0x7f379c0040f0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37abe00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37900a2cf0 0x7f37900a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 -- 172.21.15.71:0/1289296271 shutdown_connections 2024-09-05T12:46:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac416d030 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 --2- 172.21.15.71:0/1289296271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 -- 172.21.15.71:0/1289296271 >> 172.21.15.71:0/1289296271 conn(0x558ac415e920 msgr2=0x558ac415ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 -- 172.21.15.71:0/1289296271 shutdown_connections 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 -- 172.21.15.71:0/1289296271 wait complete. 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37abe00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37900a2cf0 0x7f37900a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37a4005f40 tx=0x7f37a40096a0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.906+0000 7f37b521a740 1 -- start start 2024-09-05T12:46:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 Processor -- start 2024-09-05T12:46:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac41623b0 0x558ac416a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac4162cd0 0x558ac416ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x558ac416fff0 con 0x558ac4162cd0 2024-09-05T12:46:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ac4051090 con 0x558ac416d570 2024-09-05T12:46:12.912 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ac4170170 con 0x558ac41623b0 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37b521a740 10 client.0 _ll_register_callbacks cb 0x558ac40a5f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.907+0000 7f37aaa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac4162cd0 0x558ac416ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac41623b0 0x558ac416a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37abe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42896/0 (socket says 172.21.15.71:42896) 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37abe00640 1 -- 172.21.15.71:0/1546570361 learned_addr learned my addr 172.21.15.71:0/1546570361 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 -- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 msgr2=0x558ac416b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 -- 172.21.15.71:0/1546570361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac4162cd0 msgr2=0x558ac416ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac4162cd0 0x558ac416ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 -- 172.21.15.71:0/1546570361 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac3ea7620 con 0x558ac41623b0 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37aaa00640 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac4162cd0 0x558ac416ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f37ab400640 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac41623b0 0x558ac416a5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f379c02aa70 tx=0x7f379c0045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19bab9e0fd183a15 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.908+0000 7f3794c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f3794c00640 1 -- 172.21.15.71:0/1546570361 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c052060 con 0x558ac41623b0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379c005520 con 0x558ac41623b0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f3794c00640 1 -- 172.21.15.71:0/1546570361 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f379c004950 con 0x558ac41623b0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f3794c00640 1 -- 172.21.15.71:0/1546570361 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f379c035050 con 0x558ac41623b0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f37b521a740 10 client.27475 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x558ac416fff0 con 0x558ac41623b0 2024-09-05T12:46:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.909+0000 7f3794c00640 1 -- 172.21.15.71:0/1546570361 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c005520 con 0x558ac41623b0 2024-09-05T12:46:12.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:12 smithi071 ceph-mon[29845]: pgmap v583: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f3794c00640 1 -- 172.21.15.71:0/1546570361 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f379c031ab0 con 0x558ac41623b0 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37b521a740 10 client.27475 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37b521a740 10 client.27475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37b521a740 20 client.27475 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37b521a740 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac417d010 0x558ac417f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37b521a740 4 client.27475 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ac40d5830 con 0x558ac417d010 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.910+0000 7f37aaa00640 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac417d010 0x558ac417f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f37aaa00640 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac417d010 0x558ac417f400 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e6e9b33456bf7b server_cookie=476f7b6a32623a83 in_seq=0 out_seq=0 2024-09-05T12:46:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f3794c00640 10 client.27475 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f3794c00640 1 -- 172.21.15.71:0/1546570361 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ac40d5830 con 0x558ac417d010 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f3794c00640 10 client.27475 handle_command_reply: tid=0 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f37b521a740 1 client.27475 shutdown 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac41623b0 msgr2=0x558ac416a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.911+0000 7f37b521a740 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac41623b0 0x558ac416a5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f379c02aa70 tx=0x7f379c0045d0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 shutdown_connections 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac417d010 0x558ac417f400 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac416d570 0x558ac416b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac4162cd0 0x558ac416ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 --2- 172.21.15.71:0/1546570361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac41623b0 0x558ac416a5f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f3794c00640 0 client.27475 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 >> 172.21.15.71:0/1546570361 conn(0x558ac415e920 msgr2=0x558ac4164680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 shutdown_connections 2024-09-05T12:46:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 1 -- 172.21.15.71:0/1546570361 wait complete. 2024-09-05T12:46:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.912+0000 7f37b521a740 20 client.27475 trim_cache size 0 max 16384 2024-09-05T12:46:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.928+0000 7f37b521a740 1 Processor -- start 2024-09-05T12:46:12.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.928+0000 7f37b521a740 1 -- start start 2024-09-05T12:46:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37b521a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 0x558ac416acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37b521a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac4184a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37b521a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac416d570 0x558ac4184f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37b521a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1acc00 con 0x558ac416d570 2024-09-05T12:46:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37b521a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1ae740 con 0x558ac41623b0 2024-09-05T12:46:12.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37b521a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1ad2d0 con 0x558ac4162cd0 2024-09-05T12:46:12.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:12 smithi123 ceph-mon[34321]: pgmap v583: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 34 KiB/s wr, 13 op/s 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 0x558ac416acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37abe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac416d570 0x558ac4184f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 0x558ac416acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42904/0 (socket says 172.21.15.71:42904) 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 -- 172.21.15.71:0/1929741955 learned_addr learned my addr 172.21.15.71:0/1929741955 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37aaa00640 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac4184a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 -- 172.21.15.71:0/1929741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 msgr2=0x558ac4184a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac4184a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 -- 172.21.15.71:0/1929741955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac416d570 msgr2=0x558ac4184f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac416d570 0x558ac4184f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.929+0000 7f37ab400640 1 -- 172.21.15.71:0/1929741955 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ac4179380 con 0x558ac41623b0 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.930+0000 7f37ab400640 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 0x558ac416acc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f379c009900 tx=0x7f379c02a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6716788d4c30b1be server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.930+0000 7f3797400640 1 -- 172.21.15.71:0/1929741955 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c052040 con 0x558ac41623b0 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.930+0000 7f3797400640 1 -- 172.21.15.71:0/1929741955 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c031a90 con 0x558ac41623b0 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.930+0000 7f3797400640 1 -- 172.21.15.71:0/1929741955 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c03c6a0 con 0x558ac41623b0 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 -- 172.21.15.71:0/1929741955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 msgr2=0x558ac416acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 0x558ac416acc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f379c009900 tx=0x7f379c02a480 comp rx=0 tx=0).stop 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 -- 172.21.15.71:0/1929741955 shutdown_connections 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac416d570 0x558ac4184f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac4162cd0 0x558ac4184a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 --2- 172.21.15.71:0/1929741955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac41623b0 0x558ac416acc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 -- 172.21.15.71:0/1929741955 >> 172.21.15.71:0/1929741955 conn(0x558ac415e920 msgr2=0x558ac416a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 -- 172.21.15.71:0/1929741955 shutdown_connections 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 -- 172.21.15.71:0/1929741955 wait complete. 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 -- start start 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.931+0000 7f37b521a740 1 Processor -- start 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4175a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac4162cd0 0x558ac4170da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 0x558ac41712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1ae070 con 0x558ac41623b0 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1ad9a0 con 0x558ac4162cd0 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1aac00 con 0x558ac416d570 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37b521a740 10 client.0 _ll_register_callbacks cb 0x7f37a4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37ab400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4175a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37abe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 0x558ac41712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37abe00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 0x558ac41712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:44996/0 (socket says 172.21.15.71:44996) 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.932+0000 7f37abe00640 1 -- 172.21.15.71:0/273624589 learned_addr learned my addr 172.21.15.71:0/273624589 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37aaa00640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac4162cd0 0x558ac4170da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37abe00640 1 -- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac4162cd0 msgr2=0x558ac4170da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37abe00640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac4162cd0 0x558ac4170da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37abe00640 1 -- 172.21.15.71:0/273624589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 msgr2=0x558ac4175a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37abe00640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4175a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37abe00640 1 -- 172.21.15.71:0/273624589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ac41718d0 con 0x558ac416d570 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37aaa00640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac4162cd0 0x558ac4170da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37ab400640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4175a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f37abe00640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 0x558ac41712e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37a4021d70 tx=0x7f37a4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60ade3dcee111e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f3794c00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.933+0000 7f3794c00640 1 -- 172.21.15.71:0/273624589 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4004b20 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f379c031a90 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f3794c00640 1 -- 172.21.15.71:0/273624589 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f37a4017030 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f3794c00640 1 -- 172.21.15.71:0/273624589 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f37a403a050 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 10 client.27481 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x558ac41e7860 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f3794c00640 1 -- 172.21.15.71:0/273624589 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f379c031a90 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f3794c00640 1 -- 172.21.15.71:0/273624589 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f37a401f7a0 con 0x558ac416d570 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 10 client.27481 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 10 client.27481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 20 client.27481 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac41e7fe0 0x558ac41ea400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 4 client.27481 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:46:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.934+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558ac40cfb80 con 0x558ac41e7fe0 2024-09-05T12:46:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.935+0000 7f37ab400640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac41e7fe0 0x558ac41ea400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.935+0000 7f3794c00640 10 client.27481 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.935+0000 7f37ab400640 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac41e7fe0 0x558ac41ea400 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4827d2ba3c596860 server_cookie=6d91d97d4e75cf6a in_seq=0 out_seq=0 2024-09-05T12:46:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f3794c00640 1 -- 172.21.15.71:0/273624589 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558ac40cfb80 con 0x558ac41e7fe0 2024-09-05T12:46:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f3794c00640 10 client.27481 handle_command_reply: tid=0 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 client.27481 shutdown 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 msgr2=0x558ac41712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 0x558ac41712e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37a4021d70 tx=0x7f37a4004a40 comp rx=0 tx=0).stop 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 shutdown_connections 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x558ac41e7fe0 0x558ac41ea400 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ac416d570 0x558ac41712e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ac4162cd0 0x558ac4170da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f3794c00640 0 client.27481 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 --2- 172.21.15.71:0/273624589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558ac41623b0 0x558ac4175a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 >> 172.21.15.71:0/273624589 conn(0x558ac415e920 msgr2=0x558ac4178b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 shutdown_connections 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.960+0000 7f37b521a740 1 -- 172.21.15.71:0/273624589 wait complete. 2024-09-05T12:46:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b521a740 20 client.27481 trim_cache size 0 max 16384 2024-09-05T12:46:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:46:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:46:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:46:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37900a2cf0 msgr2=0x7f37900a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37900a2cf0 0x7f37900a51b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f37a4005f40 tx=0x7f37a40096a0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 msgr2=0x7f37ac1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 0x7f37ac1987b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f379800d770 tx=0x7f379800dc30 comp rx=0 tx=0).stop 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 shutdown_connections 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac10bf00 0x7f37ac19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f37900a2cf0 0x7f37900a51b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37ac103400 0x7f37ac1987b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 --2- 172.21.15.71:0/2999258147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac102ae0 0x7f37ac198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.961+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 >> 172.21.15.71:0/2999258147 conn(0x7f37ac0777c0 msgr2=0x7f37ac0fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.962+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 shutdown_connections 2024-09-05T12:46:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:12.962+0000 7f37b0c00640 1 -- 172.21.15.71:0/2999258147 wait complete. 2024-09-05T12:46:13.014 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:46:13.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.672+0000 7f4a82400640 1 -- 172.21.15.71:0/2033287451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 msgr2=0x7f4a7c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.672+0000 7f4a82400640 1 --2- 172.21.15.71:0/2033287451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c1021d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009900 tx=0x7f4a6c02a410 comp rx=0 tx=0).stop 2024-09-05T12:46:13.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 -- 172.21.15.71:0/2033287451 shutdown_connections 2024-09-05T12:46:13.674 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 --2- 172.21.15.71:0/2033287451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a7c106f20 0x7f4a7c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 --2- 172.21.15.71:0/2033287451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 0x7f4a7c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 --2- 172.21.15.71:0/2033287451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c1021d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 -- 172.21.15.71:0/2033287451 >> 172.21.15.71:0/2033287451 conn(0x7f4a7c0fd8b0 msgr2=0x7f4a7c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:13.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 -- 172.21.15.71:0/2033287451 shutdown_connections 2024-09-05T12:46:13.675 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.673+0000 7f4a82400640 1 -- 172.21.15.71:0/2033287451 wait complete. 2024-09-05T12:46:13.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 Processor -- start 2024-09-05T12:46:13.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 -- start start 2024-09-05T12:46:13.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 0x7f4a7c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.677 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a7c106f20 0x7f4a7c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c111f80 con 0x7f4a7c101df0 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c111e00 con 0x7f4a7c1027a0 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.674+0000 7f4a82400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c112100 con 0x7f4a7c106f20 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 0x7f4a7c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45674/0 (socket says 172.21.15.71:45674) 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 0x7f4a7c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42932/0 (socket says 172.21.15.71:42932) 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 -- 172.21.15.71:0/2708313758 learned_addr learned my addr 172.21.15.71:0/2708313758 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7be00640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a7c106f20 0x7f4a7c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 -- 172.21.15.71:0/2708313758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a7c106f20 msgr2=0x7f4a7c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a7c106f20 0x7f4a7c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 -- 172.21.15.71:0/2708313758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 msgr2=0x7f4a7c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 0x7f4a7c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 -- 172.21.15.71:0/2708313758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c1a68e0 con 0x7f4a7c101df0 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.675+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c0744f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c006010 tx=0x7f4a6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.676+0000 7f4a78c00640 1 -- 172.21.15.71:0/2708313758 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c03f030 con 0x7f4a7c101df0 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.676+0000 7f4a78c00640 1 -- 172.21.15.71:0/2708313758 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c004440 con 0x7f4a7c101df0 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.676+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c1a6b70 con 0x7f4a7c101df0 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.676+0000 7f4a78c00640 1 -- 172.21.15.71:0/2708313758 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c035070 con 0x7f4a7c101df0 2024-09-05T12:46:13.679 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.676+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c1a70a0 con 0x7f4a7c101df0 2024-09-05T12:46:13.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.679+0000 7f4a78c00640 1 -- 172.21.15.71:0/2708313758 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4a6c042030 con 0x7f4a7c101df0 2024-09-05T12:46:13.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.679+0000 7f4a84df5740 1 Processor -- start 2024-09-05T12:46:13.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.679+0000 7f4a84df5740 1 -- start start 2024-09-05T12:46:13.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.679+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956ab6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.679+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.679+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956b64f0 0x5556956b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a84df5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555695611650 con 0x5556956abc30 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a84df5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5556955b8f10 con 0x5556956b64f0 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a84df5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55569560d610 con 0x5556956ab310 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45686/0 (socket says 172.21.15.71:45686) 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 -- 172.21.15.71:0/2933520239 learned_addr learned my addr 172.21.15.71:0/2933520239 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a72a00640 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956b64f0 0x5556956b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.682 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7be00640 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956ab6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 -- 172.21.15.71:0/2933520239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 msgr2=0x5556956ab6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956ab6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 -- 172.21.15.71:0/2933520239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956b64f0 msgr2=0x5556956b88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956b64f0 0x5556956b88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.683 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.680+0000 7f4a7b400640 1 -- 172.21.15.71:0/2933520239 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555695320580 con 0x5556956abc30 2024-09-05T12:46:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b5fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0ece50 tx=0x7f4a6c0f3de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf39c3edd632b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a78c00640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4a600a2d70 0x7f4a600a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a70c00640 1 -- 172.21.15.71:0/2933520239 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0ff020 con 0x5556956abc30 2024-09-05T12:46:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a78c00640 1 -- 172.21.15.71:0/2708313758 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f4a6c0efbb0 con 0x7f4a7c101df0 2024-09-05T12:46:13.687 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a70c00640 1 -- 172.21.15.71:0/2933520239 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c0bc3e0 con 0x5556956abc30 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a70c00640 1 -- 172.21.15.71:0/2933520239 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0bbab0 con 0x5556956abc30 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.681+0000 7f4a7be00640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4a600a2d70 0x7f4a600a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 -- 172.21.15.71:0/2933520239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 msgr2=0x5556956b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b5fb0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0ece50 tx=0x7f4a6c0f3de0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a7be00640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4a600a2d70 0x7f4a600a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009c70 tx=0x7f4a74005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 -- 172.21.15.71:0/2933520239 shutdown_connections 2024-09-05T12:46:13.688 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956b64f0 0x5556956b88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b5fb0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2933520239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956ab6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 -- 172.21.15.71:0/2933520239 >> 172.21.15.71:0/2933520239 conn(0x5556956a78a0 msgr2=0x5556956a7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 -- 172.21.15.71:0/2933520239 shutdown_connections 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.682+0000 7f4a84df5740 1 -- 172.21.15.71:0/2933520239 wait complete. 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 -- start start 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 Processor -- start 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956abc30 0x5556956b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x5556956b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.689 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5556956b8e80 con 0x5556956b64f0 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5556955cc350 con 0x5556956abc30 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556956b9000 con 0x5556956ab310 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.683+0000 7f4a84df5740 10 client.0 _ll_register_callbacks cb 0x5556955ef290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x5556956b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a7be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x5556956b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:45696/0 (socket says 172.21.15.71:45696) 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a7be00640 1 -- 172.21.15.71:0/1716253168 learned_addr learned my addr 172.21.15.71:0/1716253168 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956abc30 0x5556956b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 -- 172.21.15.71:0/1716253168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 msgr2=0x5556956b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 -- 172.21.15.71:0/1716253168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 msgr2=0x5556956b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x5556956b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 -- 172.21.15.71:0/1716253168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556953ea960 con 0x5556956abc30 2024-09-05T12:46:13.690 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a72a00640 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956abc30 0x5556956b3890 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c075f80 tx=0x7f4a64011e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45f777c45bd33905 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.684+0000 7f4a69600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a69600640 1 -- 172.21.15.71:0/1716253168 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6401f060 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c0bc3e0 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a69600640 1 -- 172.21.15.71:0/1716253168 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f4a6400fc80 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a69600640 1 -- 172.21.15.71:0/1716253168 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4a64026050 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a84df5740 10 client.27449 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5556956c49f0 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.685+0000 7f4a69600640 1 -- 172.21.15.71:0/1716253168 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c0bc3e0 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a69600640 1 -- 172.21.15.71:0/1716253168 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f4a64019030 con 0x5556956abc30 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a84df5740 10 client.27449 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:13.691 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a84df5740 10 client.27449 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a84df5740 20 client.27449 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a84df5740 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556956c5e50 0x5556956c8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a84df5740 4 client.27449 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55569561ebc0 con 0x5556956c5e50 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.686+0000 7f4a7b400640 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556956c5e50 0x5556956c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.687+0000 7f4a7b400640 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556956c5e50 0x5556956c8270 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64a9576740aea845 server_cookie=6c0dfb17e3899405 in_seq=0 out_seq=0 2024-09-05T12:46:13.692 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.687+0000 7f4a69600640 10 client.27449 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.687+0000 7f4a69600640 1 -- 172.21.15.71:0/1716253168 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55569561ebc0 con 0x5556956c5e50 2024-09-05T12:46:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.687+0000 7f4a69600640 10 client.27449 handle_command_reply: tid=0 2024-09-05T12:46:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.687+0000 7f4a84df5740 1 client.27449 shutdown 2024-09-05T12:46:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956abc30 msgr2=0x5556956b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956abc30 0x5556956b3890 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c075f80 tx=0x7f4a64011e60 comp rx=0 tx=0).stop 2024-09-05T12:46:13.693 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 shutdown_connections 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5556956c5e50 0x5556956c8270 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x5556956b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a69600640 0 client.27449 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956abc30 0x5556956b3890 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 --2- 172.21.15.71:0/1716253168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956ab310 0x5556956b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 >> 172.21.15.71:0/1716253168 conn(0x5556956a78a0 msgr2=0x5556956a6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 shutdown_connections 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 1 -- 172.21.15.71:0/1716253168 wait complete. 2024-09-05T12:46:13.694 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.688+0000 7f4a84df5740 20 client.27449 trim_cache size 0 max 16384 2024-09-05T12:46:13.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.705+0000 7f4a84df5740 1 Processor -- start 2024-09-05T12:46:13.706 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.705+0000 7f4a84df5740 1 -- start start 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956b64f0 0x5556956cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a84df5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c1b1220 con 0x5556956abc30 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a84df5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c1b2d60 con 0x5556956ab310 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a84df5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c1b18f0 con 0x5556956b64f0 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a72a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.708 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42964/0 (socket says 172.21.15.71:42964) 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a7b400640 1 -- 172.21.15.71:0/3046921252 learned_addr learned my addr 172.21.15.71:0/3046921252 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.706+0000 7f4a7be00640 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956b64f0 0x5556956cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a7b400640 1 -- 172.21.15.71:0/3046921252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956b64f0 msgr2=0x5556956cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a7b400640 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956b64f0 0x5556956cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a7b400640 1 -- 172.21.15.71:0/3046921252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 msgr2=0x5556956b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a7b400640 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a7b400640 1 -- 172.21.15.71:0/3046921252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556956a76d0 con 0x5556956ab310 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a72a00640 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a7b400640 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956b3c20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0b86b0 tx=0x7f4a6c0f31d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=853f90848b55e806 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:13.709 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a6be00640 1 -- 172.21.15.71:0/3046921252 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c049070 con 0x5556956ab310 2024-09-05T12:46:13.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a6be00640 1 -- 172.21.15.71:0/3046921252 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c0f3430 con 0x5556956ab310 2024-09-05T12:46:13.710 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.707+0000 7f4a6be00640 1 -- 172.21.15.71:0/3046921252 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0ff040 con 0x5556956ab310 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 -- 172.21.15.71:0/3046921252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 msgr2=0x5556956b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956b3c20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0b86b0 tx=0x7f4a6c0f31d0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 -- 172.21.15.71:0/3046921252 shutdown_connections 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956b64f0 0x5556956cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956abc30 0x5556956b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 --2- 172.21.15.71:0/3046921252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956b3c20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 -- 172.21.15.71:0/3046921252 >> 172.21.15.71:0/3046921252 conn(0x5556956a78a0 msgr2=0x5556956b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 -- 172.21.15.71:0/3046921252 shutdown_connections 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 -- 172.21.15.71:0/3046921252 wait complete. 2024-09-05T12:46:13.713 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 -- start start 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.708+0000 7f4a84df5740 1 Processor -- start 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956abc30 0x5556956babe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x55569572fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c1b2690 con 0x5556956b64f0 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c1b1fc0 con 0x5556956ab310 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c1af220 con 0x5556956abc30 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a84df5740 10 client.0 _ll_register_callbacks cb 0x7f4a64009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42980/0 (socket says 172.21.15.71:42980) 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 -- 172.21.15.71:0/2534091427 learned_addr learned my addr 172.21.15.71:0/2534091427 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7be00640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x55569572fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a72a00640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956abc30 0x5556956babe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 -- 172.21.15.71:0/2534091427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956abc30 msgr2=0x5556956babe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956abc30 0x5556956babe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 -- 172.21.15.71:0/2534091427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 msgr2=0x55569572fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x55569572fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.709+0000 7f4a7b400640 1 -- 172.21.15.71:0/2534091427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555695730120 con 0x5556956ab310 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a7b400640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956ba6a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0b7fa0 tx=0x7f4a6c0f3660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0c6f318c7f1aee server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a69600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a69600640 1 -- 172.21.15.71:0/2534091427 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c0ff030 con 0x5556956ab310 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a69600640 1 -- 172.21.15.71:0/2534091427 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f4a6c0f3860 con 0x5556956ab310 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6c0f3430 con 0x5556956ab310 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a69600640 1 -- 172.21.15.71:0/2534091427 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4a6c049070 con 0x5556956ab310 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a84df5740 10 client.27455 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:13.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x555695731420 con 0x5556956ab310 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.710+0000 7f4a69600640 1 -- 172.21.15.71:0/2534091427 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a6c0f3430 con 0x5556956ab310 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a69600640 1 -- 172.21.15.71:0/2534091427 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f4a6c0b8350 con 0x5556956ab310 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a84df5740 10 client.27455 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a84df5740 10 client.27455 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a84df5740 20 client.27455 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555695731700 0x555695733af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a84df5740 4 client.27455 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "25f6eb5a", "release": true, "await": true}] 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "25f6eb5a", "release": true, "await": true}) -- 0x5556956190d0 con 0x555695731700 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a72a00640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555695731700 0x555695733af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a72a00640 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555695731700 0x555695733af0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d33b61a875c97c server_cookie=3e26b1b0441d7192 in_seq=0 out_seq=0 2024-09-05T12:46:13.716 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.711+0000 7f4a69600640 10 client.27455 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:13.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.737+0000 7f4a69600640 1 -- 172.21.15.71:0/2534091427 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5556956190d0 con 0x555695731700 2024-09-05T12:46:13.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.737+0000 7f4a69600640 10 client.27455 handle_command_reply: tid=0 2024-09-05T12:46:13.738 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.737+0000 7f4a84df5740 1 client.27455 shutdown 2024-09-05T12:46:13.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.737+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 msgr2=0x5556956ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.737+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956ba6a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c0b7fa0 tx=0x7f4a6c0f3660 comp rx=0 tx=0).stop 2024-09-05T12:46:13.739 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 shutdown_connections 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x555695731700 0x555695733af0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556956b64f0 0x55569572fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556956abc30 0x5556956babe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 --2- 172.21.15.71:0/2534091427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556956ab310 0x5556956ba6a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a69600640 0 client.27455 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 >> 172.21.15.71:0/2534091427 conn(0x5556956a78a0 msgr2=0x5556956b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 shutdown_connections 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.738+0000 7f4a84df5740 1 -- 172.21.15.71:0/2534091427 wait complete. 2024-09-05T12:46:13.740 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.739+0000 7f4a84df5740 20 client.27455 trim_cache size 0 max 16384 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 76, 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 60, 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "25f6eb5a": { 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 60, 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:46:13.741 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 152.8, 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:46:13.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:46:13.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:46:13.743 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:46:13.743 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:46:13.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.740+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4a600a2d70 msgr2=0x7f4a600a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.743 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.740+0000 7f4a82400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4a600a2d70 0x7f4a600a5230 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009c70 tx=0x7f4a74005f90 comp rx=0 tx=0).stop 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.740+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 msgr2=0x7f4a7c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.740+0000 7f4a82400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c0744f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c006010 tx=0x7f4a6c004040 comp rx=0 tx=0).stop 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.741+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 shutdown_connections 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.741+0000 7f4a82400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a7c106f20 0x7f4a7c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.741+0000 7f4a82400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a7c1027a0 0x7f4a7c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.741+0000 7f4a82400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f4a600a2d70 0x7f4a600a5230 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.741+0000 7f4a82400640 1 --2- 172.21.15.71:0/2708313758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a7c101df0 0x7f4a7c0744f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.741+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 >> 172.21.15.71:0/2708313758 conn(0x7f4a7c0fd8b0 msgr2=0x7f4a7c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:13.744 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.742+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 shutdown_connections 2024-09-05T12:46:13.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:46:13.743+0000 7f4a82400640 1 -- 172.21.15.71:0/2708313758 wait complete. 2024-09-05T12:46:13.841 INFO:tasks.workunit.client.0.smithi071.stdout: 10 86513 13.08 MB/sec execute 187 sec latency 32179.940 ms 2024-09-05T12:46:14.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:14 smithi071 ceph-mon[29845]: pgmap v584: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-05T12:46:14.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:14 smithi174 ceph-mon[34313]: pgmap v584: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-05T12:46:14.842 INFO:tasks.workunit.client.0.smithi071.stdout: 10 87245 13.17 MB/sec execute 188 sec latency 71.943 ms 2024-09-05T12:46:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:14 smithi123 ceph-mon[34321]: pgmap v584: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-05T12:46:14.949 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '25f6eb5a', total seconds elapsed: 37.9 2024-09-05T12:46:15.843 INFO:tasks.workunit.client.0.smithi071.stdout: 10 88073 13.38 MB/sec execute 189 sec latency 81.079 ms 2024-09-05T12:46:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:16 smithi174 ceph-mon[34313]: pgmap v585: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-05T12:46:16.832 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:16 smithi071 ceph-mon[29845]: pgmap v585: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-05T12:46:16.843 INFO:tasks.workunit.client.0.smithi071.stdout: 10 88942 13.54 MB/sec execute 190 sec latency 59.924 ms 2024-09-05T12:46:16.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:16 smithi123 ceph-mon[34321]: pgmap v585: 129 pgs: 129 active+clean; 869 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 45 KiB/s wr, 14 op/s 2024-09-05T12:46:17.843 INFO:tasks.workunit.client.0.smithi071.stdout: 10 89611 13.68 MB/sec execute 191 sec latency 88.218 ms 2024-09-05T12:46:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:18 smithi174 ceph-mon[34313]: pgmap v586: 129 pgs: 129 active+clean; 881 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-09-05T12:46:18.845 INFO:tasks.workunit.client.0.smithi071.stdout: 10 90390 13.81 MB/sec execute 192 sec latency 86.550 ms 2024-09-05T12:46:18.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:18 smithi071 ceph-mon[29845]: pgmap v586: 129 pgs: 129 active+clean; 881 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-09-05T12:46:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:18 smithi123 ceph-mon[34321]: pgmap v586: 129 pgs: 129 active+clean; 881 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 125 op/s 2024-09-05T12:46:19.843 INFO:tasks.workunit.client.0.smithi071.stdout: 10 91336 14.01 MB/sec execute 193 sec latency 49.774 ms 2024-09-05T12:46:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:20 smithi174 ceph-mon[34313]: pgmap v587: 129 pgs: 129 active+clean; 889 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-09-05T12:46:20.843 INFO:tasks.workunit.client.0.smithi071.stdout: 10 91934 14.09 MB/sec execute 194 sec latency 115.338 ms 2024-09-05T12:46:20.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:20 smithi071 ceph-mon[29845]: pgmap v587: 129 pgs: 129 active+clean; 889 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-09-05T12:46:20.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:20 smithi123 ceph-mon[34321]: pgmap v587: 129 pgs: 129 active+clean; 889 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 178 op/s 2024-09-05T12:46:21.844 INFO:tasks.workunit.client.0.smithi071.stdout: 10 92807 14.29 MB/sec execute 195 sec latency 92.100 ms 2024-09-05T12:46:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:22 smithi174 ceph-mon[34313]: pgmap v588: 129 pgs: 129 active+clean; 908 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-05T12:46:22.844 INFO:tasks.workunit.client.0.smithi071.stdout: 10 93429 14.32 MB/sec execute 196 sec latency 72.724 ms 2024-09-05T12:46:22.872 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:22 smithi071 ceph-mon[29845]: pgmap v588: 129 pgs: 129 active+clean; 908 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-05T12:46:22.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:22 smithi123 ceph-mon[34321]: pgmap v588: 129 pgs: 129 active+clean; 908 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.0 MiB/s wr, 323 op/s 2024-09-05T12:46:23.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:23 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:46:23.844 INFO:tasks.workunit.client.0.smithi071.stdout: 10 94152 14.50 MB/sec execute 197 sec latency 88.022 ms 2024-09-05T12:46:23.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:23 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:46:23.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:23 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:46:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:24 smithi174 ceph-mon[34313]: pgmap v589: 129 pgs: 129 active+clean; 926 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 400 op/s 2024-09-05T12:46:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:24 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:46:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:24 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:46:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:24 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:46:24.844 INFO:tasks.workunit.client.0.smithi071.stdout: 10 94908 14.62 MB/sec execute 198 sec latency 103.496 ms 2024-09-05T12:46:24.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:24 smithi071 ceph-mon[29845]: pgmap v589: 129 pgs: 129 active+clean; 926 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 400 op/s 2024-09-05T12:46:24.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:24 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:46:24.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:24 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:46:24.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:24 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:46:24.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:24 smithi123 ceph-mon[34321]: pgmap v589: 129 pgs: 129 active+clean; 926 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 400 op/s 2024-09-05T12:46:24.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:24 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:46:24.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:24 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:46:24.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:24 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:46:25.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 95646 14.76 MB/sec execute 199 sec latency 80.012 ms 2024-09-05T12:46:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 -- 172.21.15.71:0/270506076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 msgr2=0x7fe004101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 --2- 172.21.15.71:0/270506076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe004101f40 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009890 tx=0x7fdff402a3b0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 -- 172.21.15.71:0/270506076 shutdown_connections 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 --2- 172.21.15.71:0/270506076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 --2- 172.21.15.71:0/270506076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe004101f40 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 --2- 172.21.15.71:0/270506076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe004107b80 0x7fe004107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 -- 172.21.15.71:0/270506076 >> 172.21.15.71:0/270506076 conn(0x7fe0040fd7e0 msgr2=0x7fe0040ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.143+0000 7fe00c800640 1 -- 172.21.15.71:0/270506076 shutdown_connections 2024-09-05T12:46:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.144+0000 7fe00c800640 1 -- 172.21.15.71:0/270506076 wait complete. 2024-09-05T12:46:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.145+0000 7fe00c800640 1 Processor -- start 2024-09-05T12:46:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.146+0000 7fe00c800640 1 -- start start 2024-09-05T12:46:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.146+0000 7fe00c800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe00419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.147+0000 7fe009e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe00419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.147+0000 7fe009e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe00419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50512/0 (socket says 172.21.15.71:50512) 2024-09-05T12:46:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.147+0000 7fe00c800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00419f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.147+0000 7fe00c800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe004107b80 0x7fe00410ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00419f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00419f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45910/0 (socket says 172.21.15.71:45910) 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 -- 172.21.15.71:0/1956886421 learned_addr learned my addr 172.21.15.71:0/1956886421 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe004111f30 con 0x7fe004107b80 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe004111db0 con 0x7fe004102480 2024-09-05T12:46:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041120b0 con 0x7fe004101ae0 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 -- 172.21.15.71:0/1956886421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 msgr2=0x7fe00419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe00419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 -- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe004107b80 msgr2=0x7fe00410ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe004107b80 0x7fe00410ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.148+0000 7fe009400640 1 -- 172.21.15.71:0/1956886421 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00410c350 con 0x7fe004102480 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.149+0000 7fe009400640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00419f5c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdff4035040 tx=0x7fdff4033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:26.150 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.150+0000 7fdffea00640 1 -- 172.21.15.71:0/1956886421 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff4037040 con 0x7fe004102480 2024-09-05T12:46:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.150+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00410c5e0 con 0x7fe004102480 2024-09-05T12:46:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.150+0000 7fdffea00640 1 -- 172.21.15.71:0/1956886421 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff4040070 con 0x7fe004102480 2024-09-05T12:46:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.150+0000 7fdffea00640 1 -- 172.21.15.71:0/1956886421 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40054c0 con 0x7fe004102480 2024-09-05T12:46:26.151 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.150+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00419aca0 con 0x7fe004102480 2024-09-05T12:46:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.155+0000 7fdffea00640 1 -- 172.21.15.71:0/1956886421 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fdff40304f0 con 0x7fe004102480 2024-09-05T12:46:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.157+0000 7fe00f5e0740 1 Processor -- start 2024-09-05T12:46:26.159 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.158+0000 7fe00f5e0740 1 -- start start 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.158+0000 7fdffea00640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdfec0a2da0 0x7fdfec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fdffea00640 1 -- 172.21.15.71:0/1956886421 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fdff40ef480 con 0x7fe004102480 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a6135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a61de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a620790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559e0a579100 con 0x559e0a61e3a0 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e0a5211b0 con 0x559e0a6131e0 2024-09-05T12:46:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559e0a5750c0 con 0x559e0a613b00 2024-09-05T12:46:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe009e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a6135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe009e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a6135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45914/0 (socket says 172.21.15.71:45914) 2024-09-05T12:46:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.159+0000 7fe009e00640 1 -- 172.21.15.71:0/78436028 learned_addr learned my addr 172.21.15.71:0/78436028 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe00a800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdfec0a2da0 0x7fdfec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe009e00640 1 -- 172.21.15.71:0/78436028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 msgr2=0x559e0a61de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe009e00640 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a61de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe009e00640 1 -- 172.21.15.71:0/78436028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 msgr2=0x559e0a620790 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe009e00640 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a620790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe009e00640 1 -- 172.21.15.71:0/78436028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0a28c210 con 0x559e0a6131e0 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.160+0000 7fe009e00640 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a6135c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009a70 tx=0x7fdff8009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530addf790bb2bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.161+0000 7fdff3000640 1 -- 172.21.15.71:0/78436028 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8017070 con 0x559e0a6131e0 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.161+0000 7fdff3000640 1 -- 172.21.15.71:0/78436028 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff8013470 con 0x559e0a6131e0 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.161+0000 7fdff3000640 1 -- 172.21.15.71:0/78436028 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8012630 con 0x559e0a6131e0 2024-09-05T12:46:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.162+0000 7fe00a800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdfec0a2da0 0x7fdfec0a5260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x559e0a610ad0 tx=0x7fe000009820 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 -- 172.21.15.71:0/78436028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 msgr2=0x559e0a6135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a6135c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009a70 tx=0x7fdff8009f30 comp rx=0 tx=0).stop 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 -- 172.21.15.71:0/78436028 shutdown_connections 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a620790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a61de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/78436028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a6135c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.163+0000 7fe00f5e0740 1 -- 172.21.15.71:0/78436028 >> 172.21.15.71:0/78436028 conn(0x559e0a60f750 msgr2=0x559e0a60fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.164+0000 7fe00f5e0740 1 -- 172.21.15.71:0/78436028 shutdown_connections 2024-09-05T12:46:26.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.164+0000 7fe00f5e0740 1 -- 172.21.15.71:0/78436028 wait complete. 2024-09-05T12:46:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.165+0000 7fe00f5e0740 1 -- start start 2024-09-05T12:46:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.165+0000 7fe00f5e0740 1 Processor -- start 2024-09-05T12:46:26.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a61b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a61bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe009e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe009e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45924/0 (socket says 172.21.15.71:45924) 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559e0a620fd0 con 0x559e0a613b00 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e0a502090 con 0x559e0a6131e0 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559e0a621150 con 0x559e0a61e3a0 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe009e00640 1 -- 172.21.15.71:0/1449565882 learned_addr learned my addr 172.21.15.71:0/1449565882 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe009400640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a61b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00a800640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a61bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.166+0000 7fe00f5e0740 10 client.0 _ll_register_callbacks cb 0x559e0a556f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:26.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.167+0000 7fe009e00640 1 -- 172.21.15.71:0/1449565882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 msgr2=0x559e0a61bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.167+0000 7fe009e00640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a61bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.167+0000 7fe009e00640 1 -- 172.21.15.71:0/1449565882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 msgr2=0x559e0a61b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.167+0000 7fe009e00640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a61b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.167+0000 7fe009e00640 1 -- 172.21.15.71:0/1449565882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0a358620 con 0x559e0a6131e0 2024-09-05T12:46:26.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.167+0000 7fe009400640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a61b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:46:26.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.168+0000 7fe009e00640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61b360 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdff8004810 tx=0x7fdff8013cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18bb4e7cff7c65f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.169+0000 7fdfe7400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:46:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.169+0000 7fdfe7400640 1 -- 172.21.15.71:0/1449565882 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8017050 con 0x559e0a6131e0 2024-09-05T12:46:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.169+0000 7fdfe7400640 1 -- 172.21.15.71:0/1449565882 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fdff801c650 con 0x559e0a6131e0 2024-09-05T12:46:26.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.169+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff8013470 con 0x559e0a6131e0 2024-09-05T12:46:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.170+0000 7fdfe7400640 1 -- 172.21.15.71:0/1449565882 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fdff802f050 con 0x559e0a6131e0 2024-09-05T12:46:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.170+0000 7fe00f5e0740 10 client.27467 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.170+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x559e0a62cc30 con 0x559e0a6131e0 2024-09-05T12:46:26.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.171+0000 7fdfe7400640 1 -- 172.21.15.71:0/1449565882 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff8013470 con 0x559e0a6131e0 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.171+0000 7fdfe7400640 1 -- 172.21.15.71:0/1449565882 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7fdff8026a40 con 0x559e0a6131e0 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.171+0000 7fe00f5e0740 10 client.27467 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.171+0000 7fe00f5e0740 10 client.27467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.172+0000 7fe00f5e0740 20 client.27467 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.172+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a62e030 0x559e0a630420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.172+0000 7fe00f5e0740 4 client.27467 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.172+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e0a586830 con 0x559e0a62e030 2024-09-05T12:46:26.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.172+0000 7fe009400640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a62e030 0x559e0a630420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.176+0000 7fdfe7400640 10 client.27467 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.176+0000 7fe009400640 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a62e030 0x559e0a630420 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac4b1ca63d767bd server_cookie=f30895f3253a7cab in_seq=0 out_seq=0 2024-09-05T12:46:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.179+0000 7fdfe7400640 1 -- 172.21.15.71:0/1449565882 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e0a586830 con 0x559e0a62e030 2024-09-05T12:46:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.179+0000 7fdfe7400640 10 client.27467 handle_command_reply: tid=0 2024-09-05T12:46:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.179+0000 7fe00f5e0740 1 client.27467 shutdown 2024-09-05T12:46:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.181+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 msgr2=0x559e0a61b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.181+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61b360 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdff8004810 tx=0x7fdff8013cf0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.181+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 shutdown_connections 2024-09-05T12:46:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a62e030 0x559e0a630420 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a61bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a61b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1449565882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61b360 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 >> 172.21.15.71:0/1449565882 conn(0x559e0a60f750 msgr2=0x559e0a615460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fdfe7400640 0 client.27467 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.182+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 shutdown_connections 2024-09-05T12:46:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.183+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1449565882 wait complete. 2024-09-05T12:46:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.183+0000 7fe00f5e0740 20 client.27467 trim_cache size 0 max 16384 2024-09-05T12:46:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.217+0000 7fe00f5e0740 1 Processor -- start 2024-09-05T12:46:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.217+0000 7fe00f5e0740 1 -- start start 2024-09-05T12:46:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a6359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a635f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041bdee0 con 0x559e0a613b00 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041bfa20 con 0x559e0a6131e0 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041be5b0 con 0x559e0a61e3a0 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe009400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a6359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe009400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a6359f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:56746/0 (socket says 172.21.15.71:56746) 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.218+0000 7fe009400640 1 -- 172.21.15.71:0/1193855309 learned_addr learned my addr 172.21.15.71:0/1193855309 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.219+0000 7fe009400640 1 -- 172.21.15.71:0/1193855309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 msgr2=0x559e0a635f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.219+0000 7fe009400640 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a635f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.219+0000 7fe009400640 1 -- 172.21.15.71:0/1193855309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 msgr2=0x559e0a61d1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.219+0000 7fe009400640 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.219+0000 7fe009400640 1 -- 172.21.15.71:0/1193855309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0a60ad60 con 0x559e0a613b00 2024-09-05T12:46:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.220+0000 7fe009400640 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a6359f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdff40099b0 tx=0x7fdff40ee040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b49d8dccd533b88 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.221+0000 7fdff2600640 1 -- 172.21.15.71:0/1193855309 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40313a0 con 0x559e0a613b00 2024-09-05T12:46:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.221+0000 7fdff2600640 1 -- 172.21.15.71:0/1193855309 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff40bbe20 con 0x559e0a613b00 2024-09-05T12:46:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.221+0000 7fdff2600640 1 -- 172.21.15.71:0/1193855309 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40b9850 con 0x559e0a613b00 2024-09-05T12:46:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.222+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1193855309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 msgr2=0x559e0a6359f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.222+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a6359f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdff40099b0 tx=0x7fdff40ee040 comp rx=0 tx=0).stop 2024-09-05T12:46:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1193855309 shutdown_connections 2024-09-05T12:46:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a61e3a0 0x559e0a635f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a613b00 0x559e0a6359f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/1193855309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a61d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.224 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1193855309 >> 172.21.15.71:0/1193855309 conn(0x559e0a60f750 msgr2=0x559e0a61cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1193855309 shutdown_connections 2024-09-05T12:46:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.223+0000 7fe00f5e0740 1 -- 172.21.15.71:0/1193855309 wait complete. 2024-09-05T12:46:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.224+0000 7fe00f5e0740 1 -- start start 2024-09-05T12:46:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.224+0000 7fe00f5e0740 1 Processor -- start 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a693c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a6941a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a6263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041bf350 con 0x559e0a61e3a0 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041bec80 con 0x559e0a6131e0 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0041bbee0 con 0x559e0a613b00 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe00f5e0740 10 client.0 _ll_register_callbacks cb 0x7fdff80098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe009400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a6941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe009e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a693c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe009400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a6941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50546/0 (socket says 172.21.15.71:50546) 2024-09-05T12:46:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.225+0000 7fe009400640 1 -- 172.21.15.71:0/3935739339 learned_addr learned my addr 172.21.15.71:0/3935739339 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.226+0000 7fe009400640 1 -- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 msgr2=0x559e0a693c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe00a800640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a6263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe009400640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a693c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe009400640 1 -- 172.21.15.71:0/3935739339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 msgr2=0x559e0a6263a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe009400640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a6263a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe009400640 1 -- 172.21.15.71:0/3935739339 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0a626bd0 con 0x559e0a613b00 2024-09-05T12:46:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe00a800640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a6263a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:46:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe009e00640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a693c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fe009400640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a6941a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdff4004360 tx=0x7fdff40bb720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bdd8e46ce1606c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.228+0000 7fdfe7400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.229+0000 7fdfe7400640 1 -- 172.21.15.71:0/3935739339 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40c4390 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.229+0000 7fdfe7400640 1 -- 172.21.15.71:0/3935739339 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fdff40ecac0 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.229+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdff40bbe20 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.230+0000 7fdfe7400640 1 -- 172.21.15.71:0/3935739339 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fdff40b9830 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.230+0000 7fe00f5e0740 10 client.27511 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.230+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x559e0a691d30 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fdfe7400640 1 -- 172.21.15.71:0/3935739339 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdff40bbe20 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fdfe7400640 1 -- 172.21.15.71:0/3935739339 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7fdff403a070 con 0x559e0a613b00 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fe00f5e0740 10 client.27511 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fe00f5e0740 10 client.27511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fe00f5e0740 20 client.27511 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a6a9d20 0x559e0a6ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fe00f5e0740 4 client.27511 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:46:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.231+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e0a580b80 con 0x559e0a6a9d20 2024-09-05T12:46:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.232+0000 7fe009e00640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a6a9d20 0x559e0a6ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.232+0000 7fe009e00640 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a6a9d20 0x559e0a6ac140 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99d5d800e58422c server_cookie=2350c49509f1b998 in_seq=0 out_seq=0 2024-09-05T12:46:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.232+0000 7fdfe7400640 10 client.27511 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.695+0000 7fdfe7400640 1 -- 172.21.15.71:0/3935739339 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e0a580b80 con 0x559e0a6a9d20 2024-09-05T12:46:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.695+0000 7fdfe7400640 10 client.27511 handle_command_reply: tid=0 2024-09-05T12:46:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.695+0000 7fe00f5e0740 1 client.27511 shutdown 2024-09-05T12:46:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.695+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 msgr2=0x559e0a6941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.695+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a6941a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdff4004360 tx=0x7fdff40bb720 comp rx=0 tx=0).stop 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 shutdown_connections 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x559e0a6a9d20 0x559e0a6ac140 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0a61e3a0 0x559e0a6263a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e0a613b00 0x559e0a6941a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 --2- 172.21.15.71:0/3935739339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e0a6131e0 0x559e0a693c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fdfe7400640 0 client.27511 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:26.698 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 >> 172.21.15.71:0/3935739339 conn(0x559e0a60f750 msgr2=0x559e0a6197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 shutdown_connections 2024-09-05T12:46:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.697+0000 7fe00f5e0740 1 -- 172.21.15.71:0/3935739339 wait complete. 2024-09-05T12:46:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.698+0000 7fe00f5e0740 20 client.27511 trim_cache size 0 max 16384 2024-09-05T12:46:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:46:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:46:26.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdfec0a2da0 msgr2=0x7fdfec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdfec0a2da0 0x7fdfec0a5260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x559e0a610ad0 tx=0x7fe000009820 comp rx=0 tx=0).stop 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 msgr2=0x7fe00419f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00419f5c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdff4035040 tx=0x7fdff4033d90 comp rx=0 tx=0).stop 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 shutdown_connections 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fdfec0a2da0 0x7fdfec0a5260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe004107b80 0x7fe00410ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe004102480 0x7fe00419f5c0 secure :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdff4035040 tx=0x7fdff4033d90 comp rx=0 tx=0).stop 2024-09-05T12:46:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 --2- 172.21.15.71:0/1956886421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe004101ae0 0x7fe00419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 >> 172.21.15.71:0/1956886421 conn(0x7fe0040fd7e0 msgr2=0x7fe00410d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.699+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 shutdown_connections 2024-09-05T12:46:26.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:26.700+0000 7fe00c800640 1 -- 172.21.15.71:0/1956886421 wait complete. 2024-09-05T12:46:26.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 96083 14.79 MB/sec execute 200 sec latency 318.671 ms 2024-09-05T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:26 smithi071 ceph-mon[29845]: pgmap v590: 129 pgs: 129 active+clean; 926 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 397 op/s 2024-09-05T12:46:26.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:26 smithi123 ceph-mon[34321]: pgmap v590: 129 pgs: 129 active+clean; 926 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 397 op/s 2024-09-05T12:46:27.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:26 smithi174 ceph-mon[34313]: pgmap v590: 129 pgs: 129 active+clean; 926 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.2 MiB/s wr, 397 op/s 2024-09-05T12:46:27.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 96806 14.92 MB/sec execute 201 sec latency 72.607 ms 2024-09-05T12:46:28.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 97366 14.97 MB/sec execute 202 sec latency 305.448 ms 2024-09-05T12:46:28.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:28 smithi071 ceph-mon[29845]: pgmap v591: 129 pgs: 129 active+clean; 954 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.2 MiB/s wr, 609 op/s 2024-09-05T12:46:28.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:28 smithi123 ceph-mon[34321]: pgmap v591: 129 pgs: 129 active+clean; 954 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.2 MiB/s wr, 609 op/s 2024-09-05T12:46:29.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:28 smithi174 ceph-mon[34313]: pgmap v591: 129 pgs: 129 active+clean; 954 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.2 MiB/s wr, 609 op/s 2024-09-05T12:46:29.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 97929 15.05 MB/sec execute 203 sec latency 406.517 ms 2024-09-05T12:46:30.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 98588 15.16 MB/sec execute 204 sec latency 420.576 ms 2024-09-05T12:46:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:30 smithi071 ceph-mon[29845]: pgmap v592: 129 pgs: 129 active+clean; 962 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 567 op/s 2024-09-05T12:46:30.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:30 smithi123 ceph-mon[34321]: pgmap v592: 129 pgs: 129 active+clean; 962 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 567 op/s 2024-09-05T12:46:31.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:30 smithi174 ceph-mon[34313]: pgmap v592: 129 pgs: 129 active+clean; 962 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 567 op/s 2024-09-05T12:46:31.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 99453 15.31 MB/sec execute 205 sec latency 214.759 ms 2024-09-05T12:46:32.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100218 15.47 MB/sec execute 206 sec latency 379.221 ms 2024-09-05T12:46:32.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:32 smithi071 ceph-mon[29845]: pgmap v593: 129 pgs: 129 active+clean; 980 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 731 op/s 2024-09-05T12:46:32.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:32 smithi123 ceph-mon[34321]: pgmap v593: 129 pgs: 129 active+clean; 980 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 731 op/s 2024-09-05T12:46:33.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:32 smithi174 ceph-mon[34313]: pgmap v593: 129 pgs: 129 active+clean; 980 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.2 MiB/s wr, 731 op/s 2024-09-05T12:46:33.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100596 15.47 MB/sec execute 207 sec latency 325.758 ms 2024-09-05T12:46:34.832 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:34 smithi071 ceph-mon[29845]: pgmap v594: 129 pgs: 129 active+clean; 989 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 710 op/s 2024-09-05T12:46:34.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100891 15.46 MB/sec execute 208 sec latency 405.718 ms 2024-09-05T12:46:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:34 smithi123 ceph-mon[34321]: pgmap v594: 129 pgs: 129 active+clean; 989 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 710 op/s 2024-09-05T12:46:35.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:34 smithi174 ceph-mon[34313]: pgmap v594: 129 pgs: 129 active+clean; 989 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.7 MiB/s wr, 710 op/s 2024-09-05T12:46:35.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100909 15.41 MB/sec execute 209 sec latency 1347.869 ms 2024-09-05T12:46:36.814 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:36 smithi071 ceph-mon[29845]: pgmap v595: 129 pgs: 129 active+clean; 989 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 592 op/s 2024-09-05T12:46:36.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100909 15.33 MB/sec execute 210 sec latency 2253.752 ms 2024-09-05T12:46:36.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:36 smithi123 ceph-mon[34321]: pgmap v595: 129 pgs: 129 active+clean; 989 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 592 op/s 2024-09-05T12:46:37.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:36 smithi174 ceph-mon[34313]: pgmap v595: 129 pgs: 129 active+clean; 989 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.0 MiB/s wr, 592 op/s 2024-09-05T12:46:37.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100909 15.26 MB/sec execute 211 sec latency 3222.809 ms 2024-09-05T12:46:38.795 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:38 smithi071 ceph-mon[29845]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-09-05T12:46:38.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 100953 15.20 MB/sec execute 212 sec latency 4222.850 ms 2024-09-05T12:46:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:38 smithi123 ceph-mon[34321]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-09-05T12:46:39.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:38 smithi174 ceph-mon[34313]: pgmap v596: 129 pgs: 129 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 800 op/s 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/4271397463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0241018b0 msgr2=0x7fc024101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- 172.21.15.71:0/4271397463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0241018b0 0x7fc024101d10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc010009930 tx=0x7fc01002a420 comp rx=0 tx=0).stop 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/4271397463 shutdown_connections 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- 172.21.15.71:0/4271397463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc024102250 0x7fc02410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- 172.21.15.71:0/4271397463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc0241018b0 0x7fc024101d10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- 172.21.15.71:0/4271397463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241078c0 0x7fc024107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/4271397463 >> 172.21.15.71:0/4271397463 conn(0x7fc0240fd5d0 msgr2=0x7fc0240ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/4271397463 shutdown_connections 2024-09-05T12:46:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/4271397463 wait complete. 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 Processor -- start 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- start start 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 0x7fc02419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc024102250 0x7fc02419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0241078c0 0x7fc02419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111d00 con 0x7fc024102250 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111b80 con 0x7fc0241018b0 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc024111e80 con 0x7fc0241078c0 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc023e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc024102250 0x7fc02419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 0x7fc02419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 0x7fc02419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42298/0 (socket says 172.21.15.71:42298) 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 -- 172.21.15.71:0/2125408365 learned_addr learned my addr 172.21.15.71:0/2125408365 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029c00640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0241078c0 0x7fc02419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 -- 172.21.15.71:0/2125408365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0241078c0 msgr2=0x7fc02419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0241078c0 0x7fc02419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 -- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc024102250 msgr2=0x7fc02419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc024102250 0x7fc02419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 -- 172.21.15.71:0/2125408365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02419f460 con 0x7fc0241018b0 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc029200640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 0x7fc02419a470 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc01800e9f0 tx=0x7fc01800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc021600640 1 -- 172.21.15.71:0/2125408365 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01800cd80 con 0x7fc0241018b0 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02419f750 con 0x7fc0241018b0 2024-09-05T12:46:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc021600640 1 -- 172.21.15.71:0/2125408365 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc018010910 con 0x7fc0241018b0 2024-09-05T12:46:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc021600640 1 -- 172.21.15.71:0/2125408365 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180188f0 con 0x7fc0241018b0 2024-09-05T12:46:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.824+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0241a6fe0 con 0x7fc0241018b0 2024-09-05T12:46:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 Processor -- start 2024-09-05T12:46:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- start start 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad914b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad914bcd0 0x556ad9156030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad9156570 0x556ad9158960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556ad90b1100 con 0x556ad914bcd0 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556ad90591b0 con 0x556ad9156570 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ad90ad0c0 con 0x556ad914b3b0 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc021600640 1 -- 172.21.15.71:0/2125408365 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc018010ac0 con 0x7fc0241018b0 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad914b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad914b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35400/0 (socket says 172.21.15.71:35400) 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 -- 172.21.15.71:0/3954874328 learned_addr learned my addr 172.21.15.71:0/3954874328 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc023e00640 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad9156570 0x556ad9158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029200640 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad914bcd0 0x556ad9156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 -- 172.21.15.71:0/3954874328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad9156570 msgr2=0x556ad9158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad9156570 0x556ad9158960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 -- 172.21.15.71:0/3954874328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad914bcd0 msgr2=0x556ad9156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad914bcd0 0x556ad9156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 -- 172.21.15.71:0/3954874328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad8dc4210 con 0x556ad914b3b0 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc023e00640 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad9156570 0x556ad9158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad914b790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009fc0 tx=0x7fc01c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21a9645d69533fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc016000640 1 -- 172.21.15.71:0/3954874328 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c003b70 con 0x556ad914b3b0 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc016000640 1 -- 172.21.15.71:0/3954874328 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01c003d10 con 0x556ad914b3b0 2024-09-05T12:46:39.837 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc016000640 1 -- 172.21.15.71:0/3954874328 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01c011a40 con 0x556ad914b3b0 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- 172.21.15.71:0/3954874328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 msgr2=0x556ad914b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad914b790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc01c009fc0 tx=0x7fc01c010860 comp rx=0 tx=0).stop 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- 172.21.15.71:0/3954874328 shutdown_connections 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad9156570 0x556ad9158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad914bcd0 0x556ad9156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/3954874328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad914b790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- 172.21.15.71:0/3954874328 >> 172.21.15.71:0/3954874328 conn(0x556ad9147920 msgr2=0x556ad9147d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:39.842 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc021600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc00c0a2d70 0x7fc00c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- 172.21.15.71:0/3954874328 shutdown_connections 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc021600640 1 -- 172.21.15.71:0/2125408365 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fc0180cfb60 con 0x7fc0241018b0 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- 172.21.15.71:0/3954874328 wait complete. 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc029c00640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc00c0a2d70 0x7fc00c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 -- start start 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.830+0000 7fc02ea5c740 1 Processor -- start 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029c00640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc00c0a2d70 0x7fc00c0a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc01c011010 tx=0x7fc01c005fd0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9153630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9153b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x556ad9158ff0 con 0x556ad9156570 2024-09-05T12:46:39.843 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556ad903a090 con 0x556ad914bcd0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ad9159170 con 0x556ad914b3b0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 10 client.0 _ll_register_callbacks cb 0x556ad908ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9153630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35410/0 (socket says 172.21.15.71:35410) 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 -- 172.21.15.71:0/1631272316 learned_addr learned my addr 172.21.15.71:0/1631272316 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029c00640 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 -- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 msgr2=0x556ad9153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 -- 172.21.15.71:0/1631272316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 msgr2=0x556ad91540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 -- 172.21.15.71:0/1631272316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad8e90620 con 0x556ad914b3b0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc029200640 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9153630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc01800e9c0 tx=0x7fc01809c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc7c1a7e9da551b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:39.844 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 1 -- 172.21.15.71:0/1631272316 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0180cd350 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc01c003d10 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 1 -- 172.21.15.71:0/1631272316 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fc0180d7040 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 1 -- 172.21.15.71:0/1631272316 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc0180a4900 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 10 client.27523 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x556ad9164de0 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 1 -- 172.21.15.71:0/1631272316 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01c003d10 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 1 -- 172.21.15.71:0/1631272316 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7fc018018ad0 con 0x556ad914b3b0 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 10 client.27523 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 10 client.27523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 20 client.27523 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad9166250 0x556ad9168640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.845 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 4 client.27523 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad90be830 con 0x556ad9166250 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc023e00640 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad9166250 0x556ad9168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc023e00640 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad9166250 0x556ad9168640 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07cbac717a538de server_cookie=1574abbf679cfb84 in_seq=0 out_seq=0 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 10 client.27523 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 1 -- 172.21.15.71:0/1631272316 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ad90be830 con 0x556ad9166250 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 10 client.27523 handle_command_reply: tid=0 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 client.27523 shutdown 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 msgr2=0x556ad9153630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9153630 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc01800e9c0 tx=0x7fc01809c390 comp rx=0 tx=0).stop 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 shutdown_connections 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad9166250 0x556ad9168640 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.846 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc00aa00640 0 client.27523 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/1631272316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9153630 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 >> 172.21.15.71:0/1631272316 conn(0x556ad9147920 msgr2=0x556ad914d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 shutdown_connections 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 1 -- 172.21.15.71:0/1631272316 wait complete. 2024-09-05T12:46:39.847 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.836+0000 7fc02ea5c740 20 client.27523 trim_cache size 0 max 16384 2024-09-05T12:46:39.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101077 15.15 MB/sec execute 213 sec latency 4871.930 ms 2024-09-05T12:46:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 Processor -- start 2024-09-05T12:46:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- start start 2024-09-05T12:46:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9151f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9152440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad916de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b1080 con 0x556ad9156570 2024-09-05T12:46:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b2bc0 con 0x556ad914bcd0 2024-09-05T12:46:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b1750 con 0x556ad914b3b0 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc029c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad916de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc029c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad916de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:42904/0 (socket says 172.21.15.71:42904) 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc029c00640 1 -- 172.21.15.71:0/2975094047 learned_addr learned my addr 172.21.15.71:0/2975094047 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc029200640 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 -- 172.21.15.71:0/2975094047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 msgr2=0x556ad9151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 -- 172.21.15.71:0/2975094047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 msgr2=0x556ad916de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad916de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 -- 172.21.15.71:0/2975094047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad9151b80 con 0x556ad914bcd0 2024-09-05T12:46:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc023e00640 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9152440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc010005940 tx=0x7fc010005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6f21a684582eb26 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc015600640 1 -- 172.21.15.71:0/2975094047 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010035040 con 0x556ad914bcd0 2024-09-05T12:46:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc015600640 1 -- 172.21.15.71:0/2975094047 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc010031a70 con 0x556ad914bcd0 2024-09-05T12:46:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc015600640 1 -- 172.21.15.71:0/2975094047 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010004c60 con 0x556ad914bcd0 2024-09-05T12:46:39.863 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- 172.21.15.71:0/2975094047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 msgr2=0x556ad9152440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9152440 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc010005940 tx=0x7fc010005fd0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- 172.21.15.71:0/2975094047 shutdown_connections 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad916de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9152440 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/2975094047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- 172.21.15.71:0/2975094047 >> 172.21.15.71:0/2975094047 conn(0x556ad9147920 msgr2=0x556ad914c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- 172.21.15.71:0/2975094047 shutdown_connections 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- 172.21.15.71:0/2975094047 wait complete. 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 -- start start 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.854+0000 7fc02ea5c740 1 Processor -- start 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9159990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.864 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b24f0 con 0x556ad9156570 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241b1e20 con 0x556ad914bcd0 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0241af080 con 0x556ad914b3b0 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 10 client.0 _ll_register_callbacks cb 0x7fc018009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42346/0 (socket says 172.21.15.71:42346) 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 -- 172.21.15.71:0/4001360384 learned_addr learned my addr 172.21.15.71:0/4001360384 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc029c00640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc029200640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 -- 172.21.15.71:0/4001360384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 msgr2=0x556ad9159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.865 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 -- 172.21.15.71:0/4001360384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 msgr2=0x556ad91cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 -- 172.21.15.71:0/4001360384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad91cf3e0 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc023e00640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9159ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc010004770 tx=0x7fc010000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=344c81264434c193 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 1 -- 172.21.15.71:0/4001360384 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010035050 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 1 -- 172.21.15.71:0/4001360384 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7fc010005020 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc010031a70 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 1 -- 172.21.15.71:0/4001360384 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc010062050 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 10 client.27488 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x556ad91d06f0 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 1 -- 172.21.15.71:0/4001360384 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc010031a70 con 0x556ad914bcd0 2024-09-05T12:46:39.866 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 1 -- 172.21.15.71:0/4001360384 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7fc010047b30 con 0x556ad914bcd0 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 10 client.27488 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 10 client.27488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 20 client.27488 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad91d0e40 0x556ad91d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 4 client.27488 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ad90b8b80 con 0x556ad91d0e40 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc029200640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad91d0e40 0x556ad91d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc029200640 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad91d0e40 0x556ad91d3260 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0df67c9f2b32102 server_cookie=ed77d085f45a9e74 in_seq=0 out_seq=0 2024-09-05T12:46:39.867 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:39.859+0000 7fc00aa00640 10 client.27488 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:40.275 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc00aa00640 1 -- 172.21.15.71:0/4001360384 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ad90b8b80 con 0x556ad91d0e40 2024-09-05T12:46:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc00aa00640 10 client.27488 handle_command_reply: tid=0 2024-09-05T12:46:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 client.27488 shutdown 2024-09-05T12:46:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 msgr2=0x556ad9159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9159ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc010004770 tx=0x7fc010000bc0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.276 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 shutdown_connections 2024-09-05T12:46:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x556ad91d0e40 0x556ad91d3260 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556ad9156570 0x556ad91cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ad914bcd0 0x556ad9159ed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 --2- 172.21.15.71:0/4001360384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ad914b3b0 0x556ad9159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.277 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 >> 172.21.15.71:0/4001360384 conn(0x556ad9147920 msgr2=0x556ad9151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc00aa00640 0 client.27488 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 shutdown_connections 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 1 -- 172.21.15.71:0/4001360384 wait complete. 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.271+0000 7fc02ea5c740 20 client.27488 trim_cache size 0 max 16384 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:46:40.278 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc00c0a2d70 msgr2=0x7fc00c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc00c0a2d70 0x7fc00c0a5230 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc01c011010 tx=0x7fc01c005fd0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 msgr2=0x7fc02419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 0x7fc02419a470 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc01800e9f0 tx=0x7fc01800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 shutdown_connections 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0241078c0 0x7fc02419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc00c0a2d70 0x7fc00c0a5230 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.279 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc024102250 0x7fc02419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 --2- 172.21.15.71:0/2125408365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0241018b0 0x7fc02419a470 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 >> 172.21.15.71:0/2125408365 conn(0x7fc0240fd5d0 msgr2=0x7fc0240fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 shutdown_connections 2024-09-05T12:46:40.280 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:40.276+0000 7fc02a600640 1 -- 172.21.15.71:0/2125408365 wait complete. 2024-09-05T12:46:40.802 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:40 smithi071 ceph-mon[29845]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.1 MiB/s wr, 594 op/s 2024-09-05T12:46:40.846 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101082 15.08 MB/sec execute 214 sec latency 5848.309 ms 2024-09-05T12:46:40.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:40 smithi123 ceph-mon[34321]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.1 MiB/s wr, 594 op/s 2024-09-05T12:46:41.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:40 smithi174 ceph-mon[34313]: pgmap v597: 129 pgs: 129 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.1 MiB/s wr, 594 op/s 2024-09-05T12:46:41.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101083 15.01 MB/sec execute 215 sec latency 6848.366 ms 2024-09-05T12:46:42.723 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:42 smithi071 ceph-mon[29845]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-05T12:46:42.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.94 MB/sec execute 216 sec latency 7848.426 ms 2024-09-05T12:46:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:42 smithi123 ceph-mon[34321]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-05T12:46:43.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:42 smithi174 ceph-mon[34313]: pgmap v598: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 658 op/s 2024-09-05T12:46:43.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.87 MB/sec execute 217 sec latency 8848.491 ms 2024-09-05T12:46:44.729 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:44 smithi071 ceph-mon[29845]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 518 op/s 2024-09-05T12:46:44.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.81 MB/sec execute 218 sec latency 9848.561 ms 2024-09-05T12:46:44.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:44 smithi123 ceph-mon[34321]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 518 op/s 2024-09-05T12:46:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:44 smithi174 ceph-mon[34313]: pgmap v599: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 12 MiB/s wr, 518 op/s 2024-09-05T12:46:45.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.74 MB/sec execute 219 sec latency 10848.619 ms 2024-09-05T12:46:46.732 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:46 smithi071 ceph-mon[29845]: pgmap v600: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-09-05T12:46:46.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.67 MB/sec execute 220 sec latency 11848.770 ms 2024-09-05T12:46:46.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:46 smithi123 ceph-mon[34321]: pgmap v600: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-09-05T12:46:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:46 smithi174 ceph-mon[34313]: pgmap v600: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-09-05T12:46:47.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.60 MB/sec execute 221 sec latency 12848.819 ms 2024-09-05T12:46:48.735 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:48 smithi071 ceph-mon[29845]: pgmap v601: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 523 op/s 2024-09-05T12:46:48.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101084 14.54 MB/sec execute 222 sec latency 13848.865 ms 2024-09-05T12:46:48.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:48 smithi123 ceph-mon[34321]: pgmap v601: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 523 op/s 2024-09-05T12:46:49.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:48 smithi174 ceph-mon[34313]: pgmap v601: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 13 MiB/s wr, 523 op/s 2024-09-05T12:46:49.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101085 14.47 MB/sec execute 223 sec latency 14848.929 ms 2024-09-05T12:46:50.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101085 14.41 MB/sec execute 224 sec latency 15848.977 ms 2024-09-05T12:46:50.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:50 smithi071 ceph-mon[29845]: pgmap v602: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 320 op/s 2024-09-05T12:46:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:50 smithi123 ceph-mon[34321]: pgmap v602: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 320 op/s 2024-09-05T12:46:51.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:50 smithi174 ceph-mon[34313]: pgmap v602: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.6 MiB/s wr, 320 op/s 2024-09-05T12:46:51.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101085 14.34 MB/sec execute 225 sec latency 16849.048 ms 2024-09-05T12:46:52.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101085 14.28 MB/sec execute 226 sec latency 17849.125 ms 2024-09-05T12:46:52.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:52 smithi071 ceph-mon[29845]: pgmap v603: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 320 op/s 2024-09-05T12:46:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:52 smithi123 ceph-mon[34321]: pgmap v603: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 320 op/s 2024-09-05T12:46:53.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:52 smithi174 ceph-mon[34313]: pgmap v603: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 320 op/s 2024-09-05T12:46:53.440 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/294436514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79181018b0 msgr2=0x7f7918101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- 172.21.15.71:0/294436514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79181018b0 0x7f7918101d10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7900009930 tx=0x7f790002a420 comp rx=0 tx=0).stop 2024-09-05T12:46:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/294436514 shutdown_connections 2024-09-05T12:46:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- 172.21.15.71:0/294436514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7918102250 0x7f791810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- 172.21.15.71:0/294436514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79181018b0 0x7f7918101d10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- 172.21.15.71:0/294436514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 0x7f7918107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/294436514 >> 172.21.15.71:0/294436514 conn(0x7f79180fd5d0 msgr2=0x7f79180ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/294436514 shutdown_connections 2024-09-05T12:46:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/294436514 wait complete. 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 Processor -- start 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- start start 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79181018b0 0x7f79180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 0x7f7918074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 0x7f7918074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7918111d50 con 0x7f7918102250 2024-09-05T12:46:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7918111bd0 con 0x7f79181078c0 2024-09-05T12:46:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7918111ed0 con 0x7f79181018b0 2024-09-05T12:46:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 0x7f7918074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 0x7f7918074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59024/0 (socket says 172.21.15.71:59024) 2024-09-05T12:46:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 0x7f7918074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790e000640 1 -- 172.21.15.71:0/963098964 learned_addr learned my addr 172.21.15.71:0/963098964 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7916a00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79181018b0 0x7f79180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 -- 172.21.15.71:0/963098964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79181018b0 msgr2=0x7f79180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79181018b0 0x7f79180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 -- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 msgr2=0x7f7918074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 0x7f7918074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 -- 172.21.15.71:0/963098964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79181a6740 con 0x7f79181078c0 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790e000640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 0x7f7918074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917400640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 0x7f7918074f70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7910009fd0 tx=0x7f791000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790fe00640 1 -- 172.21.15.71:0/963098964 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f791000ce20 con 0x7f79181078c0 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79181a6a30 con 0x7f79181078c0 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790fe00640 1 -- 172.21.15.71:0/963098964 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7910005020 con 0x7f79181078c0 2024-09-05T12:46:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f790fe00640 1 -- 172.21.15.71:0/963098964 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7910010720 con 0x7f79181078c0 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.436+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79181a6fd0 con 0x7f79181078c0 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.441+0000 7f792047e740 1 Processor -- start 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790fe00640 1 -- 172.21.15.71:0/963098964 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7910005540 con 0x7f79181078c0 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- start start 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843051790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f84305c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f84305e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f842fb7100 con 0x55f8430513b0 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f842f5f1b0 con 0x55f84305c570 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f842fb30c0 con 0x55f843051cd0 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843051790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843051790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59034/0 (socket says 172.21.15.71:59034) 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 -- 172.21.15.71:0/2121508968 learned_addr learned my addr 172.21.15.71:0/2121508968 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7916a00640 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f84305c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f84305e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 -- 172.21.15.71:0/2121508968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 msgr2=0x55f84305c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f84305c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 -- 172.21.15.71:0/2121508968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 msgr2=0x55f84305e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f84305e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 -- 172.21.15.71:0/2121508968 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f842cca210 con 0x55f8430513b0 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843051790 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f790002a940 tx=0x7f7900005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c30be8e37a055a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790fe00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78fc0a2cf0 0x7f78fc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790fe00640 1 -- 172.21.15.71:0/963098964 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f79100cfe40 con 0x7f79181078c0 2024-09-05T12:46:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7916a00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78fc0a2cf0 0x7f78fc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7907400640 1 -- 172.21.15.71:0/2121508968 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900044020 con 0x55f8430513b0 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7907400640 1 -- 172.21.15.71:0/2121508968 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7900004c40 con 0x55f8430513b0 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7907400640 1 -- 172.21.15.71:0/2121508968 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900035040 con 0x55f8430513b0 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- 172.21.15.71:0/2121508968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 msgr2=0x55f843051790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7916a00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78fc0a2cf0 0x7f78fc0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7908009d20 tx=0x7f7908005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843051790 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f790002a940 tx=0x7f7900005b50 comp rx=0 tx=0).stop 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- 172.21.15.71:0/2121508968 shutdown_connections 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f84305e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f84305c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- 172.21.15.71:0/2121508968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843051790 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- 172.21.15.71:0/2121508968 >> 172.21.15.71:0/2121508968 conn(0x55f84304d920 msgr2=0x55f84304dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- 172.21.15.71:0/2121508968 shutdown_connections 2024-09-05T12:46:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- 172.21.15.71:0/2121508968 wait complete. 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- start start 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 Processor -- start 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8430513b0 0x55f843059530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f843051cd0 0x55f843059a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 0x55f843059fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f84305eff0 con 0x55f84305c570 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f842f40090 con 0x55f843051cd0 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f84305f170 con 0x55f8430513b0 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f792047e740 10 client.0 _ll_register_callbacks cb 0x55f842f94f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7917400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 0x55f843059fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7917400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 0x55f843059fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59038/0 (socket says 172.21.15.71:59038) 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7917400640 1 -- 172.21.15.71:0/75148096 learned_addr learned my addr 172.21.15.71:0/75148096 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f7916a00640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8430513b0 0x55f843059530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.444+0000 7f790e000640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f843051cd0 0x55f843059a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7917400640 1 -- 172.21.15.71:0/75148096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8430513b0 msgr2=0x55f843059530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7917400640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8430513b0 0x55f843059530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7917400640 1 -- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f843051cd0 msgr2=0x55f843059a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7917400640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f843051cd0 0x55f843059a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7917400640 1 -- 172.21.15.71:0/75148096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f842d96620 con 0x55f84305c570 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7916a00640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8430513b0 0x55f843059530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7917400640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 0x55f843059fb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f79100983d0 tx=0x7f791009c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b07d931349c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 1 -- 172.21.15.71:0/75148096 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79100b6060 con 0x55f84305c570 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 1 -- 172.21.15.71:0/75148096 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f79100a4750 con 0x55f84305c570 2024-09-05T12:46:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7900004c40 con 0x55f84305c570 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 1 -- 172.21.15.71:0/75148096 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f79100bc070 con 0x55f84305c570 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 10 client.17484 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55f84306ae20 con 0x55f84305c570 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 1 -- 172.21.15.71:0/75148096 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7900004c40 con 0x55f84305c570 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 1 -- 172.21.15.71:0/75148096 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f791009b4e0 con 0x55f84305c570 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 10 client.17484 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 20 client.17484 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f84306c2e0 0x55f84306e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 4 client.17484 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f842fc4830 con 0x55f84306c2e0 2024-09-05T12:46:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7916a00640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f84306c2e0 0x55f84306e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f7916a00640 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f84306c2e0 0x55f84306e6d0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7a724b81e58484 server_cookie=a1d49e9c14a26048 in_seq=0 out_seq=0 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 10 client.17484 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 1 -- 172.21.15.71:0/75148096 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f842fc4830 con 0x55f84306c2e0 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 10 client.17484 handle_command_reply: tid=0 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 client.17484 shutdown 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 msgr2=0x55f843059fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 0x55f843059fb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f79100983d0 tx=0x7f791009c0b0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 shutdown_connections 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f84306c2e0 0x55f84306e6d0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f84305c570 0x55f843059fb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f843051cd0 0x55f843059a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 --2- 172.21.15.71:0/75148096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8430513b0 0x55f843059530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 >> 172.21.15.71:0/75148096 conn(0x55f84304d920 msgr2=0x55f843053680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f78fbe00640 0 client.17484 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 shutdown_connections 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 1 -- 172.21.15.71:0/75148096 wait complete. 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.449+0000 7f792047e740 20 client.17484 trim_cache size 0 max 16384 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 Processor -- start 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- start start 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843057de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f843058320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f843073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b0ed0 con 0x55f8430513b0 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b2a10 con 0x55f84305c570 2024-09-05T12:46:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b15a0 con 0x55f843051cd0 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843057de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843057de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:59054/0 (socket says 172.21.15.71:59054) 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/3676382014 learned_addr learned my addr 172.21.15.71:0/3676382014 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7917400640 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f843073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f790e000640 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f843058320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/3676382014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 msgr2=0x55f843058320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f843058320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/3676382014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 msgr2=0x55f843073fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f843073fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/3676382014 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f84304d1b0 con 0x55f8430513b0 2024-09-05T12:46:53.542 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f790e000640 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f843058320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843057de0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7908012cb0 tx=0x7f790801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded8c0ac55436c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7906a00640 1 -- 172.21.15.71:0/3676382014 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908005020 con 0x55f8430513b0 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7906a00640 1 -- 172.21.15.71:0/3676382014 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79080047a0 con 0x55f8430513b0 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7906a00640 1 -- 172.21.15.71:0/3676382014 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790802c820 con 0x55f8430513b0 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- 172.21.15.71:0/3676382014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 msgr2=0x55f843057de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843057de0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f7908012cb0 tx=0x7f790801b000 comp rx=0 tx=0).stop 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- 172.21.15.71:0/3676382014 shutdown_connections 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f84305c570 0x55f843073fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f843051cd0 0x55f843058320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- 172.21.15.71:0/3676382014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8430513b0 0x55f843057de0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- 172.21.15.71:0/3676382014 >> 172.21.15.71:0/3676382014 conn(0x55f84304d920 msgr2=0x55f84304dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- 172.21.15.71:0/3676382014 shutdown_connections 2024-09-05T12:46:53.543 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- 172.21.15.71:0/3676382014 wait complete. 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- start start 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 Processor -- start 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 0x55f843064dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f843051cd0 0x55f84305f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84305c570 0x55f84305fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b2340 con 0x55f843051cd0 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79181b1c70 con 0x55f8430513b0 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79181aeed0 con 0x55f84305c570 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f792047e740 10 client.0 _ll_register_callbacks cb 0x7f7910098210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f790e000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f843051cd0 0x55f84305f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 0x55f843064dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.544 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 0x55f843064dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:49596/0 (socket says 172.21.15.71:49596) 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/4056245745 learned_addr learned my addr 172.21.15.71:0/4056245745 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7917400640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84305c570 0x55f84305fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/4056245745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84305c570 msgr2=0x55f84305fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84305c570 0x55f84305fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/4056245745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f843051cd0 msgr2=0x55f84305f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f843051cd0 0x55f84305f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 -- 172.21.15.71:0/4056245745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8430d4f80 con 0x55f8430513b0 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f790e000640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f843051cd0 0x55f84305f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f7916a00640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 0x55f843064dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7908019040 tx=0x7f7908009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c620add3fe8966c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.468+0000 7f78fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f78fbe00640 1 -- 172.21.15.71:0/4056245745 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908044020 con 0x55f8430513b0 2024-09-05T12:46:53.545 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f78fbe00640 1 -- 172.21.15.71:0/4056245745 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9438+0+0 (secure 0 0 0) 0x7f7908004c10 con 0x55f8430513b0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79080047a0 con 0x55f8430513b0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f78fbe00640 1 -- 172.21.15.71:0/4056245745 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f790804f050 con 0x55f8430513b0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 10 client.27506 fetch_fsmap learned FSMap version 76 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55f8430d6110 con 0x55f8430513b0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f78fbe00640 1 -- 172.21.15.71:0/4056245745 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79080047a0 con 0x55f8430513b0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f78fbe00640 1 -- 172.21.15.71:0/4056245745 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f790802cb30 con 0x55f8430513b0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 10 client.27506 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 10 client.27506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 20 client.27506 populate_metadata read hostname 'smithi071' 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f8430d6850 0x55f8430d8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 4 client.27506 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f842fbeb80 con 0x55f8430d6850 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f790e000640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f8430d6850 0x55f8430d8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:46:53.546 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f790e000640 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f8430d6850 0x55f8430d8c70 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5356c74aff1258c3 server_cookie=5c1f9553ca8c8b60 in_seq=0 out_seq=0 2024-09-05T12:46:53.547 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.473+0000 7f78fbe00640 10 client.27506 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f78fbe00640 1 -- 172.21.15.71:0/4056245745 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f842fbeb80 con 0x55f8430d6850 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f78fbe00640 10 client.27506 handle_command_reply: tid=0 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 client.27506 shutdown 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 msgr2=0x55f843064dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 0x55f843064dc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7908019040 tx=0x7f7908009df0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 shutdown_connections 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f8430d6850 0x55f8430d8c70 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.548 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f84305c570 0x55f84305fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f843051cd0 0x55f84305f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f78fbe00640 0 client.27506 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 --2- 172.21.15.71:0/4056245745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f8430513b0 0x55f843064dc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 >> 172.21.15.71:0/4056245745 conn(0x55f84304d920 msgr2=0x55f843057d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 shutdown_connections 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 1 -- 172.21.15.71:0/4056245745 wait complete. 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f792047e740 20 client.27506 trim_cache size 0 max 16384 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:46:53.549 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:46:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:46:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:46:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78fc0a2cf0 msgr2=0x7f78fc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f7917e00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78fc0a2cf0 0x7f78fc0a51b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7908009d20 tx=0x7f7908005f90 comp rx=0 tx=0).stop 2024-09-05T12:46:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 msgr2=0x7f7918074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:46:53.550 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.520+0000 7f7917e00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 0x7f7918074f70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7910009fd0 tx=0x7f791000ef10 comp rx=0 tx=0).stop 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 shutdown_connections 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79181078c0 0x7f7918074f70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78fc0a2cf0 0x7f78fc0a51b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7918102250 0x7f7918074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 --2- 172.21.15.71:0/963098964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79181018b0 0x7f79180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 >> 172.21.15.71:0/963098964 conn(0x7f79180fd5d0 msgr2=0x7f79180fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 shutdown_connections 2024-09-05T12:46:53.551 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:46:53.525+0000 7f7917e00640 1 -- 172.21.15.71:0/963098964 wait complete. 2024-09-05T12:46:53.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101085 14.22 MB/sec execute 227 sec latency 18849.170 ms 2024-09-05T12:46:54.847 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101085 14.16 MB/sec execute 228 sec latency 19849.220 ms 2024-09-05T12:46:54.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:54 smithi071 ceph-mon[29845]: pgmap v604: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 5.5 MiB/s wr, 205 op/s 2024-09-05T12:46:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:54 smithi123 ceph-mon[34321]: pgmap v604: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 5.5 MiB/s wr, 205 op/s 2024-09-05T12:46:55.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:54 smithi174 ceph-mon[34313]: pgmap v604: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 859 KiB/s rd, 5.5 MiB/s wr, 205 op/s 2024-09-05T12:46:55.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101144 14.10 MB/sec execute 229 sec latency 20849.263 ms 2024-09-05T12:46:56.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101166 14.05 MB/sec execute 230 sec latency 21849.315 ms 2024-09-05T12:46:56.861 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:56 smithi071 ceph-mon[29845]: pgmap v605: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.1 MiB/s wr, 135 op/s 2024-09-05T12:46:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:56 smithi123 ceph-mon[34321]: pgmap v605: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.1 MiB/s wr, 135 op/s 2024-09-05T12:46:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:56 smithi174 ceph-mon[34313]: pgmap v605: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 3.1 MiB/s wr, 135 op/s 2024-09-05T12:46:57.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101168 13.99 MB/sec execute 231 sec latency 22849.409 ms 2024-09-05T12:46:58.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101168 13.93 MB/sec execute 232 sec latency 23849.460 ms 2024-09-05T12:46:58.853 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:46:58 smithi071 ceph-mon[29845]: pgmap v606: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-05T12:46:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:46:58 smithi123 ceph-mon[34321]: pgmap v606: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-05T12:46:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:46:58 smithi174 ceph-mon[34313]: pgmap v606: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.2 MiB/s wr, 154 op/s 2024-09-05T12:46:59.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101168 13.87 MB/sec execute 233 sec latency 24849.514 ms 2024-09-05T12:47:00.840 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:00 smithi071 ceph-mon[29845]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 187 KiB/s wr, 37 op/s 2024-09-05T12:47:00.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101168 13.81 MB/sec execute 234 sec latency 25849.571 ms 2024-09-05T12:47:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:00 smithi123 ceph-mon[34321]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 187 KiB/s wr, 37 op/s 2024-09-05T12:47:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:00 smithi174 ceph-mon[34313]: pgmap v607: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 187 KiB/s wr, 37 op/s 2024-09-05T12:47:01.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101168 13.76 MB/sec execute 235 sec latency 26849.646 ms 2024-09-05T12:47:02.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101168 13.70 MB/sec execute 236 sec latency 27849.697 ms 2024-09-05T12:47:02.862 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:02 smithi071 ceph-mon[29845]: pgmap v608: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 378 KiB/s wr, 71 op/s 2024-09-05T12:47:02.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:02 smithi123 ceph-mon[34321]: pgmap v608: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 378 KiB/s wr, 71 op/s 2024-09-05T12:47:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:02 smithi174 ceph-mon[34313]: pgmap v608: 129 pgs: 129 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 378 KiB/s wr, 71 op/s 2024-09-05T12:47:03.699 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:03 smithi071 ceph-mon[29845]: osdmap e123: 12 total, 12 up, 12 in 2024-09-05T12:47:03.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101198 13.65 MB/sec execute 237 sec latency 28376.105 ms 2024-09-05T12:47:03.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:03 smithi123 ceph-mon[34321]: osdmap e123: 12 total, 12 up, 12 in 2024-09-05T12:47:04.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:03 smithi174 ceph-mon[34313]: osdmap e123: 12 total, 12 up, 12 in 2024-09-05T12:47:04.838 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:04 smithi071 ceph-mon[29845]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 437 KiB/s wr, 107 op/s 2024-09-05T12:47:04.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101300 13.63 MB/sec execute 238 sec latency 26452.219 ms 2024-09-05T12:47:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:04 smithi123 ceph-mon[34321]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 437 KiB/s wr, 107 op/s 2024-09-05T12:47:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:04 smithi174 ceph-mon[34313]: pgmap v610: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 437 KiB/s wr, 107 op/s 2024-09-05T12:47:05.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101300 13.57 MB/sec execute 239 sec latency 27452.275 ms 2024-09-05T12:47:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 -- 172.21.15.71:0/3436672619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058101ac0 msgr2=0x7fd058101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 --2- 172.21.15.71:0/3436672619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058101ac0 0x7fd058101f20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd044009930 tx=0x7fd04402a440 comp rx=0 tx=0).stop 2024-09-05T12:47:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 -- 172.21.15.71:0/3436672619 shutdown_connections 2024-09-05T12:47:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 --2- 172.21.15.71:0/3436672619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd058102460 0x7fd05810e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 --2- 172.21.15.71:0/3436672619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058101ac0 0x7fd058101f20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 --2- 172.21.15.71:0/3436672619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd058107ad0 0x7fd058107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 -- 172.21.15.71:0/3436672619 >> 172.21.15.71:0/3436672619 conn(0x7fd0580fd7e0 msgr2=0x7fd0580ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 -- 172.21.15.71:0/3436672619 shutdown_connections 2024-09-05T12:47:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 -- 172.21.15.71:0/3436672619 wait complete. 2024-09-05T12:47:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.670+0000 7fd05e600640 1 Processor -- start 2024-09-05T12:47:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 -- start start 2024-09-05T12:47:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd058101ac0 0x7fd0581983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd058102460 0x7fd058198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 0x7fd05819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111f60 con 0x7fd058102460 2024-09-05T12:47:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd058111de0 con 0x7fd058101ac0 2024-09-05T12:47:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581120e0 con 0x7fd058107ad0 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 0x7fd05819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 0x7fd05819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40874/0 (socket says 172.21.15.71:40874) 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 -- 172.21.15.71:0/3298368212 learned_addr learned my addr 172.21.15.71:0/3298368212 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057400640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd058101ac0 0x7fd0581983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 -- 172.21.15.71:0/3298368212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd058101ac0 msgr2=0x7fd0581983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd058101ac0 0x7fd0581983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 -- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd058102460 msgr2=0x7fd058198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd058102460 0x7fd058198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 -- 172.21.15.71:0/3298368212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05819d3b0 con 0x7fd058107ad0 2024-09-05T12:47:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd057e00640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 0x7fd05819ccb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd05000ead0 tx=0x7fd05000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd04fe00640 1 -- 172.21.15.71:0/3298368212 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05000cde0 con 0x7fd058107ad0 2024-09-05T12:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd04fe00640 1 -- 172.21.15.71:0/3298368212 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd050005020 con 0x7fd058107ad0 2024-09-05T12:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd04fe00640 1 -- 172.21.15.71:0/3298368212 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500116d0 con 0x7fd058107ad0 2024-09-05T12:47:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05819d6a0 con 0x7fd058107ad0 2024-09-05T12:47:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.675+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05806bc10 con 0x7fd058107ad0 2024-09-05T12:47:06.683 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.682+0000 7fd04fe00640 1 -- 172.21.15.71:0/3298368212 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fd050005540 con 0x7fd058107ad0 2024-09-05T12:47:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.687+0000 7fd04fe00640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd03c0a7390 0x7fd03c0a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.689 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.687+0000 7fd04fe00640 1 -- 172.21.15.71:0/3298368212 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9526+0+0 (secure 0 0 0) 0x7fd0500cfb40 con 0x7fd058107ad0 2024-09-05T12:47:06.690 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.687+0000 7fd057400640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd03c0a7390 0x7fd03c0a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.687+0000 7fd057400640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd03c0a7390 0x7fd03c0a9850 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd048009890 tx=0x7fd048008040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.692+0000 7fd0615cd740 1 Processor -- start 2024-09-05T12:47:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.692+0000 7fd0615cd740 1 -- start start 2024-09-05T12:47:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4e9f0 0x557af1e4edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e59670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e5bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd0615cd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557af1dac100 con 0x557af1e59bb0 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd0615cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557af1d541b0 con 0x557af1e4e9f0 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd0615cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557af1da80c0 con 0x557af1e4f310 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e5bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e5bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52394/0 (socket says 172.21.15.71:52394) 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 -- 172.21.15.71:0/848030 learned_addr learned my addr 172.21.15.71:0/848030 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057400640 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4e9f0 0x557af1e4edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd056a00640 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e59670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 -- 172.21.15.71:0/848030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 msgr2=0x557af1e59670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e59670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 -- 172.21.15.71:0/848030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4e9f0 msgr2=0x557af1e4edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4e9f0 0x557af1e4edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057e00640 1 -- 172.21.15.71:0/848030 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557af1abf210 con 0x557af1e59bb0 2024-09-05T12:47:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.694+0000 7fd057400640 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4e9f0 0x557af1e4edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd057e00640 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e5bfa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c700 tx=0x7fd05009b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cfe64a52f3132c server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd04cc00640 1 -- 172.21.15.71:0/848030 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05001f040 con 0x557af1e59bb0 2024-09-05T12:47:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd04cc00640 1 -- 172.21.15.71:0/848030 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0500b2030 con 0x557af1e59bb0 2024-09-05T12:47:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd04cc00640 1 -- 172.21.15.71:0/848030 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05009b060 con 0x557af1e59bb0 2024-09-05T12:47:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd0615cd740 1 -- 172.21.15.71:0/848030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 msgr2=0x557af1e5bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd0615cd740 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e5bfa0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c700 tx=0x7fd05009b6a0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.699+0000 7fd0615cd740 1 -- 172.21.15.71:0/848030 shutdown_connections 2024-09-05T12:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e5bfa0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e59670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 --2- 172.21.15.71:0/848030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4e9f0 0x557af1e4edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 -- 172.21.15.71:0/848030 >> 172.21.15.71:0/848030 conn(0x557af1e4af60 msgr2=0x557af1e4b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 -- 172.21.15.71:0/848030 shutdown_connections 2024-09-05T12:47:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 -- 172.21.15.71:0/848030 wait complete. 2024-09-05T12:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 -- start start 2024-09-05T12:47:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.704+0000 7fd0615cd740 1 Processor -- start 2024-09-05T12:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e56bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4f310 0x557af1e57110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e57650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x557af1e5c7e0 con 0x557af1e59bb0 2024-09-05T12:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557af1d35090 con 0x557af1e4f310 2024-09-05T12:47:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557af1e5c960 con 0x557af1e4e9f0 2024-09-05T12:47:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e56bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e56bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:40890/0 (socket says 172.21.15.71:40890) 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 -- 172.21.15.71:0/340401653 learned_addr learned my addr 172.21.15.71:0/340401653 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 10 client.0 _ll_register_callbacks cb 0x557af1d89f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd056a00640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4f310 0x557af1e57110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057e00640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e57650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 -- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4f310 msgr2=0x557af1e57110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4f310 0x557af1e57110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 -- 172.21.15.71:0/340401653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 msgr2=0x557af1e57650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e57650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 -- 172.21.15.71:0/340401653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557af1b8b620 con 0x557af1e4e9f0 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd056a00640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4f310 0x557af1e57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:06.709 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057e00640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd041600640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:47:06.710 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd057400640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e56bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd048009330 tx=0x7fd0480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2694244df567136b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd041600640 1 -- 172.21.15.71:0/340401653 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048014e40 con 0x557af1e4e9f0 2024-09-05T12:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd041600640 1 -- 172.21.15.71:0/340401653 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9526+0+0 (secure 0 0 0) 0x7fd04801d040 con 0x557af1e4e9f0 2024-09-05T12:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0500b2030 con 0x557af1e4e9f0 2024-09-05T12:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd041600640 1 -- 172.21.15.71:0/340401653 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd048014e40 con 0x557af1e4e9f0 2024-09-05T12:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 10 client.27559 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:06.711 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x557af1e68530 con 0x557af1e4e9f0 2024-09-05T12:47:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd041600640 1 -- 172.21.15.71:0/340401653 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0500b2030 con 0x557af1e4e9f0 2024-09-05T12:47:06.712 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.706+0000 7fd041600640 1 -- 172.21.15.71:0/340401653 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7fd048013070 con 0x557af1e4e9f0 2024-09-05T12:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd0615cd740 10 client.27559 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd0615cd740 10 client.27559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd0615cd740 20 client.27559 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd0615cd740 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1e699f0 0x557af1e6bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.713 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd0615cd740 4 client.27559 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:47:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557af1db9830 con 0x557af1e699f0 2024-09-05T12:47:06.714 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd056a00640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1e699f0 0x557af1e6bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd056a00640 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1e699f0 0x557af1e6bde0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc65477ee8e009dc server_cookie=95313f7053b54c99 in_seq=0 out_seq=0 2024-09-05T12:47:06.715 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.712+0000 7fd041600640 10 client.27559 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd041600640 1 -- 172.21.15.71:0/340401653 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557af1db9830 con 0x557af1e699f0 2024-09-05T12:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd041600640 10 client.27559 handle_command_reply: tid=0 2024-09-05T12:47:06.716 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 client.27559 shutdown 2024-09-05T12:47:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 msgr2=0x557af1e56bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e56bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd048009330 tx=0x7fd0480046c0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 shutdown_connections 2024-09-05T12:47:06.717 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1e699f0 0x557af1e6bde0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e59bb0 0x557af1e57650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e4f310 0x557af1e57110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 --2- 172.21.15.71:0/340401653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e56bd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 >> 172.21.15.71:0/340401653 conn(0x557af1e4af60 msgr2=0x557af1e50c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:06.718 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd041600640 0 client.27559 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 shutdown_connections 2024-09-05T12:47:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.715+0000 7fd0615cd740 1 -- 172.21.15.71:0/340401653 wait complete. 2024-09-05T12:47:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.718+0000 7fd0615cd740 20 client.27559 trim_cache size 0 max 16384 2024-09-05T12:47:06.760 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 Processor -- start 2024-09-05T12:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 -- start start 2024-09-05T12:47:06.761 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e3d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 0x557af1e71680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd058076050 con 0x557af1e4f310 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581b74d0 con 0x557af1e59bb0 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd0615cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581b6060 con 0x557af1e4e9f0 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd056a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 0x557af1e71680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.762 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd056a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 0x557af1e71680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52408/0 (socket says 172.21.15.71:52408) 2024-09-05T12:47:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd056a00640 1 -- 172.21.15.71:0/3909718441 learned_addr learned my addr 172.21.15.71:0/3909718441 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.763 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 -- 172.21.15.71:0/3909718441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 msgr2=0x557af1e3d8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e3d8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 -- 172.21.15.71:0/3909718441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 msgr2=0x557af1e71680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 0x557af1e71680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 -- 172.21.15.71:0/3909718441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557af1e55b60 con 0x557af1e59bb0 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd056a00640 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 0x557af1e71680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd057e00640 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e71bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c700 tx=0x7fd05009b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a83affeed20a926 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd043e00640 1 -- 172.21.15.71:0/3909718441 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0500b8070 con 0x557af1e59bb0 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd043e00640 1 -- 172.21.15.71:0/3909718441 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0500b2040 con 0x557af1e59bb0 2024-09-05T12:47:06.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.758+0000 7fd043e00640 1 -- 172.21.15.71:0/3909718441 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05001f040 con 0x557af1e59bb0 2024-09-05T12:47:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 -- 172.21.15.71:0/3909718441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 msgr2=0x557af1e71bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e71bc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c700 tx=0x7fd05009b860 comp rx=0 tx=0).stop 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 -- 172.21.15.71:0/3909718441 shutdown_connections 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e71bc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4f310 0x557af1e71680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 --2- 172.21.15.71:0/3909718441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4e9f0 0x557af1e3d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 -- 172.21.15.71:0/3909718441 >> 172.21.15.71:0/3909718441 conn(0x557af1e4af60 msgr2=0x557af1e58b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 -- 172.21.15.71:0/3909718441 shutdown_connections 2024-09-05T12:47:06.766 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 -- 172.21.15.71:0/3909718441 wait complete. 2024-09-05T12:47:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 -- start start 2024-09-05T12:47:06.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.764+0000 7fd0615cd740 1 Processor -- start 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 0x557af1e5d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e5df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e5e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581b6e00 con 0x557af1e4e9f0 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0581b6730 con 0x557af1e59bb0 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd058101fb0 con 0x557af1e4f310 2024-09-05T12:47:06.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd0615cd740 10 client.0 _ll_register_callbacks cb 0x7fd048009480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd057400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 0x557af1e5d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd057400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 0x557af1e5d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:52424/0 (socket says 172.21.15.71:52424) 2024-09-05T12:47:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd057400640 1 -- 172.21.15.71:0/603972730 learned_addr learned my addr 172.21.15.71:0/603972730 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd056a00640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e5df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.770 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd057e00640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e5e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.772 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.766+0000 7fd057400640 1 -- 172.21.15.71:0/603972730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 msgr2=0x557af1e5df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd057400640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e5df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd057400640 1 -- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 msgr2=0x557af1e5e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd057400640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e5e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd057400640 1 -- 172.21.15.71:0/603972730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557af1ed3720 con 0x557af1e4e9f0 2024-09-05T12:47:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd056a00640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e5df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:47:06.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd057e00640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e5e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:47:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd057400640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 0x557af1e5d9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd048016010 tx=0x7fd0480125f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f41c8ad12bb439 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd041600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:06.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd041600640 1 -- 172.21.15.71:0/603972730 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048014e40 con 0x557af1e4e9f0 2024-09-05T12:47:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0500b2040 con 0x557af1e4e9f0 2024-09-05T12:47:06.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd041600640 1 -- 172.21.15.71:0/603972730 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9526+0+0 (secure 0 0 0) 0x7fd04800f040 con 0x557af1e4e9f0 2024-09-05T12:47:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.772+0000 7fd041600640 1 -- 172.21.15.71:0/603972730 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd048012ca0 con 0x557af1e4e9f0 2024-09-05T12:47:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.777+0000 7fd0615cd740 10 client.17499 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.777+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x557af1ed4a10 con 0x557af1e4e9f0 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd041600640 1 -- 172.21.15.71:0/603972730 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0500b2040 con 0x557af1e4e9f0 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd041600640 1 -- 172.21.15.71:0/603972730 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7fd04803a030 con 0x557af1e4e9f0 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd0615cd740 10 client.17499 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd0615cd740 10 client.17499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd0615cd740 20 client.17499 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd0615cd740 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1ed5150 0x557af1ed7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd0615cd740 4 client.17499 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557af1db3b80 con 0x557af1ed5150 2024-09-05T12:47:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd056a00640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1ed5150 0x557af1ed7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd041600640 10 client.17499 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:06.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.778+0000 7fd056a00640 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1ed5150 0x557af1ed7570 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f336e296e3d49043 server_cookie=fc0e4154a7913641 in_seq=0 out_seq=0 2024-09-05T12:47:06.803 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:06 smithi071 ceph-mon[29845]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 437 KiB/s wr, 107 op/s 2024-09-05T12:47:06.848 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101314 13.51 MB/sec execute 240 sec latency 28201.403 ms 2024-09-05T12:47:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.884+0000 7fd041600640 1 -- 172.21.15.71:0/603972730 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557af1db3b80 con 0x557af1ed5150 2024-09-05T12:47:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.884+0000 7fd041600640 10 client.17499 handle_command_reply: tid=0 2024-09-05T12:47:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.884+0000 7fd0615cd740 1 client.17499 shutdown 2024-09-05T12:47:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 msgr2=0x557af1e5d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 0x557af1e5d9d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd048016010 tx=0x7fd0480125f0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 shutdown_connections 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x557af1ed5150 0x557af1ed7570 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557af1e59bb0 0x557af1e5e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557af1e4f310 0x557af1e5df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 --2- 172.21.15.71:0/603972730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557af1e4e9f0 0x557af1e5d9d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 >> 172.21.15.71:0/603972730 conn(0x557af1e4af60 msgr2=0x557af1e40f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd041600640 0 client.17499 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 shutdown_connections 2024-09-05T12:47:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 1 -- 172.21.15.71:0/603972730 wait complete. 2024-09-05T12:47:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd0615cd740 20 client.17499 trim_cache size 0 max 16384 2024-09-05T12:47:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:47:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:47:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:47:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:47:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd03c0a7390 msgr2=0x7fd03c0a9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd03c0a7390 0x7fd03c0a9850 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd048009890 tx=0x7fd048008040 comp rx=0 tx=0).stop 2024-09-05T12:47:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 msgr2=0x7fd05819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 0x7fd05819ccb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd05000ead0 tx=0x7fd05000ef90 comp rx=0 tx=0).stop 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 shutdown_connections 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd058107ad0 0x7fd05819ccb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fd03c0a7390 0x7fd03c0a9850 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd058102460 0x7fd058198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 --2- 172.21.15.71:0/3298368212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd058101ac0 0x7fd0581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.889+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 >> 172.21.15.71:0/3298368212 conn(0x7fd0580fd7e0 msgr2=0x7fd0580fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.894+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 shutdown_connections 2024-09-05T12:47:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:06.894+0000 7fd05e600640 1 -- 172.21.15.71:0/3298368212 wait complete. 2024-09-05T12:47:06.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:06 smithi123 ceph-mon[34321]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 437 KiB/s wr, 107 op/s 2024-09-05T12:47:07.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:06 smithi174 ceph-mon[34313]: pgmap v611: 129 pgs: 129 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 437 KiB/s wr, 107 op/s 2024-09-05T12:47:07.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 101794 13.52 MB/sec execute 241 sec latency 18014.610 ms 2024-09-05T12:47:08.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 102525 13.63 MB/sec execute 242 sec latency 453.202 ms 2024-09-05T12:47:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:08 smithi071 ceph-mon[29845]: pgmap v612: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1005 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 696 KiB/s wr, 125 op/s 2024-09-05T12:47:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:08 smithi123 ceph-mon[34321]: pgmap v612: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1005 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 696 KiB/s wr, 125 op/s 2024-09-05T12:47:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:08 smithi174 ceph-mon[34313]: pgmap v612: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1005 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 696 KiB/s wr, 125 op/s 2024-09-05T12:47:09.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 103091 13.72 MB/sec execute 243 sec latency 534.178 ms 2024-09-05T12:47:10.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 103705 13.81 MB/sec execute 244 sec latency 218.562 ms 2024-09-05T12:47:10.900 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:10 smithi071 ceph-mon[29845]: pgmap v613: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1003 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 695 KiB/s wr, 129 op/s 2024-09-05T12:47:10.900 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:10 smithi071 ceph-mon[29845]: osdmap e124: 12 total, 12 up, 12 in 2024-09-05T12:47:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:10 smithi123 ceph-mon[34321]: pgmap v613: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1003 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 695 KiB/s wr, 129 op/s 2024-09-05T12:47:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:10 smithi123 ceph-mon[34321]: osdmap e124: 12 total, 12 up, 12 in 2024-09-05T12:47:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:10 smithi174 ceph-mon[34313]: pgmap v613: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1003 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 729 KiB/s rd, 695 KiB/s wr, 129 op/s 2024-09-05T12:47:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:10 smithi174 ceph-mon[34313]: osdmap e124: 12 total, 12 up, 12 in 2024-09-05T12:47:11.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 104148 13.83 MB/sec execute 245 sec latency 183.731 ms 2024-09-05T12:47:12.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 105067 14.01 MB/sec execute 246 sec latency 80.800 ms 2024-09-05T12:47:12.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:12 smithi071 ceph-mon[29845]: pgmap v615: 129 pgs: 129 active+clean; 1007 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-09-05T12:47:12.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:12 smithi123 ceph-mon[34321]: pgmap v615: 129 pgs: 129 active+clean; 1007 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-09-05T12:47:13.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:12 smithi174 ceph-mon[34313]: pgmap v615: 129 pgs: 129 active+clean; 1007 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-09-05T12:47:13.849 INFO:tasks.workunit.client.0.smithi071.stdout: 10 105964 14.13 MB/sec execute 247 sec latency 185.068 ms 2024-09-05T12:47:14.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 106690 14.26 MB/sec execute 248 sec latency 243.639 ms 2024-09-05T12:47:14.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:14 smithi071 ceph-mon[29845]: pgmap v616: 129 pgs: 129 active+clean; 1008 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T12:47:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:14 smithi123 ceph-mon[34321]: pgmap v616: 129 pgs: 129 active+clean; 1008 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T12:47:15.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:14 smithi174 ceph-mon[34313]: pgmap v616: 129 pgs: 129 active+clean; 1008 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T12:47:15.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 107077 14.26 MB/sec execute 249 sec latency 350.535 ms 2024-09-05T12:47:15.864 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.9 2024-09-05T12:47:15.865 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:47:16.571 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:47:16.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:16 smithi071 ceph-mon[29845]: pgmap v617: 129 pgs: 129 active+clean; 1008 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T12:47:16.662 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:16 smithi071 ceph-mon[29845]: osdmap e125: 12 total, 12 up, 12 in 2024-09-05T12:47:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:16 smithi123 ceph-mon[34321]: pgmap v617: 129 pgs: 129 active+clean; 1008 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T12:47:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:16 smithi123 ceph-mon[34321]: osdmap e125: 12 total, 12 up, 12 in 2024-09-05T12:47:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:16 smithi174 ceph-mon[34313]: pgmap v617: 129 pgs: 129 active+clean; 1008 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.4 MiB/s wr, 349 op/s 2024-09-05T12:47:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:16 smithi174 ceph-mon[34313]: osdmap e125: 12 total, 12 up, 12 in 2024-09-05T12:47:16.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 107658 14.33 MB/sec execute 250 sec latency 360.517 ms 2024-09-05T12:47:17.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 -- 172.21.15.71:0/844250774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c105f20 msgr2=0x7fc64c1063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:17.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 --2- 172.21.15.71:0/844250774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c105f20 0x7fc64c1063b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc644009be0 tx=0x7fc64402a790 comp rx=0 tx=0).stop 2024-09-05T12:47:17.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 -- 172.21.15.71:0/844250774 shutdown_connections 2024-09-05T12:47:17.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 --2- 172.21.15.71:0/844250774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c105f20 0x7fc64c1063b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 --2- 172.21.15.71:0/844250774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64c101430 0x7fc64c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 --2- 172.21.15.71:0/844250774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 -- 172.21.15.71:0/844250774 >> 172.21.15.71:0/844250774 conn(0x7fc64c0fc780 msgr2=0x7fc64c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:17.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.309+0000 7fc654600640 1 -- 172.21.15.71:0/844250774 shutdown_connections 2024-09-05T12:47:17.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.310+0000 7fc654600640 1 -- 172.21.15.71:0/844250774 wait complete. 2024-09-05T12:47:17.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.311+0000 7fc654600640 1 Processor -- start 2024-09-05T12:47:17.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 -- start start 2024-09-05T12:47:17.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c1a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:17.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c101430 0x7fc64c1a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64c105f20 0x7fc64c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c118530 con 0x7fc64c105f20 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c1183b0 con 0x7fc64c101430 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc654600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc64c1186b0 con 0x7fc64c100b10 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc651c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c1a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc651c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c1a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50606/0 (socket says 172.21.15.71:50606) 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.312+0000 7fc651c00640 1 -- 172.21.15.71:0/1171716391 learned_addr learned my addr 172.21.15.71:0/1171716391 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc652600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64c105f20 0x7fc64c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651200640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c101430 0x7fc64c1a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:17.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651c00640 1 -- 172.21.15.71:0/1171716391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c101430 msgr2=0x7fc64c1a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:17.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651c00640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c101430 0x7fc64c1a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651c00640 1 -- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64c105f20 msgr2=0x7fc64c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:17.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651c00640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64c105f20 0x7fc64c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651c00640 1 -- 172.21.15.71:0/1171716391 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc64c19efb0 con 0x7fc64c100b10 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.313+0000 7fc651c00640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c1a4690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc64000e9c0 tx=0x7fc64000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.314+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc64000ccc0 con 0x7fc64c100b10 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.314+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc64c19f2a0 con 0x7fc64c100b10 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.314+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc64c06bc10 con 0x7fc64c100b10 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.314+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc640005500 con 0x7fc64c100b10 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.314+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc640010430 con 0x7fc64c100b10 2024-09-05T12:47:17.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.316+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc62c005290 con 0x7fc64c100b10 2024-09-05T12:47:17.320 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.318+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fc6400046c0 con 0x7fc64c100b10 2024-09-05T12:47:17.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.320+0000 7fc64a000640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6340a2d70 0x7fc6340a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:17.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.321+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7fc6400cfa80 con 0x7fc64c100b10 2024-09-05T12:47:17.322 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.321+0000 7fc651200640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6340a2d70 0x7fc6340a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:17.323 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.322+0000 7fc651200640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6340a2d70 0x7fc6340a5230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0ffe50 tx=0x7fc63800b2e0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:17.327 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.326+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6400d1e20 con 0x7fc64c100b10 2024-09-05T12:47:17.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.757+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc62c005850 con 0x7fc64c100b10 2024-09-05T12:47:17.759 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.758+0000 7fc64a000640 1 -- 172.21.15.71:0/1171716391 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7fc64009c2d0 con 0x7fc64c100b10 2024-09-05T12:47:17.759 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:47:17.759 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:47:17.760 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.763+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6340a2d70 msgr2=0x7fc6340a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.763+0000 7fc654600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6340a2d70 0x7fc6340a5230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0ffe50 tx=0x7fc63800b2e0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 msgr2=0x7fc64c1a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c1a4690 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc64000e9c0 tx=0x7fc64000ee80 comp rx=0 tx=0).stop 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 shutdown_connections 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fc6340a2d70 0x7fc6340a5230 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.765 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc64c105f20 0x7fc64c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc64c101430 0x7fc64c1a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 --2- 172.21.15.71:0/1171716391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc64c100b10 0x7fc64c1a4690 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.764+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 >> 172.21.15.71:0/1171716391 conn(0x7fc64c0fc780 msgr2=0x7fc64c10f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.765+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 shutdown_connections 2024-09-05T12:47:17.766 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:17.765+0000 7fc654600640 1 -- 172.21.15.71:0/1171716391 wait complete. 2024-09-05T12:47:17.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 108528 14.43 MB/sec execute 251 sec latency 64.669 ms 2024-09-05T12:47:18.642 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:18 smithi071 ceph-mon[29845]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.4 MiB/s wr, 737 op/s 2024-09-05T12:47:18.642 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:18 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1171716391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:47:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:18 smithi123 ceph-mon[34321]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.4 MiB/s wr, 737 op/s 2024-09-05T12:47:18.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:18 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1171716391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:47:18.702 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 90 --expiration 140.9 --await-for 45.0' 2024-09-05T12:47:18.702 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db / --timeout 90 --expiration 140.9 --await-for 45.0 2024-09-05T12:47:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:18 smithi174 ceph-mon[34313]: pgmap v619: 129 pgs: 129 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.4 MiB/s wr, 737 op/s 2024-09-05T12:47:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:18 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1171716391' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:47:18.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 109524 14.62 MB/sec execute 252 sec latency 69.130 ms 2024-09-05T12:47:19.363 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:47:19.851 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110151 14.74 MB/sec execute 253 sec latency 105.726 ms 2024-09-05T12:47:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 -- 172.21.15.71:0/3690507579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1078c0 msgr2=0x7f940c107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.082 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 --2- 172.21.15.71:0/3690507579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1078c0 0x7f940c107ca0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f93fc009900 tx=0x7f93fc02a400 comp rx=0 tx=0).stop 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 -- 172.21.15.71:0/3690507579 shutdown_connections 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 --2- 172.21.15.71:0/3690507579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 --2- 172.21.15.71:0/3690507579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f940c1018b0 0x7f940c101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 --2- 172.21.15.71:0/3690507579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1078c0 0x7f940c107ca0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.081+0000 7f940be00640 1 -- 172.21.15.71:0/3690507579 >> 172.21.15.71:0/3690507579 conn(0x7f940c0fd5d0 msgr2=0x7f940c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.082+0000 7f940be00640 1 -- 172.21.15.71:0/3690507579 shutdown_connections 2024-09-05T12:47:20.083 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.082+0000 7f940be00640 1 -- 172.21.15.71:0/3690507579 wait complete. 2024-09-05T12:47:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.085+0000 7f940be00640 1 Processor -- start 2024-09-05T12:47:20.086 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.086+0000 7f940be00640 1 -- start start 2024-09-05T12:47:20.087 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1018b0 0x7f940c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940be00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c19a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f940c1078c0 0x7f940c19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940be00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f940c111d00 con 0x7f940c1078c0 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f940c111b80 con 0x7f940c1018b0 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940be00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f940c111e80 con 0x7f940c102250 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c19a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c19a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50626/0 (socket says 172.21.15.71:50626) 2024-09-05T12:47:20.088 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940a000640 1 -- 172.21.15.71:0/57318656 learned_addr learned my addr 172.21.15.71:0/57318656 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.087+0000 7f940aa00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1018b0 0x7f940c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.088+0000 7f940a000640 1 -- 172.21.15.71:0/57318656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1018b0 msgr2=0x7f940c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.088+0000 7f940a000640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1018b0 0x7f940c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.088+0000 7f940a000640 1 -- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f940c1078c0 msgr2=0x7f940c19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.088+0000 7f940a000640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f940c1078c0 0x7f940c19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.089 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.088+0000 7f940a000640 1 -- 172.21.15.71:0/57318656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c19f400 con 0x7f940c102250 2024-09-05T12:47:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.089+0000 7f940a000640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c19a950 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93f800d950 tx=0x7f93f800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.089+0000 7f9403400640 1 -- 172.21.15.71:0/57318656 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8017070 con 0x7f940c102250 2024-09-05T12:47:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.089+0000 7f9403400640 1 -- 172.21.15.71:0/57318656 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8009c30 con 0x7f940c102250 2024-09-05T12:47:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.089+0000 7f9403400640 1 -- 172.21.15.71:0/57318656 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f800f6b0 con 0x7f940c102250 2024-09-05T12:47:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.089+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c19f6f0 con 0x7f940c102250 2024-09-05T12:47:20.090 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.089+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c1a6f80 con 0x7f940c102250 2024-09-05T12:47:20.095 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.094+0000 7f9403400640 1 -- 172.21.15.71:0/57318656 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f93f8010460 con 0x7f940c102250 2024-09-05T12:47:20.098 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.097+0000 7f9403400640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f93f00a2df0 0x7f93f00a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.097+0000 7f9403400640 1 -- 172.21.15.71:0/57318656 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f93f80cf490 con 0x7f940c102250 2024-09-05T12:47:20.099 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.098+0000 7f940aa00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f93f00a2df0 0x7f93f00a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.102 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.100+0000 7f940aa00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f93f00a2df0 0x7f93f00a52b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0096c0 tx=0x7f93fc000bc0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.102+0000 7f9414a24740 1 Processor -- start 2024-09-05T12:47:20.103 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.102+0000 7f9414a24740 1 -- start start 2024-09-05T12:47:20.104 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f9414a24740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 0x5595e18dbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f9414a24740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dc4e0 0x5595e18e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f9414a24740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f9414a24740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5595e1839100 con 0x5595e18dc4e0 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f9414a24740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595e17e11b0 con 0x5595e18e6d80 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f9414a24740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595e18350c0 con 0x5595e18dbbc0 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.103+0000 7f940aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 0x5595e18dbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.104+0000 7f940aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 0x5595e18dbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50636/0 (socket says 172.21.15.71:50636) 2024-09-05T12:47:20.105 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.104+0000 7f940aa00640 1 -- 172.21.15.71:0/3907184796 learned_addr learned my addr 172.21.15.71:0/3907184796 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.104+0000 7f940aa00640 1 -- 172.21.15.71:0/3907184796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 msgr2=0x5595e18e9170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.105+0000 7f940aa00640 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e9170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.105+0000 7f940aa00640 1 -- 172.21.15.71:0/3907184796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dc4e0 msgr2=0x5595e18e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.105+0000 7f940aa00640 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dc4e0 0x5595e18e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.105+0000 7f940aa00640 1 -- 172.21.15.71:0/3907184796 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e154c210 con 0x5595e18dbbc0 2024-09-05T12:47:20.106 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.105+0000 7f940aa00640 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 0x5595e18dbfa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93fc004480 tx=0x7f93fc03b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ae7932ac779dd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.106+0000 7f93f7e00640 1 -- 172.21.15.71:0/3907184796 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc041070 con 0x5595e18dbbc0 2024-09-05T12:47:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.107+0000 7f93f7e00640 1 -- 172.21.15.71:0/3907184796 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93fc04d070 con 0x5595e18dbbc0 2024-09-05T12:47:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.107+0000 7f93f7e00640 1 -- 172.21.15.71:0/3907184796 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc041070 con 0x5595e18dbbc0 2024-09-05T12:47:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 -- 172.21.15.71:0/3907184796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 msgr2=0x5595e18dbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 0x5595e18dbfa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93fc004480 tx=0x7f93fc03b000 comp rx=0 tx=0).stop 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 -- 172.21.15.71:0/3907184796 shutdown_connections 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dc4e0 0x5595e18e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 --2- 172.21.15.71:0/3907184796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dbbc0 0x5595e18dbfa0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 -- 172.21.15.71:0/3907184796 >> 172.21.15.71:0/3907184796 conn(0x5595e18d8130 msgr2=0x5595e18d8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 -- 172.21.15.71:0/3907184796 shutdown_connections 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.108+0000 7f9414a24740 1 -- 172.21.15.71:0/3907184796 wait complete. 2024-09-05T12:47:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.109+0000 7f9414a24740 1 -- start start 2024-09-05T12:47:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.109+0000 7f9414a24740 1 Processor -- start 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.110+0000 7f9414a24740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f9414a24740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f9414a24740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f9414a24740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5595e18e9950 con 0x5595e18dbbc0 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f9414a24740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5595e17c2090 con 0x5595e18e6d80 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f9414a24740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5595e18e9ad0 con 0x5595e18dc4e0 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f9414a24740 10 client.0 _ll_register_callbacks cb 0x5595e1816f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:20.112 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f940a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f940a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50642/0 (socket says 172.21.15.71:50642) 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f940a000640 1 -- 172.21.15.71:0/2937495459 learned_addr learned my addr 172.21.15.71:0/2937495459 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f940aa00640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.111+0000 7f940b400640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.112+0000 7f940aa00640 1 -- 172.21.15.71:0/2937495459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 msgr2=0x5595e18e4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.112+0000 7f940aa00640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.112+0000 7f940aa00640 1 -- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 msgr2=0x5595e18e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.112+0000 7f940aa00640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.112+0000 7f940aa00640 1 -- 172.21.15.71:0/2937495459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e1618620 con 0x5595e18dbbc0 2024-09-05T12:47:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.113+0000 7f940aa00640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18e3d50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f93fc005410 tx=0x7f93fc003670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20637314c0a365f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.113+0000 7f93f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.114+0000 7f93f4c00640 1 -- 172.21.15.71:0/2937495459 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc056070 con 0x5595e18dbbc0 2024-09-05T12:47:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.114+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93fc04d070 con 0x5595e18dbbc0 2024-09-05T12:47:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.115+0000 7f93f4c00640 1 -- 172.21.15.71:0/2937495459 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f93fc003ba0 con 0x5595e18dbbc0 2024-09-05T12:47:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.115+0000 7f93f4c00640 1 -- 172.21.15.71:0/2937495459 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f93fc062050 con 0x5595e18dbbc0 2024-09-05T12:47:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.115+0000 7f9414a24740 10 client.17511 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.115+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5595e1906660 con 0x5595e18dbbc0 2024-09-05T12:47:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.116+0000 7f93f4c00640 1 -- 172.21.15.71:0/2937495459 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93fc04d070 con 0x5595e18dbbc0 2024-09-05T12:47:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f93f4c00640 1 -- 172.21.15.71:0/2937495459 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f93fc032910 con 0x5595e18dbbc0 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f9414a24740 10 client.17511 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f9414a24740 10 client.17511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f9414a24740 20 client.17511 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f9414a24740 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1907b60 0x5595e1909f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f9414a24740 4 client.17511 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595e1846830 con 0x5595e1907b60 2024-09-05T12:47:20.118 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.117+0000 7f940a000640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1907b60 0x5595e1909f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.118+0000 7f940a000640 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1907b60 0x5595e1909f50 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67dfb6a6012370c7 server_cookie=e2241574d56c8b46 in_seq=0 out_seq=0 2024-09-05T12:47:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.119+0000 7f93f4c00640 10 client.17511 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.120+0000 7f93f4c00640 1 -- 172.21.15.71:0/2937495459 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595e1846830 con 0x5595e1907b60 2024-09-05T12:47:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.120+0000 7f93f4c00640 10 client.17511 handle_command_reply: tid=0 2024-09-05T12:47:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.120+0000 7f9414a24740 1 client.17511 shutdown 2024-09-05T12:47:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.121+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 msgr2=0x5595e18e3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.121+0000 7f9414a24740 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18e3d50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f93fc005410 tx=0x7f93fc003670 comp rx=0 tx=0).stop 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 shutdown_connections 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1907b60 0x5595e1909f50 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18e6d80 0x5595e18e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 --2- 172.21.15.71:0/2937495459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18e3d50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f93f4c00640 0 client.17511 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 >> 172.21.15.71:0/2937495459 conn(0x5595e18d8130 msgr2=0x5595e18dde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.122+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 shutdown_connections 2024-09-05T12:47:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.123+0000 7f9414a24740 1 -- 172.21.15.71:0/2937495459 wait complete. 2024-09-05T12:47:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.124+0000 7f9414a24740 20 client.17511 trim_cache size 0 max 16384 2024-09-05T12:47:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.159+0000 7f9414a24740 1 Processor -- start 2024-09-05T12:47:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.159+0000 7f9414a24740 1 -- start start 2024-09-05T12:47:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.159+0000 7f9414a24740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 0x5595e18de5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.160 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.159+0000 7f9414a24740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.159+0000 7f9414a24740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 0x5595e190f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 0x5595e18de5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.161 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 0x5595e18de5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57902/0 (socket says 172.21.15.71:57902) 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50652/0 (socket says 172.21.15.71:50652) 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 0x5595e190f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 0x5595e190f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44120/0 (socket says 172.21.15.71:44120) 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f940aa00640 1 -- 172.21.15.71:0/1152527691 learned_addr learned my addr 172.21.15.71:0/1152527691 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f940c071610 con 0x5595e18e6d80 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f940c073150 con 0x5595e18dbbc0 2024-09-05T12:47:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.160+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f940c071ce0 con 0x5595e18dc4e0 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940b400640 1 -- 172.21.15.71:0/1152527691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 msgr2=0x5595e18e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940b400640 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940b400640 1 -- 172.21.15.71:0/1152527691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 msgr2=0x5595e18de5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940b400640 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 0x5595e18de5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940b400640 1 -- 172.21.15.71:0/1152527691 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595e18d7bd0 con 0x5595e18e6d80 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940aa00640 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 0x5595e18de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.161+0000 7f940b400640 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 0x5595e190f7d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9404005fd0 tx=0x7f9404011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca02f1a2e683a1d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.162+0000 7f93f7400640 1 -- 172.21.15.71:0/1152527691 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9404004b20 con 0x5595e18e6d80 2024-09-05T12:47:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.163+0000 7f93f7400640 1 -- 172.21.15.71:0/1152527691 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404003e50 con 0x5595e18e6d80 2024-09-05T12:47:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 msgr2=0x5595e190f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 0x5595e190f7d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9404005fd0 tx=0x7f9404011810 comp rx=0 tx=0).stop 2024-09-05T12:47:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 shutdown_connections 2024-09-05T12:47:20.165 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18e6d80 0x5595e190f7d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18dc4e0 0x5595e18e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 --2- 172.21.15.71:0/1152527691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dbbc0 0x5595e18de5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 >> 172.21.15.71:0/1152527691 conn(0x5595e18d8130 msgr2=0x5595e18e5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 shutdown_connections 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 -- 172.21.15.71:0/1152527691 wait complete. 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.164+0000 7f9414a24740 1 -- start start 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.165+0000 7f9414a24740 1 Processor -- start 2024-09-05T12:47:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.165+0000 7f9414a24740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.165+0000 7f9414a24740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dc4e0 0x5595e18e1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.165+0000 7f9414a24740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18e6d80 0x5595e18e1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f9414a24740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f940c072a80 con 0x5595e18dbbc0 2024-09-05T12:47:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f940b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18e6d80 0x5595e18e1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f940b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18e6d80 0x5595e18e1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50654/0 (socket says 172.21.15.71:50654) 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f940b400640 1 -- 172.21.15.71:0/3256347318 learned_addr learned my addr 172.21.15.71:0/3256347318 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f940a000640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dc4e0 0x5595e18e1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f940c0723b0 con 0x5595e18dc4e0 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f940c06f650 con 0x5595e18e6d80 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f9414a24740 10 client.0 _ll_register_callbacks cb 0x7f93fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.166+0000 7f940aa00640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.167+0000 7f940aa00640 1 -- 172.21.15.71:0/3256347318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18e6d80 msgr2=0x5595e18e1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.167+0000 7f940aa00640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18e6d80 0x5595e18e1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.167+0000 7f940aa00640 1 -- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dc4e0 msgr2=0x5595e18e1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.167+0000 7f940aa00640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dc4e0 0x5595e18e1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.167+0000 7f940aa00640 1 -- 172.21.15.71:0/3256347318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595e18e1bb0 con 0x5595e18dbbc0 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.167+0000 7f940a000640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dc4e0 0x5595e18e1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.168+0000 7f940aa00640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18ef240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0092a0 tx=0x7f93fc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514ab4fbba365d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.169 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.168+0000 7f93f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.169+0000 7f93f4c00640 1 -- 172.21.15.71:0/3256347318 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93fc06b060 con 0x5595e18dbbc0 2024-09-05T12:47:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.169+0000 7f93f4c00640 1 -- 172.21.15.71:0/3256347318 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f93fc032550 con 0x5595e18dbbc0 2024-09-05T12:47:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.169+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9404003e50 con 0x5595e18dbbc0 2024-09-05T12:47:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.170+0000 7f93f4c00640 1 -- 172.21.15.71:0/3256347318 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f93fc056070 con 0x5595e18dbbc0 2024-09-05T12:47:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.170+0000 7f9414a24740 10 client.17517 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.170+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x5595e19717a0 con 0x5595e18dbbc0 2024-09-05T12:47:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.170+0000 7f93f4c00640 1 -- 172.21.15.71:0/3256347318 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404003e50 con 0x5595e18dbbc0 2024-09-05T12:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f93f4c00640 1 -- 172.21.15.71:0/3256347318 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f93fc032910 con 0x5595e18dbbc0 2024-09-05T12:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f9414a24740 10 client.17517 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f9414a24740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f9414a24740 20 client.17517 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f9414a24740 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1971ce0 0x5595e1974100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f9414a24740 4 client.17517 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:47:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.171+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595e1840b80 con 0x5595e1971ce0 2024-09-05T12:47:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.173+0000 7f940a000640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1971ce0 0x5595e1974100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.173+0000 7f940a000640 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1971ce0 0x5595e1974100 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac4e6a97ef3c69b server_cookie=3419bd7bbb67f296 in_seq=0 out_seq=0 2024-09-05T12:47:20.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.173+0000 7f93f4c00640 10 client.17517 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.201+0000 7f457c200640 1 -- 172.21.15.71:0/577377765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109450 msgr2=0x7f4574109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.203 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.201+0000 7f457c200640 1 --2- 172.21.15.71:0/577377765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109450 0x7f4574109830 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f4568008570 tx=0x7f456802e360 comp rx=0 tx=0).stop 2024-09-05T12:47:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 -- 172.21.15.71:0/577377765 shutdown_connections 2024-09-05T12:47:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 --2- 172.21.15.71:0/577377765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f457410ea80 0x7f45741154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 --2- 172.21.15.71:0/577377765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4574109e00 0x7f457410e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 --2- 172.21.15.71:0/577377765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109450 0x7f4574109830 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 -- 172.21.15.71:0/577377765 >> 172.21.15.71:0/577377765 conn(0x7f457406e2d0 msgr2=0x7f457406e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 -- 172.21.15.71:0/577377765 shutdown_connections 2024-09-05T12:47:20.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.203+0000 7f457c200640 1 -- 172.21.15.71:0/577377765 wait complete. 2024-09-05T12:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.205+0000 7f457c200640 1 Processor -- start 2024-09-05T12:47:20.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.205+0000 7f457c200640 1 -- start start 2024-09-05T12:47:20.208 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457c200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4574109450 0x7f45741a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457c200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109e00 0x7f45741a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457c200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 0x7f45741abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457c200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f45741185d0 con 0x7f4574109e00 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457c200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4574118450 con 0x7f4574109450 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457c200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4574118750 con 0x7f457410ea80 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 0x7f45741abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457a200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 0x7f45741abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50668/0 (socket says 172.21.15.71:50668) 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.206+0000 7f457a200640 1 -- 172.21.15.71:0/3998262202 learned_addr learned my addr 172.21.15.71:0/3998262202 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.207+0000 7f4579800640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4574109450 0x7f45741a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.209 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.207+0000 7f4578e00640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109e00 0x7f45741a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.208+0000 7f457a200640 1 -- 172.21.15.71:0/3998262202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4574109450 msgr2=0x7f45741a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.208+0000 7f457a200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4574109450 0x7f45741a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.208+0000 7f457a200640 1 -- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109e00 msgr2=0x7f45741a77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.209+0000 7f457a200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109e00 0x7f45741a77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.209+0000 7f457a200640 1 -- 172.21.15.71:0/3998262202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45741ac210 con 0x7f457410ea80 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.209+0000 7f4578e00640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109e00 0x7f45741a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:47:20.210 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.209+0000 7f457a200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 0x7f45741abb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f456c007bd0 tx=0x7f456c00f500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.210+0000 7f4572000640 1 -- 172.21.15.71:0/3998262202 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c019030 con 0x7f457410ea80 2024-09-05T12:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.210+0000 7f4572000640 1 -- 172.21.15.71:0/3998262202 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456c00fc10 con 0x7f457410ea80 2024-09-05T12:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.210+0000 7f4572000640 1 -- 172.21.15.71:0/3998262202 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c005930 con 0x7f457410ea80 2024-09-05T12:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.210+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45741ac500 con 0x7f457410ea80 2024-09-05T12:47:20.211 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.210+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457419e7b0 con 0x7f457410ea80 2024-09-05T12:47:20.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.216+0000 7f457eddb740 1 Processor -- start 2024-09-05T12:47:20.218 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.217+0000 7f4572000640 1 -- 172.21.15.71:0/3998262202 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f456c01c070 con 0x7f457410ea80 2024-09-05T12:47:20.219 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.218+0000 7f457eddb740 1 -- start start 2024-09-05T12:47:20.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.219+0000 7f457eddb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb583790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.220 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.220+0000 7f457eddb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb58e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.220+0000 7f457eddb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb590980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.221 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.220+0000 7f4572000640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f455c0a2df0 0x7f455c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f457eddb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb4e9530 con 0x55f9bb583cd0 2024-09-05T12:47:20.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f4579800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb58e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f457eddb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb490860 con 0x55f9bb58e590 2024-09-05T12:47:20.222 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f457eddb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb4e54f0 con 0x55f9bb5833b0 2024-09-05T12:47:20.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f4572000640 1 -- 172.21.15.71:0/3998262202 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f456c0cfba0 con 0x7f457410ea80 2024-09-05T12:47:20.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f4578e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb583790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f4578e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb583790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50670/0 (socket says 172.21.15.71:50670) 2024-09-05T12:47:20.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f4578e00640 1 -- 172.21.15.71:0/4157732675 learned_addr learned my addr 172.21.15.71:0/4157732675 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.223 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.221+0000 7f457a200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f455c0a2df0 0x7f455c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.223+0000 7f4579800640 1 -- 172.21.15.71:0/4157732675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 msgr2=0x55f9bb583790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.223+0000 7f4579800640 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb583790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.223+0000 7f4579800640 1 -- 172.21.15.71:0/4157732675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 msgr2=0x55f9bb590980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.223+0000 7f4579800640 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb590980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.225 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.223+0000 7f4579800640 1 -- 172.21.15.71:0/4157732675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bb25ab90 con 0x55f9bb583cd0 2024-09-05T12:47:20.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.226+0000 7f457a200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f455c0a2df0 0x7f455c0a52b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f456c0a4430 tx=0x7f456c00ba90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.227 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.226+0000 7f4579800640 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb58e050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4568006010 tx=0x7f4568005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2c6bc9c9fff661 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.227+0000 7f4566a00640 1 -- 172.21.15.71:0/4157732675 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456804b030 con 0x55f9bb583cd0 2024-09-05T12:47:20.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.227+0000 7f4566a00640 1 -- 172.21.15.71:0/4157732675 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456800f1d0 con 0x55f9bb583cd0 2024-09-05T12:47:20.228 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.227+0000 7f4566a00640 1 -- 172.21.15.71:0/4157732675 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568003850 con 0x55f9bb583cd0 2024-09-05T12:47:20.231 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.229+0000 7f457eddb740 1 -- 172.21.15.71:0/4157732675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 msgr2=0x55f9bb58e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.229+0000 7f457eddb740 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb58e050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4568006010 tx=0x7f4568005de0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.230+0000 7f457eddb740 1 -- 172.21.15.71:0/4157732675 shutdown_connections 2024-09-05T12:47:20.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.231+0000 7f457eddb740 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb590980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.231+0000 7f457eddb740 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb58e050 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.231+0000 7f457eddb740 1 --2- 172.21.15.71:0/4157732675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb583790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.232 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.231+0000 7f457eddb740 1 -- 172.21.15.71:0/4157732675 >> 172.21.15.71:0/4157732675 conn(0x55f9bb57f940 msgr2=0x55f9bb57fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.232+0000 7f457eddb740 1 -- 172.21.15.71:0/4157732675 shutdown_connections 2024-09-05T12:47:20.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.232+0000 7f457eddb740 1 -- 172.21.15.71:0/4157732675 wait complete. 2024-09-05T12:47:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.233+0000 7f457eddb740 1 -- start start 2024-09-05T12:47:20.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.234+0000 7f457eddb740 1 Processor -- start 2024-09-05T12:47:20.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 0x55f9bb58b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb583cd0 0x55f9bb58bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb58c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.239 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb590fb0 con 0x55f9bb5833b0 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb4a3e60 con 0x55f9bb58e590 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9bb591130 con 0x55f9bb583cd0 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.235+0000 7f457eddb740 10 client.0 _ll_register_callbacks cb 0x55f9bb4a8cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.236+0000 7f4578e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 0x55f9bb58b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.236+0000 7f4578e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 0x55f9bb58b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44168/0 (socket says 172.21.15.71:44168) 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.236+0000 7f4578e00640 1 -- 172.21.15.71:0/3112454987 learned_addr learned my addr 172.21.15.71:0/3112454987 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.236+0000 7f4578e00640 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb58c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.237+0000 7f4578e00640 1 -- 172.21.15.71:0/3112454987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb583cd0 msgr2=0x55f9bb58bc10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.237+0000 7f4578e00640 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb583cd0 0x55f9bb58bc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.237+0000 7f4578e00640 1 -- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 msgr2=0x55f9bb58c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.240 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.237+0000 7f4578e00640 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb58c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.237+0000 7f4578e00640 1 -- 172.21.15.71:0/3112454987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bb4af990 con 0x55f9bb5833b0 2024-09-05T12:47:20.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.238+0000 7f4578e00640 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 0x55f9bb58b6d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f45600057c0 tx=0x7f4560005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb23185badaa44b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.238+0000 7f455b400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:20.241 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.239+0000 7f455b400640 1 -- 172.21.15.71:0/3112454987 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456000b930 con 0x55f9bb5833b0 2024-09-05T12:47:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.239+0000 7f455b400640 1 -- 172.21.15.71:0/3112454987 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f456000bad0 con 0x55f9bb5833b0 2024-09-05T12:47:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.239+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456800f1d0 con 0x55f9bb5833b0 2024-09-05T12:47:20.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.241+0000 7f455b400640 1 -- 172.21.15.71:0/3112454987 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f456000bcd0 con 0x55f9bb5833b0 2024-09-05T12:47:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.241+0000 7f457eddb740 10 client.17526 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:20.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.241+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55f9bb59ce70 con 0x55f9bb5833b0 2024-09-05T12:47:20.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.242+0000 7f455b400640 1 -- 172.21.15.71:0/3112454987 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f456800f1d0 con 0x55f9bb5833b0 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f455b400640 1 -- 172.21.15.71:0/3112454987 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f4560041060 con 0x55f9bb5833b0 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f457eddb740 10 client.17526 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f457eddb740 10 client.17526 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f457eddb740 20 client.17526 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f457eddb740 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb5a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f457eddb740 4 client.17526 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:47:20.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.243+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9bb48c310 con 0x55f9bb59e330 2024-09-05T12:47:20.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.246+0000 7f4579800640 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb5a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.248+0000 7f4579800640 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb5a07f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f70ffcc89b265e server_cookie=c2cad21c18286864 in_seq=0 out_seq=0 2024-09-05T12:47:20.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.248+0000 7f455b400640 10 client.17526 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.250+0000 7f455b400640 1 -- 172.21.15.71:0/3112454987 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9bb48c310 con 0x55f9bb59e330 2024-09-05T12:47:20.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.250+0000 7f455b400640 10 client.17526 handle_command_reply: tid=0 2024-09-05T12:47:20.252 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.250+0000 7f457eddb740 1 client.17526 shutdown 2024-09-05T12:47:20.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 msgr2=0x55f9bb58b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 0x55f9bb58b6d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f45600057c0 tx=0x7f4560005230 comp rx=0 tx=0).stop 2024-09-05T12:47:20.253 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 shutdown_connections 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb5a07f0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb58e590 0x55f9bb58c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f455b400640 0 client.17526 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb583cd0 0x55f9bb58bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 --2- 172.21.15.71:0/3112454987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb5833b0 0x55f9bb58b6d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 >> 172.21.15.71:0/3112454987 conn(0x55f9bb57f940 msgr2=0x55f9bb57ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 shutdown_connections 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.252+0000 7f457eddb740 1 -- 172.21.15.71:0/3112454987 wait complete. 2024-09-05T12:47:20.254 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.253+0000 7f457eddb740 20 client.17526 trim_cache size 0 max 16384 2024-09-05T12:47:20.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.297+0000 7f457eddb740 1 Processor -- start 2024-09-05T12:47:20.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.297+0000 7f457eddb740 1 -- start start 2024-09-05T12:47:20.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f457eddb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb5833b0 0x55f9bb57ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f457eddb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb5a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.299 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f457eddb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 0x55f9bb5a6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f457eddb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b55e0 con 0x55f9bb583cd0 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f457eddb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b7120 con 0x55f9bb5833b0 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f457eddb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b5cb0 con 0x55f9bb58e590 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f4578e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb5833b0 0x55f9bb57ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f4578e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 0x55f9bb5a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f4578e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 0x55f9bb5a6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:50672/0 (socket says 172.21.15.71:50672) 2024-09-05T12:47:20.300 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.298+0000 7f4578e00640 1 -- 172.21.15.71:0/3728119707 learned_addr learned my addr 172.21.15.71:0/3728119707 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.299+0000 7f4578e00640 1 -- 172.21.15.71:0/3728119707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb5833b0 msgr2=0x55f9bb57ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.299+0000 7f4578e00640 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb5833b0 0x55f9bb57ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.299+0000 7f4578e00640 1 -- 172.21.15.71:0/3728119707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 msgr2=0x55f9bb5a60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:20.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.299+0000 7f4578e00640 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb5a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.301 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.299+0000 7f4578e00640 1 -- 172.21.15.71:0/3728119707 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9bb58dfa0 con 0x55f9bb58e590 2024-09-05T12:47:20.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.300+0000 7f4578e00640 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 0x55f9bb5a6600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4560004ab0 tx=0x7f45600048e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601ac94c7cc1beca server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.302 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.301+0000 7f4566000640 1 -- 172.21.15.71:0/3728119707 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600036d0 con 0x55f9bb58e590 2024-09-05T12:47:20.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.301+0000 7f4566000640 1 -- 172.21.15.71:0/3728119707 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4560003b40 con 0x55f9bb58e590 2024-09-05T12:47:20.303 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.301+0000 7f4566000640 1 -- 172.21.15.71:0/3728119707 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600116a0 con 0x55f9bb58e590 2024-09-05T12:47:20.304 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 -- 172.21.15.71:0/3728119707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 msgr2=0x55f9bb5a6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 0x55f9bb5a6600 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4560004ab0 tx=0x7f45600048e0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 -- 172.21.15.71:0/3728119707 shutdown_connections 2024-09-05T12:47:20.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb58e590 0x55f9bb5a6600 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb583cd0 0x55f9bb5a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 --2- 172.21.15.71:0/3728119707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb5833b0 0x55f9bb57ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.305 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.303+0000 7f457eddb740 1 -- 172.21.15.71:0/3728119707 >> 172.21.15.71:0/3728119707 conn(0x55f9bb57f940 msgr2=0x55f9bb585620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.304+0000 7f457eddb740 1 -- 172.21.15.71:0/3728119707 shutdown_connections 2024-09-05T12:47:20.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.304+0000 7f457eddb740 1 -- 172.21.15.71:0/3728119707 wait complete. 2024-09-05T12:47:20.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.305+0000 7f457eddb740 1 -- start start 2024-09-05T12:47:20.306 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.305+0000 7f457eddb740 1 Processor -- start 2024-09-05T12:47:20.308 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb5932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 0x55f9bb5937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb58e590 0x55f9bb593d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b6a50 con 0x55f9bb58e590 2024-09-05T12:47:20.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45741b6380 con 0x55f9bb583cd0 2024-09-05T12:47:20.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45741a67c0 con 0x55f9bb5833b0 2024-09-05T12:47:20.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.307+0000 7f457eddb740 10 client.0 _ll_register_callbacks cb 0x7f45600047b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 0x55f9bb5937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 0x55f9bb5937e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:57962/0 (socket says 172.21.15.71:57962) 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 -- 172.21.15.71:0/949089081 learned_addr learned my addr 172.21.15.71:0/949089081 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 -- 172.21.15.71:0/949089081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 msgr2=0x55f9bb5932a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb5932a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 -- 172.21.15.71:0/949089081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb58e590 msgr2=0x55f9bb593d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb58e590 0x55f9bb593d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.308+0000 7f4579800640 1 -- 172.21.15.71:0/949089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9bb6092a0 con 0x55f9bb583cd0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.309+0000 7f4579800640 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 0x55f9bb5937e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4568031020 tx=0x7f45680044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=899a36f830b72460 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.309+0000 7f455b400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.309+0000 7f455b400640 1 -- 172.21.15.71:0/949089081 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45680046c0 con 0x55f9bb583cd0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.309+0000 7f455b400640 1 -- 172.21.15.71:0/949089081 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f4568004860 con 0x55f9bb583cd0 2024-09-05T12:47:20.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.309+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4560003b40 con 0x55f9bb583cd0 2024-09-05T12:47:20.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.309+0000 7f455b400640 1 -- 172.21.15.71:0/949089081 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4568004a60 con 0x55f9bb583cd0 2024-09-05T12:47:20.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.311+0000 7f457eddb740 10 client.27557 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:20.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.311+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55f9bb60a560 con 0x55f9bb583cd0 2024-09-05T12:47:20.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.311+0000 7f455b400640 1 -- 172.21.15.71:0/949089081 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4560003b40 con 0x55f9bb583cd0 2024-09-05T12:47:20.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f455b400640 1 -- 172.21.15.71:0/949089081 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f4568004e20 con 0x55f9bb583cd0 2024-09-05T12:47:20.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f457eddb740 10 client.27557 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:20.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f457eddb740 10 client.27557 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:47:20.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f457eddb740 20 client.27557 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:20.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f457eddb740 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb60c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:20.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f457eddb740 4 client.27557 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.9, "await_for": 45.0}] 2024-09-05T12:47:20.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.9, "await_for": 45.0}) -- 0x55f9bb4f6c60 con 0x55f9bb59e330 2024-09-05T12:47:20.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.312+0000 7f4578e00640 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb60c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:20.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.315+0000 7f4578e00640 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb60c9c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b0e097c80f0bd7 server_cookie=8be0ec9188153195 in_seq=0 out_seq=0 2024-09-05T12:47:20.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.315+0000 7f455b400640 10 client.27557 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.629+0000 7f93f4c00640 1 -- 172.21.15.71:0/3256347318 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5595e1840b80 con 0x5595e1971ce0 2024-09-05T12:47:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.629+0000 7f93f4c00640 10 client.17517 handle_command_reply: tid=0 2024-09-05T12:47:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.629+0000 7f9414a24740 1 client.17517 shutdown 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.629+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 msgr2=0x5595e18ef240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.629+0000 7f9414a24740 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18ef240 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0092a0 tx=0x7f93fc031400 comp rx=0 tx=0).stop 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 shutdown_connections 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x5595e1971ce0 0x5595e1974100 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5595e18e6d80 0x5595e18e1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5595e18dc4e0 0x5595e18e1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 --2- 172.21.15.71:0/3256347318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5595e18dbbc0 0x5595e18ef240 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 >> 172.21.15.71:0/3256347318 conn(0x5595e18d8130 msgr2=0x5595e18ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f93f4c00640 0 client.17517 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.630+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 shutdown_connections 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.632+0000 7f9414a24740 1 -- 172.21.15.71:0/3256347318 wait complete. 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.632+0000 7f9414a24740 20 client.17517 trim_cache size 0 max 16384 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:47:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:47:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.633+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f93f00a2df0 msgr2=0x7f93f00a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.633+0000 7f940be00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f93f00a2df0 0x7f93f00a52b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93fc0096c0 tx=0x7f93fc000bc0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.633+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 msgr2=0x7f940c19a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.633+0000 7f940be00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c19a950 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93f800d950 tx=0x7f93f800de10 comp rx=0 tx=0).stop 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 shutdown_connections 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f93f00a2df0 0x7f93f00a52b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f940c1078c0 0x7f940c19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f940c102250 0x7f940c19a950 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 --2- 172.21.15.71:0/57318656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f940c1018b0 0x7f940c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 >> 172.21.15.71:0/57318656 conn(0x7f940c0fd5d0 msgr2=0x7f940c10cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 shutdown_connections 2024-09-05T12:47:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:20.634+0000 7f940be00640 1 -- 172.21.15.71:0/57318656 wait complete. 2024-09-05T12:47:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:20 smithi071 ceph-mon[29845]: pgmap v620: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-05T12:47:20.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:20 smithi123 ceph-mon[34321]: pgmap v620: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-05T12:47:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:20 smithi174 ceph-mon[34313]: pgmap v620: 129 pgs: 129 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-05T12:47:20.850 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.71 MB/sec execute 254 sec latency 618.678 ms 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.893+0000 7f455b400640 1 -- 172.21.15.71:0/949089081 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f9bb4f6c60 con 0x55f9bb59e330 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.893+0000 7f455b400640 10 client.27557 handle_command_reply: tid=0 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.893+0000 7f457eddb740 1 client.27557 shutdown 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.893+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 msgr2=0x55f9bb5937e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.893+0000 7f457eddb740 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 0x55f9bb5937e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4568031020 tx=0x7f45680044b0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 shutdown_connections 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55f9bb59e330 0x55f9bb60c9c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9bb58e590 0x55f9bb593d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f9bb583cd0 0x55f9bb5937e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.895 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 --2- 172.21.15.71:0/949089081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9bb5833b0 0x55f9bb5932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f455b400640 0 client.27557 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 >> 172.21.15.71:0/949089081 conn(0x55f9bb57f940 msgr2=0x55f9bb58cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 shutdown_connections 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 1 -- 172.21.15.71:0/949089081 wait complete. 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.894+0000 7f457eddb740 20 client.27557 trim_cache size 0 max 16384 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:47:20.896 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 76, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 62, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "c9325530": { 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 62, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:47:20.897 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 140.9, 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "QUIESCED", 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:20.898 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f455c0a2df0 msgr2=0x7f455c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f455c0a2df0 0x7f455c0a52b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f456c0a4430 tx=0x7f456c00ba90 comp rx=0 tx=0).stop 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 msgr2=0x7f45741abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 0x7f45741abb30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f456c007bd0 tx=0x7f456c00f500 comp rx=0 tx=0).stop 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 shutdown_connections 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457410ea80 0x7f45741abb30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f455c0a2df0 0x7f455c0a52b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4574109e00 0x7f45741a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 --2- 172.21.15.71:0/3998262202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4574109450 0x7f45741a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:20.899 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 >> 172.21.15.71:0/3998262202 conn(0x7f457406e2d0 msgr2=0x7f457410c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:20.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 shutdown_connections 2024-09-05T12:47:20.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:20.895+0000 7f457c200640 1 -- 172.21.15.71:0/3998262202 wait complete. 2024-09-05T12:47:21.851 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.66 MB/sec execute 255 sec latency 1619.003 ms 2024-09-05T12:47:22.496 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c9325530', quiesce took 6.6 seconds. Will release after: 14.299999999999999 2024-09-05T12:47:22.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:22 smithi071 ceph-mon[29845]: pgmap v621: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 615 op/s 2024-09-05T12:47:22.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:22 smithi123 ceph-mon[34321]: pgmap v621: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 615 op/s 2024-09-05T12:47:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:22 smithi174 ceph-mon[34313]: pgmap v621: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.0 MiB/s wr, 615 op/s 2024-09-05T12:47:22.851 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.60 MB/sec execute 256 sec latency 2619.129 ms 2024-09-05T12:47:23.851 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.54 MB/sec execute 257 sec latency 3619.246 ms 2024-09-05T12:47:24.660 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:24 smithi071 ceph-mon[29845]: pgmap v622: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.4 MiB/s wr, 657 op/s 2024-09-05T12:47:24.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:24 smithi123 ceph-mon[34321]: pgmap v622: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.4 MiB/s wr, 657 op/s 2024-09-05T12:47:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:24 smithi174 ceph-mon[34313]: pgmap v622: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.4 MiB/s wr, 657 op/s 2024-09-05T12:47:24.851 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.49 MB/sec execute 258 sec latency 4619.339 ms 2024-09-05T12:47:25.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:25 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:47:25.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:25 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:47:25.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:25 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:47:25.851 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.43 MB/sec execute 259 sec latency 5619.398 ms 2024-09-05T12:47:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:26 smithi123 ceph-mon[34321]: pgmap v623: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.4 MiB/s wr, 657 op/s 2024-09-05T12:47:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:26 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:47:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:26 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:47:26.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:26 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:47:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:26 smithi174 ceph-mon[34313]: pgmap v623: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.4 MiB/s wr, 657 op/s 2024-09-05T12:47:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:26 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:47:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:26 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:47:26.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:26 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:47:26.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.37 MB/sec execute 260 sec latency 6619.633 ms 2024-09-05T12:47:26.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:26 smithi071 ceph-mon[29845]: pgmap v623: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.4 MiB/s wr, 657 op/s 2024-09-05T12:47:26.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:26 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:47:26.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:26 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:47:26.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:26 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:47:27.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.32 MB/sec execute 261 sec latency 7619.721 ms 2024-09-05T12:47:28.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:28 smithi123 ceph-mon[34321]: pgmap v624: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-09-05T12:47:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:28 smithi174 ceph-mon[34313]: pgmap v624: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-09-05T12:47:28.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.26 MB/sec execute 262 sec latency 8619.869 ms 2024-09-05T12:47:28.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:28 smithi071 ceph-mon[29845]: pgmap v624: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.5 MiB/s wr, 593 op/s 2024-09-05T12:47:29.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.21 MB/sec execute 263 sec latency 9620.015 ms 2024-09-05T12:47:30.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:30 smithi123 ceph-mon[34321]: pgmap v625: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.4 MiB/s wr, 330 op/s 2024-09-05T12:47:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:30 smithi174 ceph-mon[34313]: pgmap v625: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.4 MiB/s wr, 330 op/s 2024-09-05T12:47:30.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.16 MB/sec execute 264 sec latency 10620.100 ms 2024-09-05T12:47:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:30 smithi071 ceph-mon[29845]: pgmap v625: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.4 MiB/s wr, 330 op/s 2024-09-05T12:47:31.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.10 MB/sec execute 265 sec latency 11620.168 ms 2024-09-05T12:47:32.607 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:32 smithi071 ceph-mon[29845]: pgmap v626: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-05T12:47:32.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:32 smithi174 ceph-mon[34313]: pgmap v626: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-05T12:47:32.852 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.05 MB/sec execute 266 sec latency 12620.251 ms 2024-09-05T12:47:32.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:32 smithi123 ceph-mon[34321]: pgmap v626: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-05T12:47:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.761+0000 7f7f87000640 1 -- 172.21.15.71:0/79926284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 msgr2=0x7f7f8010b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.761+0000 7f7f87000640 1 --2- 172.21.15.71:0/79926284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f8010b8e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009900 tx=0x7f7f6c02a4a0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.761+0000 7f7f87000640 1 -- 172.21.15.71:0/79926284 shutdown_connections 2024-09-05T12:47:33.764 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 --2- 172.21.15.71:0/79926284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f8010bf00 0x7f7f8010e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 --2- 172.21.15.71:0/79926284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f8010b8e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 --2- 172.21.15.71:0/79926284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f80102ae0 0x7f7f80102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 -- 172.21.15.71:0/79926284 >> 172.21.15.71:0/79926284 conn(0x7f7f800777c0 msgr2=0x7f7f800feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 -- 172.21.15.71:0/79926284 shutdown_connections 2024-09-05T12:47:33.765 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 -- 172.21.15.71:0/79926284 wait complete. 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.762+0000 7f7f87000640 1 Processor -- start 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 -- start start 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f80102ae0 0x7f7f80100730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f80100c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8010bf00 0x7f7f801011b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80111da0 con 0x7f7f80103400 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80111c20 con 0x7f7f8010bf00 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f87000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f80111f20 con 0x7f7f80102ae0 2024-09-05T12:47:33.767 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f85200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f80100c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f85200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f80100c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60032/0 (socket says 172.21.15.71:60032) 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f85200640 1 -- 172.21.15.71:0/85984820 learned_addr learned my addr 172.21.15.71:0/85984820 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.763+0000 7f7f85c00640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f80102ae0 0x7f7f80100730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f85200640 1 -- 172.21.15.71:0/85984820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f80102ae0 msgr2=0x7f7f80100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f85200640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f80102ae0 0x7f7f80100730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f85200640 1 -- 172.21.15.71:0/85984820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8010bf00 msgr2=0x7f7f801011b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f85200640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8010bf00 0x7f7f801011b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f85200640 1 -- 172.21.15.71:0/85984820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f801a67d0 con 0x7f7f80103400 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f85200640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f80100c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c004770 tx=0x7f7f6c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:33.768 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f7e000640 1 -- 172.21.15.71:0/85984820 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c03f070 con 0x7f7f80103400 2024-09-05T12:47:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f7e000640 1 -- 172.21.15.71:0/85984820 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f6c005d10 con 0x7f7f80103400 2024-09-05T12:47:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f801a6a60 con 0x7f7f80103400 2024-09-05T12:47:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f7e000640 1 -- 172.21.15.71:0/85984820 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c035030 con 0x7f7f80103400 2024-09-05T12:47:33.769 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.764+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f801a6f50 con 0x7f7f80103400 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.767+0000 7f7f8b646740 1 Processor -- start 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.767+0000 7f7f7e000640 1 -- 172.21.15.71:0/85984820 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7f6c042070 con 0x7f7f80103400 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.767+0000 7f7f8b646740 1 -- start start 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c2a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2acd0 0x563502c35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c37960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.773 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f8b646740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563502b90100 con 0x563502c2a3b0 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f8b646740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563502b381b0 con 0x563502c35570 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f8b646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563502b8c0c0 con 0x563502c2acd0 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c2a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c2a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60044/0 (socket says 172.21.15.71:60044) 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 -- 172.21.15.71:0/231898273 learned_addr learned my addr 172.21.15.71:0/231898273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f85c00640 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2acd0 0x563502c35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f85200640 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c37960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 -- 172.21.15.71:0/231898273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2acd0 msgr2=0x563502c35030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2acd0 0x563502c35030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 -- 172.21.15.71:0/231898273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 msgr2=0x563502c37960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c37960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f86600640 1 -- 172.21.15.71:0/231898273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635028a3210 con 0x563502c2a3b0 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.768+0000 7f7f85c00640 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2acd0 0x563502c35030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f86600640 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c2a790 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009e30 tx=0x7f7f7800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7fe1f8a19cdfa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f7e000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7f680a2cf0 0x7f7f680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f72a00640 1 -- 172.21.15.71:0/231898273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78004d20 con 0x563502c2a3b0 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f72a00640 1 -- 172.21.15.71:0/231898273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7800ee70 con 0x563502c2a3b0 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f72a00640 1 -- 172.21.15.71:0/231898273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f78010830 con 0x563502c2a3b0 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f7e000640 1 -- 172.21.15.71:0/85984820 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f7f6c0eff30 con 0x7f7f80103400 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.769+0000 7f7f85c00640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7f680a2cf0 0x7f7f680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 -- 172.21.15.71:0/231898273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 msgr2=0x563502c2a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c2a790 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7f78009e30 tx=0x7f7f7800e720 comp rx=0 tx=0).stop 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f85c00640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7f680a2cf0 0x7f7f680a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006fd0 tx=0x7f7f74008040 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 -- 172.21.15.71:0/231898273 shutdown_connections 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c37960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2acd0 0x563502c35030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 --2- 172.21.15.71:0/231898273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c2a790 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 -- 172.21.15.71:0/231898273 >> 172.21.15.71:0/231898273 conn(0x563502c26920 msgr2=0x563502c26d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 -- 172.21.15.71:0/231898273 shutdown_connections 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 -- 172.21.15.71:0/231898273 wait complete. 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.770+0000 7f7f8b646740 1 -- start start 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 Processor -- start 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c32590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c32ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c33010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563502c37ff0 con 0x563502c2a3b0 2024-09-05T12:47:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563502b19090 con 0x563502c2acd0 2024-09-05T12:47:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563502c38170 con 0x563502c35570 2024-09-05T12:47:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f8b646740 10 client.0 _ll_register_callbacks cb 0x563502b6df30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c32590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c32590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60058/0 (socket says 172.21.15.71:60058) 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.771+0000 7f7f85c00640 1 -- 172.21.15.71:0/2016896585 learned_addr learned my addr 172.21.15.71:0/2016896585 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f86600640 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c33010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f85c00640 1 -- 172.21.15.71:0/2016896585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 msgr2=0x563502c33010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f85c00640 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c33010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f85c00640 1 -- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 msgr2=0x563502c32ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f85c00640 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c32ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f85c00640 1 -- 172.21.15.71:0/2016896585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350296f620 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f85c00640 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c32590 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7f740156c0 tx=0x7f7f740046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851a741e83138064 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f67400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.772+0000 7f7f67400640 1 -- 172.21.15.71:0/2016896585 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74025020 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f67400640 1 -- 172.21.15.71:0/2016896585 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f7f74021440 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7800ee70 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f67400640 1 -- 172.21.15.71:0/2016896585 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7f74030050 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f8b646740 10 client.17538 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563502c43e10 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f67400640 1 -- 172.21.15.71:0/2016896585 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7800ee70 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f67400640 1 -- 172.21.15.71:0/2016896585 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f7f74014830 con 0x563502c2a3b0 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f8b646740 10 client.17538 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f8b646740 10 client.17538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.773+0000 7f7f8b646740 20 client.17538 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.774+0000 7f7f8b646740 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502c452d0 0x563502c47720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.774+0000 7f7f8b646740 4 client.17538 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.774+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563502b9d830 con 0x563502c452d0 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.774+0000 7f7f85200640 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502c452d0 0x563502c47720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.774+0000 7f7f85200640 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502c452d0 0x563502c47720 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=881d7e70f03cb6d4 server_cookie=b0daac358557ea90 in_seq=0 out_seq=0 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.774+0000 7f7f67400640 10 client.17538 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.775+0000 7f7f67400640 1 -- 172.21.15.71:0/2016896585 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563502b9d830 con 0x563502c452d0 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.775+0000 7f7f67400640 10 client.17538 handle_command_reply: tid=0 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.775+0000 7f7f8b646740 1 client.17538 shutdown 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.775+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 msgr2=0x563502c32590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.775+0000 7f7f8b646740 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c32590 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7f740156c0 tx=0x7f7f740046e0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 shutdown_connections 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502c452d0 0x563502c47720 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.782 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c33010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c32ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f67400640 0 client.17538 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 --2- 172.21.15.71:0/2016896585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c32590 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 >> 172.21.15.71:0/2016896585 conn(0x563502c26920 msgr2=0x563502c2c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 shutdown_connections 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 1 -- 172.21.15.71:0/2016896585 wait complete. 2024-09-05T12:47:33.783 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.776+0000 7f7f8b646740 20 client.17538 trim_cache size 0 max 16384 2024-09-05T12:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.803+0000 7f7f8b646740 1 Processor -- start 2024-09-05T12:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 -- start start 2024-09-05T12:47:33.806 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 0x563502c4d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2acd0 0x563502c4d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801b0f60 con 0x563502c2acd0 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801b2aa0 con 0x563502c35570 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f8b646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801b1630 con 0x563502c2a3b0 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f85c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 0x563502c4d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f85200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2acd0 0x563502c4d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f85c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 0x563502c4d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:54304/0 (socket says 172.21.15.71:54304) 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f85c00640 1 -- 172.21.15.71:0/1403917359 learned_addr learned my addr 172.21.15.71:0/1403917359 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f85c00640 1 -- 172.21.15.71:0/1403917359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 msgr2=0x563502c2be10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.804+0000 7f7f85c00640 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c2be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f85c00640 1 -- 172.21.15.71:0/1403917359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2acd0 msgr2=0x563502c4d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:33.807 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f85c00640 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2acd0 0x563502c4d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.808 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f85c00640 1 -- 172.21.15.71:0/1403917359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563502c41300 con 0x563502c2a3b0 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f85c00640 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 0x563502c4d080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f74004790 tx=0x7f7f740117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118098b55d515892 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f72000640 1 -- 172.21.15.71:0/1403917359 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74038070 con 0x563502c2a3b0 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f72000640 1 -- 172.21.15.71:0/1403917359 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7401a030 con 0x563502c2a3b0 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.805+0000 7f7f72000640 1 -- 172.21.15.71:0/1403917359 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74025040 con 0x563502c2a3b0 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 -- 172.21.15.71:0/1403917359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 msgr2=0x563502c4d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 0x563502c4d080 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7f74004790 tx=0x7f7f740117b0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.810 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 -- 172.21.15.71:0/1403917359 shutdown_connections 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c35570 0x563502c2be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2acd0 0x563502c4d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 --2- 172.21.15.71:0/1403917359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c2a3b0 0x563502c4d080 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 -- 172.21.15.71:0/1403917359 >> 172.21.15.71:0/1403917359 conn(0x563502c26920 msgr2=0x563502c30e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 -- 172.21.15.71:0/1403917359 shutdown_connections 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.806+0000 7f7f8b646740 1 -- 172.21.15.71:0/1403917359 wait complete. 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 -- start start 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 Processor -- start 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c3d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c39120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801b23d0 con 0x563502c2a3b0 2024-09-05T12:47:33.811 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801b1d00 con 0x563502c2acd0 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f801aef60 con 0x563502c35570 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.807+0000 7f7f8b646740 10 client.0 _ll_register_callbacks cb 0x7f7f740093b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c3d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c3d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60072/0 (socket says 172.21.15.71:60072) 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 -- 172.21.15.71:0/3493504524 learned_addr learned my addr 172.21.15.71:0/3493504524 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85200640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f86600640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c39120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 -- 172.21.15.71:0/3493504524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 msgr2=0x563502c39120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c39120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 -- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 msgr2=0x563502c38be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c38be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 -- 172.21.15.71:0/3493504524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563502c39660 con 0x563502c2a3b0 2024-09-05T12:47:33.812 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f85c00640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c3d8c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7f74014810 tx=0x7f7f740099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9b5b81f8bfc077 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:33.813 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.808+0000 7f7f67400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f67400640 1 -- 172.21.15.71:0/3493504524 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74040020 con 0x563502c2a3b0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f67400640 1 -- 172.21.15.71:0/3493504524 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f7f74004020 con 0x563502c2a3b0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f7401a030 con 0x563502c2a3b0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f67400640 1 -- 172.21.15.71:0/3493504524 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f7f7404b050 con 0x563502c2a3b0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f8b646740 10 client.17544 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x563502cafd50 con 0x563502c2a3b0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f67400640 1 -- 172.21.15.71:0/3493504524 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f7401a030 con 0x563502c2a3b0 2024-09-05T12:47:33.814 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.809+0000 7f7f67400640 1 -- 172.21.15.71:0/3493504524 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f7f74003cd0 con 0x563502c2a3b0 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f8b646740 10 client.17544 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f8b646740 10 client.17544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f8b646740 20 client.17544 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f8b646740 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502cb03b0 0x563502cb27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f8b646740 4 client.17544 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563502b97b80 con 0x563502cb03b0 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f85200640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502cb03b0 0x563502cb27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f85200640 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502cb03b0 0x563502cb27a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734e92e809e4ed95 server_cookie=c7cee7572233c197 in_seq=0 out_seq=0 2024-09-05T12:47:33.815 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.810+0000 7f7f67400640 10 client.17544 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:33.853 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 14.00 MB/sec execute 267 sec latency 13621.281 ms 2024-09-05T12:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.876+0000 7f7f67400640 1 -- 172.21.15.71:0/3493504524 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563502b97b80 con 0x563502cb03b0 2024-09-05T12:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.876+0000 7f7f67400640 10 client.17544 handle_command_reply: tid=0 2024-09-05T12:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.876+0000 7f7f8b646740 1 client.17544 shutdown 2024-09-05T12:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 msgr2=0x563502c3d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c3d8c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7f74014810 tx=0x7f7f740099e0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 shutdown_connections 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x563502cb03b0 0x563502cb27a0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563502c35570 0x563502c39120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563502c2acd0 0x563502c38be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 --2- 172.21.15.71:0/3493504524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563502c2a3b0 0x563502c3d8c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 >> 172.21.15.71:0/3493504524 conn(0x563502c26920 msgr2=0x563502c40b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f67400640 0 client.17544 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 shutdown_connections 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 1 -- 172.21.15.71:0/3493504524 wait complete. 2024-09-05T12:47:33.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.877+0000 7f7f8b646740 20 client.17544 trim_cache size 0 max 16384 2024-09-05T12:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:47:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7f680a2cf0 msgr2=0x7f7f680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7f680a2cf0 0x7f7f680a51b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7f74006fd0 tx=0x7f7f74008040 comp rx=0 tx=0).stop 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 msgr2=0x7f7f80100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f80100c70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c004770 tx=0x7f7f6c003c10 comp rx=0 tx=0).stop 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 shutdown_connections 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f8010bf00 0x7f7f801011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f7f680a2cf0 0x7f7f680a51b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80103400 0x7f7f80100c70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 --2- 172.21.15.71:0/85984820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f80102ae0 0x7f7f80100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 >> 172.21.15.71:0/85984820 conn(0x7f7f800777c0 msgr2=0x7f7f800fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 shutdown_connections 2024-09-05T12:47:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:33.878+0000 7f7f87000640 1 -- 172.21.15.71:0/85984820 wait complete. 2024-09-05T12:47:34.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:34 smithi174 ceph-mon[34313]: pgmap v627: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-05T12:47:34.853 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.94 MB/sec execute 268 sec latency 14621.360 ms 2024-09-05T12:47:34.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:34 smithi071 ceph-mon[29845]: pgmap v627: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-05T12:47:34.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:34 smithi123 ceph-mon[34321]: pgmap v627: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 141 op/s 2024-09-05T12:47:35.853 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.89 MB/sec execute 269 sec latency 15621.432 ms 2024-09-05T12:47:36.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:36 smithi174 ceph-mon[34313]: pgmap v628: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 558 KiB/s wr, 40 op/s 2024-09-05T12:47:36.797 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:47:36.845 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:36 smithi071 ceph-mon[29845]: pgmap v628: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 558 KiB/s wr, 40 op/s 2024-09-05T12:47:36.853 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.84 MB/sec execute 270 sec latency 16621.473 ms 2024-09-05T12:47:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:36 smithi123 ceph-mon[34321]: pgmap v628: 129 pgs: 129 active+clean; 1.0 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 558 KiB/s wr, 40 op/s 2024-09-05T12:47:37.448 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:47:37.853 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.79 MB/sec execute 271 sec latency 17621.529 ms 2024-09-05T12:47:38.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.74 MB/sec execute 272 sec latency 18621.707 ms 2024-09-05T12:47:39.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 -- 172.21.15.71:0/1165157352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390074510 msgr2=0x7fe3900748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:39.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 --2- 172.21.15.71:0/1165157352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390074510 0x7fe3900748f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe380009900 tx=0x7fe38002a480 comp rx=0 tx=0).stop 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 -- 172.21.15.71:0/1165157352 shutdown_connections 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 --2- 172.21.15.71:0/1165157352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe39010e280 0x7fe390110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 --2- 172.21.15.71:0/1165157352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 --2- 172.21.15.71:0/1165157352 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe390074510 0x7fe3900748f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 -- 172.21.15.71:0/1165157352 >> 172.21.15.71:0/1165157352 conn(0x7fe3900fd740 msgr2=0x7fe3900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 -- 172.21.15.71:0/1165157352 shutdown_connections 2024-09-05T12:47:39.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.746+0000 7fe394a00640 1 -- 172.21.15.71:0/1165157352 wait complete. 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 Processor -- start 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- start start 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe390074510 0x7fe39019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe39010e280 0x7fe39019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe390114060 con 0x7fe39010e280 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe390113ee0 con 0x7fe390074e30 2024-09-05T12:47:39.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901141e0 con 0x7fe390074510 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe39010e280 0x7fe39019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53568/0 (socket says 172.21.15.71:53568) 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe39010e280 0x7fe39019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:60094/0 (socket says 172.21.15.71:60094) 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 -- 172.21.15.71:0/2098556905 learned_addr learned my addr 172.21.15.71:0/2098556905 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38e800640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe390074510 0x7fe39019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 -- 172.21.15.71:0/2098556905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe390074510 msgr2=0x7fe39019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe390074510 0x7fe39019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 -- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe39010e280 msgr2=0x7fe39019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe39010e280 0x7fe39019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 -- 172.21.15.71:0/2098556905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39019f4f0 con 0x7fe390074e30 2024-09-05T12:47:39.752 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe38de00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39019aa80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00b4a0 tx=0x7fe37c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c014070 con 0x7fe390074e30 2024-09-05T12:47:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe37c010af0 con 0x7fe390074e30 2024-09-05T12:47:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39019f7e0 con 0x7fe390074e30 2024-09-05T12:47:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c017d10 con 0x7fe390074e30 2024-09-05T12:47:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3901a70c0 con 0x7fe390074e30 2024-09-05T12:47:39.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.747+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3900748f0 con 0x7fe390074e30 2024-09-05T12:47:39.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.753+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe37c017070 con 0x7fe390074e30 2024-09-05T12:47:39.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.753+0000 7fe387400640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3740a2cf0 0x7fe3740a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:39.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.753+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7fe37c0cea20 con 0x7fe390074e30 2024-09-05T12:47:39.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.753+0000 7fe38e800640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3740a2cf0 0x7fe3740a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:39.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.753+0000 7fe38e800640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3740a2cf0 0x7fe3740a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe380000c00 tx=0x7fe380005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:39.758 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.753+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe37c09ad30 con 0x7fe390074e30 2024-09-05T12:47:39.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.69 MB/sec execute 273 sec latency 19621.753 ms 2024-09-05T12:47:39.976 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.974+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe39019b8a0 con 0x7fe390074e30 2024-09-05T12:47:39.977 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:39 smithi071 ceph-mon[29845]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.2 MiB/s wr, 121 op/s 2024-09-05T12:47:39.977 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.975+0000 7fe387400640 1 -- 172.21.15.71:0/2098556905 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7fe39019b8a0 con 0x7fe390074e30 2024-09-05T12:47:39.978 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:47:39.978 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:47:39.978 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:47:39.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3740a2cf0 msgr2=0x7fe3740a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:39.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3740a2cf0 0x7fe3740a51b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe380000c00 tx=0x7fe380005b50 comp rx=0 tx=0).stop 2024-09-05T12:47:39.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 msgr2=0x7fe39019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:39.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39019aa80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe37c00b4a0 tx=0x7fe37c00b960 comp rx=0 tx=0).stop 2024-09-05T12:47:39.980 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 shutdown_connections 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3740a2cf0 0x7fe3740a51b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe39010e280 0x7fe39019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe390074e30 0x7fe39019aa80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 --2- 172.21.15.71:0/2098556905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe390074510 0x7fe39019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 >> 172.21.15.71:0/2098556905 conn(0x7fe3900fd740 msgr2=0x7fe3900fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 shutdown_connections 2024-09-05T12:47:39.981 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:39.978+0000 7fe394a00640 1 -- 172.21.15.71:0/2098556905 wait complete. 2024-09-05T12:47:40.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:39 smithi123 ceph-mon[34321]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.2 MiB/s wr, 121 op/s 2024-09-05T12:47:40.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:39 smithi174 ceph-mon[34313]: pgmap v629: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 3.2 MiB/s wr, 121 op/s 2024-09-05T12:47:40.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:40 smithi071 ceph-mon[29845]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 3.5 MiB/s wr, 122 op/s 2024-09-05T12:47:40.839 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:40 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2098556905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:47:40.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.64 MB/sec execute 274 sec latency 20621.807 ms 2024-09-05T12:47:41.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:40 smithi123 ceph-mon[34321]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 3.5 MiB/s wr, 122 op/s 2024-09-05T12:47:41.180 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:40 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2098556905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:47:41.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:40 smithi174 ceph-mon[34313]: pgmap v630: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 3.5 MiB/s wr, 122 op/s 2024-09-05T12:47:41.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:40 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2098556905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:47:41.624 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id c9325530 --release --await' 2024-09-05T12:47:41.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --set-id c9325530 --release --await 2024-09-05T12:47:41.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.59 MB/sec execute 275 sec latency 21621.865 ms 2024-09-05T12:47:42.006 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 -- 172.21.15.71:0/2080763009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 msgr2=0x7f342c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 --2- 172.21.15.71:0/2080763009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 0x7f342c101e20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3418009900 tx=0x7f341802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 -- 172.21.15.71:0/2080763009 shutdown_connections 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 --2- 172.21.15.71:0/2080763009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 --2- 172.21.15.71:0/2080763009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 0x7f342c101e20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 --2- 172.21.15.71:0/2080763009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f342c1079c0 0x7f342c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 -- 172.21.15.71:0/2080763009 >> 172.21.15.71:0/2080763009 conn(0x7f342c0fd740 msgr2=0x7f342c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 -- 172.21.15.71:0/2080763009 shutdown_connections 2024-09-05T12:47:42.840 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.837+0000 7f3432a00640 1 -- 172.21.15.71:0/2080763009 wait complete. 2024-09-05T12:47:42.841 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.838+0000 7f3432a00640 1 Processor -- start 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.838+0000 7f3432a00640 1 -- start start 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 0x7f342c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f342c1079c0 0x7f342c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f342c111e10 con 0x7f342c102360 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f342c111c90 con 0x7f342c1019c0 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f342c111f90 con 0x7f342c1079c0 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3430c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3430c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43732/0 (socket says 172.21.15.71:43732) 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3430c00640 1 -- 172.21.15.71:0/533600978 learned_addr learned my addr 172.21.15.71:0/533600978 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:42.842 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.839+0000 7f3432000640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f342c1079c0 0x7f342c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f3430c00640 1 -- 172.21.15.71:0/533600978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f342c1079c0 msgr2=0x7f342c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f3430c00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f342c1079c0 0x7f342c19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f3430c00640 1 -- 172.21.15.71:0/533600978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 msgr2=0x7f342c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f3430c00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 0x7f342c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f3430c00640 1 -- 172.21.15.71:0/533600978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c19f440 con 0x7f342c102360 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f3430c00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c19a9d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f341802a9e0 tx=0x7f3418003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:42.843 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.840+0000 7f342a000640 1 -- 172.21.15.71:0/533600978 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341803f070 con 0x7f342c102360 2024-09-05T12:47:42.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.841+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c19f6d0 con 0x7f342c102360 2024-09-05T12:47:42.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.841+0000 7f342a000640 1 -- 172.21.15.71:0/533600978 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3418031c30 con 0x7f342c102360 2024-09-05T12:47:42.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.841+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1a7010 con 0x7f342c102360 2024-09-05T12:47:42.844 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.841+0000 7f342a000640 1 -- 172.21.15.71:0/533600978 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418049060 con 0x7f342c102360 2024-09-05T12:47:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f3436aa2740 1 Processor -- start 2024-09-05T12:47:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f342a000640 1 -- 172.21.15.71:0/533600978 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f3418042070 con 0x7f342c102360 2024-09-05T12:47:42.846 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f3436aa2740 1 -- start start 2024-09-05T12:47:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 0x56268fd82900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82e40 0x56268fd8d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd8d700 0x56268fd8faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.844+0000 7f3436aa2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56268fce8650 con 0x56268fd82520 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3436aa2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56268fc8ff10 con 0x56268fd82e40 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3436aa2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56268fce4610 con 0x56268fd8d700 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 0x56268fd82900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 0x56268fd82900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43734/0 (socket says 172.21.15.71:43734) 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 -- 172.21.15.71:0/2388778886 learned_addr learned my addr 172.21.15.71:0/2388778886 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3430c00640 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd8d700 0x56268fd8faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 -- 172.21.15.71:0/2388778886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd8d700 msgr2=0x56268fd8faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd8d700 0x56268fd8faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 -- 172.21.15.71:0/2388778886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82e40 msgr2=0x56268fd8d1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82e40 0x56268fd8d1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.845+0000 7f3432000640 1 -- 172.21.15.71:0/2388778886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268f9f7580 con 0x56268fd82520 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f3432000640 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 0x56268fd82900 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3424011440 tx=0x7f3424011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a538c799d08ff589 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f342a000640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34140a2cf0 0x7f34140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f342a000640 1 -- 172.21.15.71:0/533600978 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f34180f0330 con 0x7f342c102360 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f3431600640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34140a2cf0 0x7f34140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f341e600640 1 -- 172.21.15.71:0/2388778886 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424003c30 con 0x56268fd82520 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f341e600640 1 -- 172.21.15.71:0/2388778886 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3424003dd0 con 0x56268fd82520 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.846+0000 7f341e600640 1 -- 172.21.15.71:0/2388778886 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400fb50 con 0x56268fd82520 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 -- 172.21.15.71:0/2388778886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 msgr2=0x56268fd82900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3431600640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34140a2cf0 0x7f34140a51b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3420005fd0 tx=0x7f3420005f40 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 0x56268fd82900 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3424011440 tx=0x7f3424011900 comp rx=0 tx=0).stop 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 -- 172.21.15.71:0/2388778886 shutdown_connections 2024-09-05T12:47:42.850 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd8d700 0x56268fd8faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82e40 0x56268fd8d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 --2- 172.21.15.71:0/2388778886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82520 0x56268fd82900 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.851 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.847+0000 7f3436aa2740 1 -- 172.21.15.71:0/2388778886 >> 172.21.15.71:0/2388778886 conn(0x56268fd7eab0 msgr2=0x56268fd7ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.848+0000 7f3436aa2740 1 -- 172.21.15.71:0/2388778886 shutdown_connections 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.848+0000 7f3436aa2740 1 -- 172.21.15.71:0/2388778886 wait complete. 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.848+0000 7f3436aa2740 1 -- start start 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.848+0000 7f3436aa2740 1 Processor -- start 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd8a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd8ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd8b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.853 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56268fd902d0 con 0x56268fd82e40 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56268fca3350 con 0x56268fd8d700 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56268fd90450 con 0x56268fd82520 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3436aa2740 10 client.0 _ll_register_callbacks cb 0x56268fcc6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3430c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd8ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3430c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd8ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43750/0 (socket says 172.21.15.71:43750) 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3430c00640 1 -- 172.21.15.71:0/3058011974 learned_addr learned my addr 172.21.15.71:0/3058011974 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3431600640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd8a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.849+0000 7f3432000640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd8b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3430c00640 1 -- 172.21.15.71:0/3058011974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 msgr2=0x56268fd8a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3430c00640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd8a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3430c00640 1 -- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 msgr2=0x56268fd8b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:42.854 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3430c00640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd8b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3430c00640 1 -- 172.21.15.71:0/3058011974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268fac1960 con 0x56268fd82e40 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3432000640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd8b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3430c00640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd8ac20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3418041000 tx=0x7f34180b87e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5640e1db2255b960 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3412a00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3412a00640 1 -- 172.21.15.71:0/3058011974 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3418031de0 con 0x56268fd82e40 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3424003dd0 con 0x56268fd82e40 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.850+0000 7f3412a00640 1 -- 172.21.15.71:0/3058011974 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f34180f00d0 con 0x56268fd82e40 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3412a00640 1 -- 172.21.15.71:0/3058011974 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f34180bcb80 con 0x56268fd82e40 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3436aa2740 10 client.17562 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:42.855 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56268fd9bfd0 con 0x56268fd82e40 2024-09-05T12:47:42.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3412a00640 1 -- 172.21.15.71:0/3058011974 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3424003dd0 con 0x56268fd82e40 2024-09-05T12:47:42.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3412a00640 1 -- 172.21.15.71:0/3058011974 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f34180ed930 con 0x56268fd82e40 2024-09-05T12:47:42.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3436aa2740 10 client.17562 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:42.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3436aa2740 10 client.17562 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:47:42.856 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3436aa2740 20 client.17562 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:42.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.851+0000 7f3436aa2740 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fd9d390 0x56268fd9f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.852+0000 7f3436aa2740 4 client.17562 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:47:42.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.852+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56268fcf5bc0 con 0x56268fd9d390 2024-09-05T12:47:42.857 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110368 13.54 MB/sec execute 276 sec latency 22621.910 ms 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.852+0000 7f3431600640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fd9d390 0x56268fd9f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.853+0000 7f3431600640 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fd9d390 0x56268fd9f7b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=313277450b9f7eb5 server_cookie=183c74adf12fe3d in_seq=0 out_seq=0 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.853+0000 7f3412a00640 10 client.17562 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.853+0000 7f3412a00640 1 -- 172.21.15.71:0/3058011974 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56268fcf5bc0 con 0x56268fd9d390 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.853+0000 7f3412a00640 10 client.17562 handle_command_reply: tid=0 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.853+0000 7f3436aa2740 1 client.17562 shutdown 2024-09-05T12:47:42.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 msgr2=0x56268fd8ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd8ac20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3418041000 tx=0x7f34180b87e0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 shutdown_connections 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fd9d390 0x56268fd9f7b0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd8b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd8ac20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 --2- 172.21.15.71:0/3058011974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd8a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 >> 172.21.15.71:0/3058011974 conn(0x56268fd7eab0 msgr2=0x56268fd7dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3412a00640 0 client.17562 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:42.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 shutdown_connections 2024-09-05T12:47:42.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 1 -- 172.21.15.71:0/3058011974 wait complete. 2024-09-05T12:47:42.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.854+0000 7f3436aa2740 20 client.17562 trim_cache size 0 max 16384 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.871+0000 7f3436aa2740 1 Processor -- start 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 -- start start 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 0x56268fda5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82e40 0x56268fda55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd8d700 0x56268fd87940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1b0ef0 con 0x56268fd8d700 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1b2a30 con 0x56268fd82520 2024-09-05T12:47:42.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3436aa2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1b15c0 con 0x56268fd82e40 2024-09-05T12:47:42.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3432000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd8d700 0x56268fd87940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3431600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 0x56268fda5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3431600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 0x56268fda5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53570/0 (socket says 172.21.15.71:53570) 2024-09-05T12:47:42.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.872+0000 7f3431600640 1 -- 172.21.15.71:0/1291286906 learned_addr learned my addr 172.21.15.71:0/1291286906 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f3431600640 1 -- 172.21.15.71:0/1291286906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82e40 msgr2=0x56268fda55c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f3431600640 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82e40 0x56268fda55c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f3431600640 1 -- 172.21.15.71:0/1291286906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd8d700 msgr2=0x56268fd87940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f3431600640 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd8d700 0x56268fd87940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f3431600640 1 -- 172.21.15.71:0/1291286906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268fd82c70 con 0x56268fd82520 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f3431600640 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 0x56268fda5080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f342000b720 tx=0x7f3420010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67687738d9361225 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f341dc00640 1 -- 172.21.15.71:0/1291286906 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420003b40 con 0x56268fd82520 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f341dc00640 1 -- 172.21.15.71:0/1291286906 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420019040 con 0x56268fd82520 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.873+0000 7f341dc00640 1 -- 172.21.15.71:0/1291286906 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34200106c0 con 0x56268fd82520 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 -- 172.21.15.71:0/1291286906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 msgr2=0x56268fda5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 0x56268fda5080 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f342000b720 tx=0x7f3420010000 comp rx=0 tx=0).stop 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 -- 172.21.15.71:0/1291286906 shutdown_connections 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd8d700 0x56268fd87940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82e40 0x56268fda55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 --2- 172.21.15.71:0/1291286906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd82520 0x56268fda5080 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 -- 172.21.15.71:0/1291286906 >> 172.21.15.71:0/1291286906 conn(0x56268fd7eab0 msgr2=0x56268fd7dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 -- 172.21.15.71:0/1291286906 shutdown_connections 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 -- 172.21.15.71:0/1291286906 wait complete. 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.874+0000 7f3436aa2740 1 -- start start 2024-09-05T12:47:42.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 Processor -- start 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd95c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd91f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1b2360 con 0x56268fd82e40 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1b1c90 con 0x56268fd8d700 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f342c1aeef0 con 0x56268fd82520 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3436aa2740 10 client.0 _ll_register_callbacks cb 0x7f3418009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3430c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd91f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3432000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd924a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53576/0 (socket says 172.21.15.71:53576) 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.875+0000 7f3430c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd91f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43774/0 (socket says 172.21.15.71:43774) 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 -- 172.21.15.71:0/162957670 learned_addr learned my addr 172.21.15.71:0/162957670 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:42.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3431600640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd95c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 -- 172.21.15.71:0/162957670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 msgr2=0x56268fd95c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:42.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd95c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 -- 172.21.15.71:0/162957670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 msgr2=0x56268fd91f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:42.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd91f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 -- 172.21.15.71:0/162957670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268fd92ae0 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3430c00640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd91f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3432000640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd924a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f342401a000 tx=0x7f3424013fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e3eb59f951329fe server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.876+0000 7f3412a00640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3412a00640 1 -- 172.21.15.71:0/162957670 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342400fa10 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3420019040 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3412a00640 1 -- 172.21.15.71:0/162957670 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f342400fbb0 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3412a00640 1 -- 172.21.15.71:0/162957670 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f3424035050 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3436aa2740 10 client.27578 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x56268fe08f20 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3412a00640 1 -- 172.21.15.71:0/162957670 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3420019040 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3412a00640 1 -- 172.21.15.71:0/162957670 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f342402f4c0 con 0x56268fd8d700 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3436aa2740 10 client.27578 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:42.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.877+0000 7f3436aa2740 10 client.27578 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:47:42.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.878+0000 7f3436aa2740 20 client.27578 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:42.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.878+0000 7f3436aa2740 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fe09460 0x56268fe0b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:42.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.878+0000 7f3436aa2740 4 client.27578 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c9325530", "release": true, "await": true}] 2024-09-05T12:47:42.883 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.878+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c9325530", "release": true, "await": true}) -- 0x56268fcf00d0 con 0x56268fe09460 2024-09-05T12:47:42.900 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.898+0000 7f3431600640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fe09460 0x56268fe0b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.957+0000 7f3431600640 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fe09460 0x56268fe0b880 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=952bc7cd85e84e75 server_cookie=6a5c547a7decd041 in_seq=0 out_seq=0 2024-09-05T12:47:42.959 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.957+0000 7f3412a00640 10 client.27578 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3412a00640 1 -- 172.21.15.71:0/162957670 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56268fcf00d0 con 0x56268fe09460 2024-09-05T12:47:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3412a00640 10 client.27578 handle_command_reply: tid=0 2024-09-05T12:47:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 client.27578 shutdown 2024-09-05T12:47:42.986 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 msgr2=0x56268fd924a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd924a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f342401a000 tx=0x7f3424013fd0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 shutdown_connections 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x56268fe09460 0x56268fe0b880 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56268fd8d700 0x56268fd924a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56268fd82e40 0x56268fd91f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 --2- 172.21.15.71:0/162957670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56268fd82520 0x56268fd95c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 >> 172.21.15.71:0/162957670 conn(0x56268fd7eab0 msgr2=0x56268fd7dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3412a00640 0 client.27578 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:42.987 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.984+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 shutdown_connections 2024-09-05T12:47:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.985+0000 7f3436aa2740 1 -- 172.21.15.71:0/162957670 wait complete. 2024-09-05T12:47:42.988 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.985+0000 7f3436aa2740 20 client.27578 trim_cache size 0 max 16384 2024-09-05T12:47:42.988 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:47:42.988 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 76, 2024-09-05T12:47:42.988 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:47:42.988 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 64, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": { 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "c9325530": { 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "version": 64, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "age_ref": 0.0, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "timeout": 90.0, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "expiration": 140.9, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "members": { 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "file:/": { 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "excluded": false, 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "state": { 2024-09-05T12:47:42.989 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RELEASED", 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout: "age": 0.0 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:47:42.990 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34140a2cf0 msgr2=0x7f34140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34140a2cf0 0x7f34140a51b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3420005fd0 tx=0x7f3420005f40 comp rx=0 tx=0).stop 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 msgr2=0x7f342c19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c19a9d0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f341802a9e0 tx=0x7f3418003c10 comp rx=0 tx=0).stop 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 shutdown_connections 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f342c1079c0 0x7f342c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34140a2cf0 0x7f34140a51b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f342c102360 0x7f342c19a9d0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 --2- 172.21.15.71:0/533600978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f342c1019c0 0x7f342c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 >> 172.21.15.71:0/533600978 conn(0x7f342c0fd740 msgr2=0x7f342c0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 shutdown_connections 2024-09-05T12:47:42.991 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:47:42.986+0000 7f3432a00640 1 -- 172.21.15.71:0/533600978 wait complete. 2024-09-05T12:47:43.066 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:42 smithi071 ceph-mon[29845]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 6.7 MiB/s wr, 192 op/s 2024-09-05T12:47:43.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:42 smithi123 ceph-mon[34321]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 6.7 MiB/s wr, 192 op/s 2024-09-05T12:47:43.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:42 smithi174 ceph-mon[34313]: pgmap v631: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 6.7 MiB/s wr, 192 op/s 2024-09-05T12:47:43.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110369 13.49 MB/sec execute 277 sec latency 23621.957 ms 2024-09-05T12:47:44.033 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:43 smithi071 ceph-mon[29845]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 7.2 MiB/s wr, 206 op/s 2024-09-05T12:47:44.179 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:43 smithi123 ceph-mon[34321]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 7.2 MiB/s wr, 206 op/s 2024-09-05T12:47:44.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:43 smithi174 ceph-mon[34313]: pgmap v632: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 7.2 MiB/s wr, 206 op/s 2024-09-05T12:47:44.360 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c9325530', total seconds elapsed: 28.5 2024-09-05T12:47:44.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110369 13.44 MB/sec execute 278 sec latency 24621.996 ms 2024-09-05T12:47:45.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110369 13.40 MB/sec execute 279 sec latency 25622.060 ms 2024-09-05T12:47:45.999 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:45 smithi071 ceph-mon[29845]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 7.0 MiB/s wr, 197 op/s 2024-09-05T12:47:46.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:45 smithi174 ceph-mon[34313]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 7.0 MiB/s wr, 197 op/s 2024-09-05T12:47:46.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:45 smithi123 ceph-mon[34321]: pgmap v633: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 7.0 MiB/s wr, 197 op/s 2024-09-05T12:47:46.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110369 13.35 MB/sec execute 280 sec latency 26622.121 ms 2024-09-05T12:47:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 -- 172.21.15.71:0/828487160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 msgr2=0x7f1d30103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 --2- 172.21.15.71:0/828487160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d30103f60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c009860 tx=0x7f1d1c02a340 comp rx=0 tx=0).stop 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 -- 172.21.15.71:0/828487160 shutdown_connections 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 --2- 172.21.15.71:0/828487160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d301109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 --2- 172.21.15.71:0/828487160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d30103f60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 --2- 172.21.15.71:0/828487160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30109ba0 0x7f1d30109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 -- 172.21.15.71:0/828487160 >> 172.21.15.71:0/828487160 conn(0x7f1d300ff7c0 msgr2=0x7f1d30101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 -- 172.21.15.71:0/828487160 shutdown_connections 2024-09-05T12:47:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.995+0000 7f1d2f200640 1 -- 172.21.15.71:0/828487160 wait complete. 2024-09-05T12:47:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.996+0000 7f1d2f200640 1 Processor -- start 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.996+0000 7f1d2f200640 1 -- start start 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.996+0000 7f1d2f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d3019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d3019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30109ba0 0x7f1d3019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30113f50 con 0x7f1d301044a0 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30113dd0 con 0x7f1d30103b00 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301140d0 con 0x7f1d30109ba0 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d3019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d3019a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43798/0 (socket says 172.21.15.71:43798) 2024-09-05T12:47:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 -- 172.21.15.71:0/3577278581 learned_addr learned my addr 172.21.15.71:0/3577278581 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d3019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2e800640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30109ba0 0x7f1d3019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 -- 172.21.15.71:0/3577278581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30109ba0 msgr2=0x7f1d3019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30109ba0 0x7f1d3019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 -- 172.21.15.71:0/3577278581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 msgr2=0x7f1d3019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d3019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 -- 172.21.15.71:0/3577278581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3019f450 con 0x7f1d301044a0 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d3019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:47:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d3019a9a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c035040 tx=0x7f1d1c033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d26a00640 1 -- 172.21.15.71:0/3577278581 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c037040 con 0x7f1d301044a0 2024-09-05T12:47:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d26a00640 1 -- 172.21.15.71:0/3577278581 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d1c040070 con 0x7f1d301044a0 2024-09-05T12:47:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3019f6e0 con 0x7f1d301044a0 2024-09-05T12:47:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d26a00640 1 -- 172.21.15.71:0/3577278581 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c003a10 con 0x7f1d301044a0 2024-09-05T12:47:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a6fd0 con 0x7f1d301044a0 2024-09-05T12:47:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 Processor -- start 2024-09-05T12:47:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d26a00640 1 -- 172.21.15.71:0/3577278581 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1d1c004150 con 0x7f1d301044a0 2024-09-05T12:47:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 -- start start 2024-09-05T12:47:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b88790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b93030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b95960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560851aee100 con 0x560851b93570 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560851a961b0 con 0x560851b883b0 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d37c28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560851aea0c0 con 0x560851b88cd0 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b95960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2e800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b88790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b95960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43814/0 (socket says 172.21.15.71:43814) 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2d400640 1 -- 172.21.15.71:0/652891010 learned_addr learned my addr 172.21.15.71:0/652891010 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:46.998+0000 7f1d2de00640 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b93030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 -- 172.21.15.71:0/652891010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 msgr2=0x560851b93030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b93030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 -- 172.21.15.71:0/652891010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 msgr2=0x560851b95960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b95960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 -- 172.21.15.71:0/652891010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560851801210 con 0x560851b883b0 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d26a00640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d140a2cf0 0x7f1d140a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b88790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099b0 tx=0x7f1d280095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a45887eecf290c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d26a00640 1 -- 172.21.15.71:0/3577278581 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f1d1c0ef6a0 con 0x7f1d301044a0 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d1b000640 1 -- 172.21.15.71:0/652891010 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28003970 con 0x560851b883b0 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d1b000640 1 -- 172.21.15.71:0/652891010 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2800d6b0 con 0x560851b883b0 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d1b000640 1 -- 172.21.15.71:0/652891010 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28004370 con 0x560851b883b0 2024-09-05T12:47:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d140a2cf0 0x7f1d140a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/652891010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 msgr2=0x560851b88790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b88790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099b0 tx=0x7f1d280095c0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.010 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d140a2cf0 0x7f1d140a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d20012560 tx=0x7f1d20005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/652891010 shutdown_connections 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b95960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b93030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- 172.21.15.71:0/652891010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b88790 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/652891010 >> 172.21.15.71:0/652891010 conn(0x560851b84920 msgr2=0x560851b84d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/652891010 shutdown_connections 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/652891010 wait complete. 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- start start 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 Processor -- start 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b905d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b90b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.011 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b91050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x560851b95ff0 con 0x560851b93570 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560851a77090 con 0x560851b883b0 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560851b96170 con 0x560851b88cd0 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 10 client.0 _ll_register_callbacks cb 0x560851acbf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b91050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b905d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b905d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:53612/0 (socket says 172.21.15.71:53612) 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b91050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43820/0 (socket says 172.21.15.71:43820) 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2de00640 1 -- 172.21.15.71:0/1143186135 learned_addr learned my addr 172.21.15.71:0/1143186135 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 -- 172.21.15.71:0/1143186135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 msgr2=0x560851b90b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b90b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.012 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 -- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 msgr2=0x560851b905d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b905d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 -- 172.21.15.71:0/1143186135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608518cd620 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2de00640 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d2e800640 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b91050 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099b0 tx=0x7f1d28009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2299e13e3ebe7cb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d0f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d0f400640 1 -- 172.21.15.71:0/1143186135 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28004bf0 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2800d6b0 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d0f400640 1 -- 172.21.15.71:0/1143186135 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f1d28004d90 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d0f400640 1 -- 172.21.15.71:0/1143186135 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1d28032050 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 10 client.17580 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560851ba1dd0 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d0f400640 1 -- 172.21.15.71:0/1143186135 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2800d6b0 con 0x560851b93570 2024-09-05T12:47:47.013 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d0f400640 1 -- 172.21.15.71:0/1143186135 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f1d2800b040 con 0x560851b93570 2024-09-05T12:47:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 10 client.17580 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 10 client.17580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 20 client.17580 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851ba3230 0x560851ba5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 4 client.17580 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:47:47.014 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.003+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560851afb830 con 0x560851ba3230 2024-09-05T12:47:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d2de00640 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851ba3230 0x560851ba5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d0f400640 10 client.17580 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d2de00640 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851ba3230 0x560851ba5620 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9990289328719e33 server_cookie=6c078f3f1881b1c in_seq=0 out_seq=0 2024-09-05T12:47:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d0f400640 1 -- 172.21.15.71:0/1143186135 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560851afb830 con 0x560851ba3230 2024-09-05T12:47:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d0f400640 10 client.17580 handle_command_reply: tid=0 2024-09-05T12:47:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 client.17580 shutdown 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 msgr2=0x560851b91050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b91050 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f1d280099b0 tx=0x7f1d28009be0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 shutdown_connections 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851ba3230 0x560851ba5620 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b93570 0x560851b91050 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b88cd0 0x560851b90b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 --2- 172.21.15.71:0/1143186135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b883b0 0x560851b905d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d0f400640 0 client.17580 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 >> 172.21.15.71:0/1143186135 conn(0x560851b84920 msgr2=0x560851b8a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 shutdown_connections 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 1 -- 172.21.15.71:0/1143186135 wait complete. 2024-09-05T12:47:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.008+0000 7f1d37c28740 20 client.17580 trim_cache size 0 max 16384 2024-09-05T12:47:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 Processor -- start 2024-09-05T12:47:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 -- start start 2024-09-05T12:47:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851baaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851bab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b91d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.041 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301b0f30 con 0x560851b883b0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301b2a70 con 0x560851b88cd0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d37c28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301b1600 con 0x560851b93570 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851baaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851baaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43830/0 (socket says 172.21.15.71:43830) 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 -- 172.21.15.71:0/3385148261 learned_addr learned my addr 172.21.15.71:0/3385148261 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2e800640 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b91d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 -- 172.21.15.71:0/3385148261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 msgr2=0x560851b91d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b91d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 -- 172.21.15.71:0/3385148261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 msgr2=0x560851bab1e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851bab1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.038+0000 7f1d2de00640 1 -- 172.21.15.71:0/3385148261 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560851b96170 con 0x560851b883b0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851baaca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1d20012df0 tx=0x7f1d2001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc837f8a487086b server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d1a600640 1 -- 172.21.15.71:0/3385148261 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2002b420 con 0x560851b883b0 2024-09-05T12:47:47.042 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d1a600640 1 -- 172.21.15.71:0/3385148261 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2002baa0 con 0x560851b883b0 2024-09-05T12:47:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d1a600640 1 -- 172.21.15.71:0/3385148261 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2001f430 con 0x560851b883b0 2024-09-05T12:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/3385148261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 msgr2=0x560851baaca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851baaca0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f1d20012df0 tx=0x7f1d2001b000 comp rx=0 tx=0).stop 2024-09-05T12:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/3385148261 shutdown_connections 2024-09-05T12:47:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b91d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851bab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- 172.21.15.71:0/3385148261 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851baaca0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/3385148261 >> 172.21.15.71:0/3385148261 conn(0x560851b84920 msgr2=0x560851b89550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/3385148261 shutdown_connections 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/3385148261 wait complete. 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- start start 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 Processor -- start 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851b97420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851b97960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b97ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301b23a0 con 0x560851b883b0 2024-09-05T12:47:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301b1cd0 con 0x560851b88cd0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d301aef30 con 0x560851b93570 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 10 client.0 _ll_register_callbacks cb 0x7f1d1c0f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851b97420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851b97420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43842/0 (socket says 172.21.15.71:43842) 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 -- 172.21.15.71:0/241613020 learned_addr learned my addr 172.21.15.71:0/241613020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2d400640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851b97960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2e800640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b97ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 -- 172.21.15.71:0/241613020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 msgr2=0x560851b97ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b97ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 -- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 msgr2=0x560851b97960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851b97960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 -- 172.21.15.71:0/241613020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560851c0cec0 con 0x560851b883b0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d2de00640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851b97420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1d200037e0 tx=0x7f1d2000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5001cfbc0dfc9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:47:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d0f400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d0f400640 1 -- 172.21.15.71:0/241613020 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2000f540 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d0f400640 1 -- 172.21.15.71:0/241613020 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f1d2000faf0 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2002baa0 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d0f400640 1 -- 172.21.15.71:0/241613020 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1d20046050 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 10 client.17586 fetch_fsmap learned FSMap version 76 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x560851c0e190 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d0f400640 1 -- 172.21.15.71:0/241613020 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d2002baa0 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d0f400640 1 -- 172.21.15.71:0/241613020 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f1d20005310 con 0x560851b883b0 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 10 client.17586 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 10 client.17586 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 20 client.17586 populate_metadata read hostname 'smithi071' 2024-09-05T12:47:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851c0e7f0 0x560851c10c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 4 client.17586 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:47:47.049 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.040+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560851af5b80 con 0x560851c0e7f0 2024-09-05T12:47:47.064 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.063+0000 7f1d2d400640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851c0e7f0 0x560851c10c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:47:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.123+0000 7f1d2d400640 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851c0e7f0 0x560851c10c10 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71495c45c96e3aaf server_cookie=b64c583d433bc276 in_seq=0 out_seq=0 2024-09-05T12:47:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.123+0000 7f1d0f400640 10 client.17586 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.171+0000 7f1d0f400640 1 -- 172.21.15.71:0/241613020 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560851af5b80 con 0x560851c0e7f0 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.171+0000 7f1d0f400640 10 client.17586 handle_command_reply: tid=0 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.171+0000 7f1d37c28740 1 client.17586 shutdown 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.171+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 msgr2=0x560851b97420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.171+0000 7f1d37c28740 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851b97420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1d200037e0 tx=0x7f1d2000f810 comp rx=0 tx=0).stop 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 shutdown_connections 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x560851c0e7f0 0x560851c10c10 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560851b93570 0x560851b97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560851b88cd0 0x560851b97960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 --2- 172.21.15.71:0/241613020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560851b883b0 0x560851b97420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d0f400640 0 client.17586 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 >> 172.21.15.71:0/241613020 conn(0x560851b84920 msgr2=0x560851b90570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 shutdown_connections 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 1 -- 172.21.15.71:0/241613020 wait complete. 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d37c28740 20 client.17586 trim_cache size 0 max 16384 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:47:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:47:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:47:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d140a2cf0 msgr2=0x7f1d140a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d140a2cf0 0x7f1d140a51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1d20012560 tx=0x7f1d20005f90 comp rx=0 tx=0).stop 2024-09-05T12:47:47.176 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 msgr2=0x7f1d3019a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d3019a9a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c035040 tx=0x7f1d1c033b60 comp rx=0 tx=0).stop 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 shutdown_connections 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30109ba0 0x7f1d3019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f1d140a2cf0 0x7f1d140a51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d301044a0 0x7f1d3019a9a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 --2- 172.21.15.71:0/3577278581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d30103b00 0x7f1d3019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 >> 172.21.15.71:0/3577278581 conn(0x7f1d300ff7c0 msgr2=0x7f1d30100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 shutdown_connections 2024-09-05T12:47:47.177 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:47:47.172+0000 7f1d2f200640 1 -- 172.21.15.71:0/3577278581 wait complete. 2024-09-05T12:47:47.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110373 13.30 MB/sec execute 281 sec latency 27622.200 ms 2024-09-05T12:47:48.248 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:47 smithi071 ceph-mon[29845]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 7.2 MiB/s wr, 213 op/s 2024-09-05T12:47:48.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:47 smithi174 ceph-mon[34313]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 7.2 MiB/s wr, 213 op/s 2024-09-05T12:47:48.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:47 smithi123 ceph-mon[34321]: pgmap v634: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 7.2 MiB/s wr, 213 op/s 2024-09-05T12:47:48.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110374 13.25 MB/sec execute 282 sec latency 28339.917 ms 2024-09-05T12:47:49.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110454 13.22 MB/sec execute 283 sec latency 28863.840 ms 2024-09-05T12:47:50.163 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:49 smithi071 ceph-mon[29845]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 4.5 MiB/s wr, 138 op/s 2024-09-05T12:47:50.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:49 smithi174 ceph-mon[34313]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 4.5 MiB/s wr, 138 op/s 2024-09-05T12:47:50.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:49 smithi123 ceph-mon[34321]: pgmap v635: 129 pgs: 129 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 4.5 MiB/s wr, 138 op/s 2024-09-05T12:47:50.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110549 13.19 MB/sec execute 284 sec latency 4547.690 ms 2024-09-05T12:47:51.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 110996 13.17 MB/sec execute 285 sec latency 4856.441 ms 2024-09-05T12:47:52.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:51 smithi174 ceph-mon[34313]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 184 op/s 2024-09-05T12:47:52.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:51 smithi071 ceph-mon[29845]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 184 op/s 2024-09-05T12:47:52.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:51 smithi123 ceph-mon[34321]: pgmap v636: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 184 op/s 2024-09-05T12:47:52.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 111939 13.33 MB/sec execute 286 sec latency 60.027 ms 2024-09-05T12:47:53.854 INFO:tasks.workunit.client.0.smithi071.stdout: 10 112876 13.45 MB/sec execute 287 sec latency 64.497 ms 2024-09-05T12:47:54.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:53 smithi174 ceph-mon[34313]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-05T12:47:54.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:53 smithi071 ceph-mon[29845]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-05T12:47:54.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:53 smithi123 ceph-mon[34321]: pgmap v637: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 155 op/s 2024-09-05T12:47:54.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 113555 13.55 MB/sec execute 288 sec latency 65.442 ms 2024-09-05T12:47:55.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 114168 13.61 MB/sec execute 289 sec latency 83.449 ms 2024-09-05T12:47:56.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:55 smithi174 ceph-mon[34313]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1007 KiB/s wr, 133 op/s 2024-09-05T12:47:56.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:55 smithi071 ceph-mon[29845]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1007 KiB/s wr, 133 op/s 2024-09-05T12:47:56.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:55 smithi123 ceph-mon[34321]: pgmap v638: 129 pgs: 129 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1007 KiB/s wr, 133 op/s 2024-09-05T12:47:56.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 114872 13.69 MB/sec execute 290 sec latency 126.795 ms 2024-09-05T12:47:57.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 115687 13.80 MB/sec execute 291 sec latency 61.645 ms 2024-09-05T12:47:58.046 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:47:57 smithi071 ceph-mon[29845]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-09-05T12:47:58.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:47:57 smithi174 ceph-mon[34313]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-09-05T12:47:58.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:47:57 smithi123 ceph-mon[34321]: pgmap v639: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.9 MiB/s wr, 363 op/s 2024-09-05T12:47:58.855 INFO:tasks.workunit.client.0.smithi071.stdout: 10 116364 13.88 MB/sec execute 292 sec latency 232.136 ms 2024-09-05T12:47:59.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 117145 13.98 MB/sec execute 293 sec latency 50.630 ms 2024-09-05T12:48:00.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:00 smithi174 ceph-mon[34313]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.7 MiB/s wr, 420 op/s 2024-09-05T12:48:00.318 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:00 smithi071 ceph-mon[29845]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.7 MiB/s wr, 420 op/s 2024-09-05T12:48:00.366 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 -- 172.21.15.71:0/4264057156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c103f40 msgr2=0x7f0d2c104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 --2- 172.21.15.71:0/4264057156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c103f40 0x7f0d2c104320 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009ca0 tx=0x7f0d2402a8c0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 -- 172.21.15.71:0/4264057156 shutdown_connections 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 --2- 172.21.15.71:0/4264057156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c109090 0x7f0d2c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 --2- 172.21.15.71:0/4264057156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c1048f0 0x7f0d2c108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 --2- 172.21.15.71:0/4264057156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c103f40 0x7f0d2c104320 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.365+0000 7f0d30a00640 1 -- 172.21.15.71:0/4264057156 >> 172.21.15.71:0/4264057156 conn(0x7f0d2c0ff9b0 msgr2=0x7f0d2c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:00.367 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.367+0000 7f0d30a00640 1 -- 172.21.15.71:0/4264057156 shutdown_connections 2024-09-05T12:48:00.369 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.367+0000 7f0d30a00640 1 -- 172.21.15.71:0/4264057156 wait complete. 2024-09-05T12:48:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.369+0000 7f0d30a00640 1 Processor -- start 2024-09-05T12:48:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.369+0000 7f0d30a00640 1 -- start start 2024-09-05T12:48:00.370 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d30a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c103f40 0x7f0d2c073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d30a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c109090 0x7f0d2c073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d30a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c06caf0 0x7f0d2c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d30a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c114180 con 0x7f0d2c103f40 2024-09-05T12:48:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d30a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c114000 con 0x7f0d2c06caf0 2024-09-05T12:48:00.371 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d30a00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c114300 con 0x7f0d2c109090 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d29e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c103f40 0x7f0d2c073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d29400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c109090 0x7f0d2c073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d29400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c109090 0x7f0d2c073ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35510/0 (socket says 172.21.15.71:35510) 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d29400640 1 -- 172.21.15.71:0/370600892 learned_addr learned my addr 172.21.15.71:0/370600892 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.370+0000 7f0d2a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c06caf0 0x7f0d2c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.371+0000 7f0d2a800640 1 -- 172.21.15.71:0/370600892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c109090 msgr2=0x7f0d2c073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.372 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.371+0000 7f0d2a800640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c109090 0x7f0d2c073ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.371+0000 7f0d2a800640 1 -- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c103f40 msgr2=0x7f0d2c073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.371+0000 7f0d2a800640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c103f40 0x7f0d2c073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.371+0000 7f0d2a800640 1 -- 172.21.15.71:0/370600892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2c0714a0 con 0x7f0d2c06caf0 2024-09-05T12:48:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.372+0000 7f0d2a800640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c06caf0 0x7f0d2c06cfa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800e970 tx=0x7f0d1800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.373+0000 7f0d22a00640 1 -- 172.21.15.71:0/370600892 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800ccc0 con 0x7f0d2c06caf0 2024-09-05T12:48:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.373+0000 7f0d22a00640 1 -- 172.21.15.71:0/370600892 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d18005020 con 0x7f0d2c06caf0 2024-09-05T12:48:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.373+0000 7f0d22a00640 1 -- 172.21.15.71:0/370600892 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004a00 con 0x7f0d2c06caf0 2024-09-05T12:48:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.373+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2c071790 con 0x7f0d2c06caf0 2024-09-05T12:48:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.373+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d2c071c90 con 0x7f0d2c06caf0 2024-09-05T12:48:00.379 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.379+0000 7f0d33899740 1 Processor -- start 2024-09-05T12:48:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.381+0000 7f0d33899740 1 -- start start 2024-09-05T12:48:00.382 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d22a00640 1 -- 172.21.15.71:0/370600892 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0d180051d0 con 0x7f0d2c06caf0 2024-09-05T12:48:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d33899740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d33899740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d33899740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3ebfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d33899740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561fdd33c100 con 0x561fdd3d6a20 2024-09-05T12:48:00.383 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d33899740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561fdd2e41b0 con 0x561fdd3e9bb0 2024-09-05T12:48:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d33899740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fdd3380c0 con 0x561fdd3d6100 2024-09-05T12:48:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d29400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d29400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3d64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35526/0 (socket says 172.21.15.71:35526) 2024-09-05T12:48:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d29400640 1 -- 172.21.15.71:0/3070762920 learned_addr learned my addr 172.21.15.71:0/3070762920 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:00.384 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.382+0000 7f0d29400640 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3ebfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.386 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.383+0000 7f0d29e00640 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d29400640 1 -- 172.21.15.71:0/3070762920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 msgr2=0x561fdd3d64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d29400640 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3d64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d29400640 1 -- 172.21.15.71:0/3070762920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 msgr2=0x561fdd3e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d29400640 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d29400640 1 -- 172.21.15.71:0/3070762920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdd04f210 con 0x561fdd3e9bb0 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d22a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0d100a2f80 0x7f0d100a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d29e00640 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.384+0000 7f0d22a00640 1 -- 172.21.15.71:0/370600892 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f0d180cf800 con 0x7f0d2c06caf0 2024-09-05T12:48:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.385+0000 7f0d29e00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0d100a2f80 0x7f0d100a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.386+0000 7f0d29e00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0d100a2f80 0x7f0d100a5440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561fdd3d8d80 tx=0x7f0d2402f440 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.388+0000 7f0d29400640 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3ebfa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c016540 tx=0x7f0d1c006e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c348135254c28f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.389+0000 7f0d17000640 1 -- 172.21.15.71:0/3070762920 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c018430 con 0x561fdd3e9bb0 2024-09-05T12:48:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.389+0000 7f0d17000640 1 -- 172.21.15.71:0/3070762920 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c018ab0 con 0x561fdd3e9bb0 2024-09-05T12:48:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.389+0000 7f0d17000640 1 -- 172.21.15.71:0/3070762920 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c017a20 con 0x561fdd3e9bb0 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.390+0000 7f0d33899740 1 -- 172.21.15.71:0/3070762920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 msgr2=0x561fdd3ebfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.390+0000 7f0d33899740 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3ebfa0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c016540 tx=0x7f0d1c006e30 comp rx=0 tx=0).stop 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.391+0000 7f0d33899740 1 -- 172.21.15.71:0/3070762920 shutdown_connections 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.391+0000 7f0d33899740 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3ebfa0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.391+0000 7f0d33899740 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.391+0000 7f0d33899740 1 --2- 172.21.15.71:0/3070762920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3d64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.391+0000 7f0d33899740 1 -- 172.21.15.71:0/3070762920 >> 172.21.15.71:0/3070762920 conn(0x561fdd3d2750 msgr2=0x561fdd3d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.391+0000 7f0d33899740 1 -- 172.21.15.71:0/3070762920 shutdown_connections 2024-09-05T12:48:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.392+0000 7f0d33899740 1 -- 172.21.15.71:0/3070762920 wait complete. 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.394+0000 7f0d33899740 1 -- start start 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 Processor -- start 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561fdd3ec720 con 0x561fdd3d6a20 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561fdd2c5090 con 0x561fdd3e9bb0 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.395+0000 7f0d33899740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561fdd3ec8a0 con 0x561fdd3d6100 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.396+0000 7f0d33899740 10 client.0 _ll_register_callbacks cb 0x561fdd319f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.396+0000 7f0d29e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.396+0000 7f0d29e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3e6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35528/0 (socket says 172.21.15.71:35528) 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.396+0000 7f0d29e00640 1 -- 172.21.15.71:0/610626401 learned_addr learned my addr 172.21.15.71:0/610626401 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29e00640 1 -- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 msgr2=0x561fdd3e76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29e00640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29e00640 1 -- 172.21.15.71:0/610626401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 msgr2=0x561fdd3e7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29400640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29e00640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29e00640 1 -- 172.21.15.71:0/610626401 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdd11b620 con 0x561fdd3d6100 2024-09-05T12:48:00.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.397+0000 7f0d29400640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:48:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.398+0000 7f0d29e00640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3e6c20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009ca0 tx=0x7f0d240095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bca1dedb5d545d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:00.401 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.398+0000 7f0d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-05T12:48:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.399+0000 7f0d0b400640 1 -- 172.21.15.71:0/610626401 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2404f070 con 0x561fdd3d6100 2024-09-05T12:48:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.399+0000 7f0d0b400640 1 -- 172.21.15.71:0/610626401 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f0d2402f810 con 0x561fdd3d6100 2024-09-05T12:48:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.399+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1c018ab0 con 0x561fdd3d6100 2024-09-05T12:48:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.400+0000 7f0d0b400640 1 -- 172.21.15.71:0/610626401 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f0d2402fa10 con 0x561fdd3d6100 2024-09-05T12:48:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.400+0000 7f0d33899740 10 client.27664 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:00.402 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.400+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x561fdd3f84b0 con 0x561fdd3d6100 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.401+0000 7f0d0b400640 1 -- 172.21.15.71:0/610626401 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c018ab0 con 0x561fdd3d6100 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.401+0000 7f0d0b400640 1 -- 172.21.15.71:0/610626401 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f0d2405c020 con 0x561fdd3d6100 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.401+0000 7f0d33899740 10 client.27664 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.401+0000 7f0d33899740 10 client.27664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.402+0000 7f0d33899740 20 client.27664 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.402+0000 7f0d33899740 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd3f9a90 0x561fdd3fbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.402+0000 7f0d33899740 4 client.27664 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:48:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.402+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fdd349830 con 0x561fdd3f9a90 2024-09-05T12:48:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.404+0000 7f0d29400640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd3f9a90 0x561fdd3fbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.409+0000 7f0d29400640 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd3f9a90 0x561fdd3fbf50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe529e694af59e01 server_cookie=82942b0510da4fee in_seq=0 out_seq=0 2024-09-05T12:48:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.409+0000 7f0d0b400640 10 client.27664 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.412+0000 7f0d0b400640 1 -- 172.21.15.71:0/610626401 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fdd349830 con 0x561fdd3f9a90 2024-09-05T12:48:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.412+0000 7f0d0b400640 10 client.27664 handle_command_reply: tid=0 2024-09-05T12:48:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.412+0000 7f0d33899740 1 client.27664 shutdown 2024-09-05T12:48:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.413+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 msgr2=0x561fdd3e6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3e6c20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009ca0 tx=0x7f0d240095c0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 shutdown_connections 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd3f9a90 0x561fdd3fbf50 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 --2- 172.21.15.71:0/610626401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3e6c20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 >> 172.21.15.71:0/610626401 conn(0x561fdd3d2750 msgr2=0x561fdd3d84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.414+0000 7f0d0b400640 0 client.27664 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.415+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 shutdown_connections 2024-09-05T12:48:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.415+0000 7f0d33899740 1 -- 172.21.15.71:0/610626401 wait complete. 2024-09-05T12:48:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.415+0000 7f0d33899740 20 client.27664 trim_cache size 0 max 16384 2024-09-05T12:48:00.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:00 smithi123 ceph-mon[34321]: pgmap v640: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.7 MiB/s wr, 420 op/s 2024-09-05T12:48:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 Processor -- start 2024-09-05T12:48:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 -- start start 2024-09-05T12:48:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3d6100 0x561fdd3e9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 0x561fdd401820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.449 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3e9bb0 0x561fdd401d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1b5490 con 0x561fdd3e9bb0 2024-09-05T12:48:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1b6fd0 con 0x561fdd3d6100 2024-09-05T12:48:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.448+0000 7f0d33899740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1b5b60 con 0x561fdd3d6a20 2024-09-05T12:48:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.449+0000 7f0d29400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 0x561fdd401820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.449+0000 7f0d29400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 0x561fdd401820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35534/0 (socket says 172.21.15.71:35534) 2024-09-05T12:48:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.449+0000 7f0d29400640 1 -- 172.21.15.71:0/3678195470 learned_addr learned my addr 172.21.15.71:0/3678195470 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.449+0000 7f0d2a800640 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3e9bb0 0x561fdd401d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.449+0000 7f0d29e00640 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3d6100 0x561fdd3e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.450+0000 7f0d29400640 1 -- 172.21.15.71:0/3678195470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3d6100 msgr2=0x561fdd3e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.450+0000 7f0d29400640 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3d6100 0x561fdd3e9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.450+0000 7f0d29400640 1 -- 172.21.15.71:0/3678195470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3e9bb0 msgr2=0x561fdd401d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.450+0000 7f0d29400640 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3e9bb0 0x561fdd401d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.450+0000 7f0d29400640 1 -- 172.21.15.71:0/3678195470 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fdd3d6810 con 0x561fdd3d6a20 2024-09-05T12:48:00.452 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.450+0000 7f0d2a800640 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3e9bb0 0x561fdd401d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:48:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.451+0000 7f0d29400640 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 0x561fdd401820 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c01a040 tx=0x7f0d1c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dbffdf158725c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.451+0000 7f0d16600640 1 -- 172.21.15.71:0/3678195470 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c018430 con 0x561fdd3d6a20 2024-09-05T12:48:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.452+0000 7f0d16600640 1 -- 172.21.15.71:0/3678195470 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c026030 con 0x561fdd3d6a20 2024-09-05T12:48:00.453 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.452+0000 7f0d16600640 1 -- 172.21.15.71:0/3678195470 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c017df0 con 0x561fdd3d6a20 2024-09-05T12:48:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 -- 172.21.15.71:0/3678195470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 msgr2=0x561fdd401820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 0x561fdd401820 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c01a040 tx=0x7f0d1c0046c0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 -- 172.21.15.71:0/3678195470 shutdown_connections 2024-09-05T12:48:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3e9bb0 0x561fdd401d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6a20 0x561fdd401820 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 --2- 172.21.15.71:0/3678195470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3d6100 0x561fdd3e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.453+0000 7f0d33899740 1 -- 172.21.15.71:0/3678195470 >> 172.21.15.71:0/3678195470 conn(0x561fdd3d2750 msgr2=0x561fdd3e7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.454+0000 7f0d33899740 1 -- 172.21.15.71:0/3678195470 shutdown_connections 2024-09-05T12:48:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.454+0000 7f0d33899740 1 -- 172.21.15.71:0/3678195470 wait complete. 2024-09-05T12:48:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.454+0000 7f0d33899740 1 -- start start 2024-09-05T12:48:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.455+0000 7f0d33899740 1 Processor -- start 2024-09-05T12:48:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.456+0000 7f0d33899740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3f1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.456+0000 7f0d33899740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.456+0000 7f0d33899740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d29e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3f1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d29e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3f1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:35546/0 (socket says 172.21.15.71:35546) 2024-09-05T12:48:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d2a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d2a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:48580/0 (socket says 172.21.15.71:48580) 2024-09-05T12:48:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d2a800640 1 -- 172.21.15.71:0/2466000316 learned_addr learned my addr 172.21.15.71:0/2466000316 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1b6900 con 0x561fdd3d6a20 2024-09-05T12:48:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1b6230 con 0x561fdd3e9bb0 2024-09-05T12:48:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2c1b34f0 con 0x561fdd3d6100 2024-09-05T12:48:00.459 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.457+0000 7f0d33899740 10 client.0 _ll_register_callbacks cb 0x7f0d18005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d29400640 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d2a800640 1 -- 172.21.15.71:0/2466000316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 msgr2=0x561fdd3f1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d2a800640 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3f1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.460 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d2a800640 1 -- 172.21.15.71:0/2466000316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 msgr2=0x561fdd3e3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d2a800640 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d2a800640 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fdd3e40d0 con 0x561fdd3e9bb0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d2a800640 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e3b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d1809b8e0 tx=0x7f0d18005620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1edbfcddfdfd137a server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d0b400640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.458+0000 7f0d0b400640 1 -- 172.21.15.71:0/2466000316 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d180b3070 con 0x561fdd3e9bb0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.459+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1c026030 con 0x561fdd3e9bb0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.459+0000 7f0d0b400640 1 -- 172.21.15.71:0/2466000316 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(125..125 src has 1..125) ==== 9394+0+0 (secure 0 0 0) 0x7f0d180a4130 con 0x561fdd3e9bb0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.459+0000 7f0d0b400640 1 -- 172.21.15.71:0/2466000316 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f0d1801c040 con 0x561fdd3e9bb0 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.459+0000 7f0d33899740 10 client.27605 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:00.461 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.459+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x561fdd464560 con 0x561fdd3e9bb0 2024-09-05T12:48:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d0b400640 1 -- 172.21.15.71:0/2466000316 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c026030 con 0x561fdd3e9bb0 2024-09-05T12:48:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d0b400640 1 -- 172.21.15.71:0/2466000316 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f0d180a6ce0 con 0x561fdd3e9bb0 2024-09-05T12:48:00.462 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d33899740 10 client.27605 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d33899740 10 client.27605 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d33899740 20 client.27605 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d33899740 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd464c60 0x561fdd467080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d33899740 4 client.27605 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.460+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fdd343b80 con 0x561fdd464c60 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.461+0000 7f0d29e00640 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd464c60 0x561fdd467080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.462+0000 7f0d29e00640 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd464c60 0x561fdd467080 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6adbfb7cf5495efd server_cookie=524030a31f72d315 in_seq=0 out_seq=0 2024-09-05T12:48:00.463 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.462+0000 7f0d0b400640 10 client.27605 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:00.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 117593 14.02 MB/sec execute 294 sec latency 296.115 ms 2024-09-05T12:48:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d0b400640 1 -- 172.21.15.71:0/2466000316 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fdd343b80 con 0x561fdd464c60 2024-09-05T12:48:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d0b400640 10 client.27605 handle_command_reply: tid=0 2024-09-05T12:48:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 client.27605 shutdown 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 msgr2=0x561fdd3e3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e3b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0d1809b8e0 tx=0x7f0d18005620 comp rx=0 tx=0).stop 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 shutdown_connections 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x561fdd464c60 0x561fdd467080 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561fdd3e9bb0 0x561fdd3e3b90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fdd3d6a20 0x561fdd3e3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 --2- 172.21.15.71:0/2466000316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561fdd3d6100 0x561fdd3f1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 >> 172.21.15.71:0/2466000316 conn(0x561fdd3d2750 msgr2=0x561fdd3d7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.876+0000 7f0d0b400640 0 client.27605 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.877+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 shutdown_connections 2024-09-05T12:48:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.877+0000 7f0d33899740 1 -- 172.21.15.71:0/2466000316 wait complete. 2024-09-05T12:48:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.877+0000 7f0d33899740 20 client.27605 trim_cache size 0 max 16384 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.878+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0d100a2f80 msgr2=0x7f0d100a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.878+0000 7f0d30a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0d100a2f80 0x7f0d100a5440 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x561fdd3d8d80 tx=0x7f0d2402f440 comp rx=0 tx=0).stop 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.878+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c06caf0 msgr2=0x7f0d2c06cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.878+0000 7f0d30a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c06caf0 0x7f0d2c06cfa0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800e970 tx=0x7f0d1800ee30 comp rx=0 tx=0).stop 2024-09-05T12:48:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 shutdown_connections 2024-09-05T12:48:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d2c06caf0 0x7f0d2c06cfa0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d2c109090 0x7f0d2c073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f0d100a2f80 0x7f0d100a5440 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 --2- 172.21.15.71:0/370600892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d2c103f40 0x7f0d2c073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 >> 172.21.15.71:0/370600892 conn(0x7f0d2c0ff9b0 msgr2=0x7f0d2c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 shutdown_connections 2024-09-05T12:48:00.882 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:00.880+0000 7f0d30a00640 1 -- 172.21.15.71:0/370600892 wait complete. 2024-09-05T12:48:01.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 117827 13.99 MB/sec execute 295 sec latency 570.863 ms 2024-09-05T12:48:02.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:02 smithi174 ceph-mon[34313]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 6.9 MiB/s wr, 608 op/s 2024-09-05T12:48:02.412 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:02 smithi071 ceph-mon[29845]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 6.9 MiB/s wr, 608 op/s 2024-09-05T12:48:02.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:02 smithi123 ceph-mon[34321]: pgmap v641: 129 pgs: 129 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 6.9 MiB/s wr, 608 op/s 2024-09-05T12:48:02.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 118082 13.96 MB/sec execute 296 sec latency 320.806 ms 2024-09-05T12:48:03.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 118119 13.92 MB/sec execute 297 sec latency 961.428 ms 2024-09-05T12:48:04.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:04 smithi071 ceph-mon[29845]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.7 MiB/s wr, 645 op/s 2024-09-05T12:48:04.162 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:04 smithi071 ceph-mon[29845]: osdmap e126: 12 total, 12 up, 12 in 2024-09-05T12:48:04.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:04 smithi123 ceph-mon[34321]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.7 MiB/s wr, 645 op/s 2024-09-05T12:48:04.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:04 smithi123 ceph-mon[34321]: osdmap e126: 12 total, 12 up, 12 in 2024-09-05T12:48:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:04 smithi174 ceph-mon[34313]: pgmap v642: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 7.7 MiB/s wr, 645 op/s 2024-09-05T12:48:04.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:04 smithi174 ceph-mon[34313]: osdmap e126: 12 total, 12 up, 12 in 2024-09-05T12:48:04.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 118125 13.87 MB/sec execute 298 sec latency 1315.888 ms 2024-09-05T12:48:05.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 118162 13.84 MB/sec execute 299 sec latency 841.910 ms 2024-09-05T12:48:06.411 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:06 smithi071 ceph-mon[29845]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 8.6 MiB/s wr, 723 op/s 2024-09-05T12:48:06.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:06 smithi123 ceph-mon[34321]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 8.6 MiB/s wr, 723 op/s 2024-09-05T12:48:06.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:06 smithi174 ceph-mon[34313]: pgmap v644: 129 pgs: 129 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 8.6 MiB/s wr, 723 op/s 2024-09-05T12:48:06.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 118398 13.86 MB/sec execute 300 sec latency 445.199 ms 2024-09-05T12:48:07.858 INFO:tasks.workunit.client.0.smithi071.stdout: 10 118854 13.90 MB/sec execute 301 sec latency 299.593 ms 2024-09-05T12:48:08.368 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:08 smithi071 ceph-mon[29845]: pgmap v645: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.2 MiB/s wr, 630 op/s 2024-09-05T12:48:08.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:08 smithi123 ceph-mon[34321]: pgmap v645: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.2 MiB/s wr, 630 op/s 2024-09-05T12:48:08.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:08 smithi174 ceph-mon[34313]: pgmap v645: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.2 MiB/s wr, 630 op/s 2024-09-05T12:48:08.856 INFO:tasks.workunit.client.0.smithi071.stdout: 10 119618 14.01 MB/sec execute 302 sec latency 244.263 ms 2024-09-05T12:48:09.857 INFO:tasks.workunit.client.0.smithi071.stdout: 10 120069 14.04 MB/sec execute 303 sec latency 392.758 ms 2024-09-05T12:48:09.863 INFO:tasks.workunit.client.0.smithi071.stdout:[122158] write failed on handle 10327 (Resource temporarily unavailable) 2024-09-05T12:48:09.865 INFO:tasks.workunit.client.0.smithi071.stdout:Child failed with status 1 2024-09-05T12:48:09.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:48:09.888 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-05T12:48:09.888 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-05T12:48:10.161 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:10 smithi071 ceph-mon[29845]: pgmap v646: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.3 MiB/s wr, 588 op/s 2024-09-05T12:48:10.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:10 smithi123 ceph-mon[34321]: pgmap v646: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.3 MiB/s wr, 588 op/s 2024-09-05T12:48:10.510 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:10 smithi174 ceph-mon[34313]: pgmap v646: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.3 MiB/s wr, 588 op/s 2024-09-05T12:48:11.000 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e856d6b2462509a377ef39149a05475ed006ebac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-05T12:48:15.886 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=74a2ce9eec454e6bae926a672758bbab Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_e856d6b2462509a377ef39149a05475ed006ebac/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi071 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e856d6b2462509a377ef39149a05475ed006ebac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-05T12:48:15.888 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-05T12:48:15.899 INFO:tasks.quiescer:joining Quiescers 2024-09-05T12:48:15.899 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-05T12:48:15.899 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:48:15.904 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:11 smithi071 ceph-mon[29845]: osdmap e127: 12 total, 12 up, 12 in 2024-09-05T12:48:15.905 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:12 smithi071 ceph-mon[29845]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-09-05T12:48:15.905 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:11 smithi174 ceph-mon[34313]: osdmap e127: 12 total, 12 up, 12 in 2024-09-05T12:48:15.905 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:12 smithi174 ceph-mon[34313]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-09-05T12:48:15.905 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:14 smithi174 ceph-mon[34313]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-09-05T12:48:15.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:11 smithi123 ceph-mon[34321]: osdmap e127: 12 total, 12 up, 12 in 2024-09-05T12:48:15.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:12 smithi123 ceph-mon[34321]: pgmap v648: 129 pgs: 129 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.7 MiB/s wr, 512 op/s 2024-09-05T12:48:15.906 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:14 smithi123 ceph-mon[34321]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-09-05T12:48:15.928 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 -- 172.21.15.71:0/2658233835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834103f30 msgr2=0x7f2834104310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 --2- 172.21.15.71:0/2658233835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834103f30 0x7f2834104310 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2824009900 tx=0x7f282402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 -- 172.21.15.71:0/2658233835 shutdown_connections 2024-09-05T12:48:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 --2- 172.21.15.71:0/2658233835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834109080 0x7f2834110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 --2- 172.21.15.71:0/2658233835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28341048e0 0x7f2834108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 --2- 172.21.15.71:0/2658233835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834103f30 0x7f2834104310 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.929 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 -- 172.21.15.71:0/2658233835 >> 172.21.15.71:0/2658233835 conn(0x7f28340ff980 msgr2=0x7f2834101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 -- 172.21.15.71:0/2658233835 shutdown_connections 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.014+0000 7f283aa00640 1 -- 172.21.15.71:0/2658233835 wait complete. 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.015+0000 7f283aa00640 1 Processor -- start 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.015+0000 7f283aa00640 1 -- start start 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f283aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834103f30 0x7f283419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f283aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 0x7f283419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.930 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f283aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f283419ee20 0x7f283419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f283aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28341140f0 con 0x7f2834109080 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f283aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2834113f70 con 0x7f283419ee20 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f283aa00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2834114270 con 0x7f2834103f30 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 0x7f283419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834103f30 0x7f283419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 0x7f283419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39528/0 (socket says 172.21.15.71:39528) 2024-09-05T12:48:15.931 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834103f30 0x7f283419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:45270/0 (socket says 172.21.15.71:45270) 2024-09-05T12:48:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 -- 172.21.15.71:0/114364787 learned_addr learned my addr 172.21.15.71:0/114364787 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2838a00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f283419ee20 0x7f283419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 -- 172.21.15.71:0/114364787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834103f30 msgr2=0x7f283419a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834103f30 0x7f283419a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 -- 172.21.15.71:0/114364787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f283419ee20 msgr2=0x7f283419f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.932 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f283419ee20 0x7f283419f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.016+0000 7f2833400640 1 -- 172.21.15.71:0/114364787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283419f910 con 0x7f2834109080 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.017+0000 7f2833400640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 0x7f283419aaa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f282000e9c0 tx=0x7f282000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.017+0000 7f2830c00640 1 -- 172.21.15.71:0/114364787 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282000cd80 con 0x7f2834109080 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.017+0000 7f2830c00640 1 -- 172.21.15.71:0/114364787 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2820005500 con 0x7f2834109080 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.017+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283419fc00 con 0x7f2834109080 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.017+0000 7f2830c00640 1 -- 172.21.15.71:0/114364787 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2820010690 con 0x7f2834109080 2024-09-05T12:48:15.933 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.017+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28341a73f0 con 0x7f2834109080 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 Processor -- start 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 -- start start 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f2830c00640 1 -- 172.21.15.71:0/114364787 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f28200046c0 con 0x7f2834109080 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bcb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 0x55ef45bd8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.934 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ef45b31100 con 0x55ef45bd63a0 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef45ad91b0 con 0x55ef45bcbb00 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.020+0000 7f283d887740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef45b2d0c0 con 0x55ef45bcb1e0 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 0x55ef45bd8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2833e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 0x55ef45bd8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39538/0 (socket says 172.21.15.71:39538) 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2838a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bcb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.935 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2833400640 1 -- 172.21.15.71:0/1096340088 learned_addr learned my addr 172.21.15.71:0/1096340088 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2838a00640 1 -- 172.21.15.71:0/1096340088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 msgr2=0x55ef45bd5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2838a00640 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2838a00640 1 -- 172.21.15.71:0/1096340088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 msgr2=0x55ef45bd8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2838a00640 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 0x55ef45bd8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2838a00640 1 -- 172.21.15.71:0/1096340088 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef45844210 con 0x55ef45bcb1e0 2024-09-05T12:48:15.936 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2833e00640 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.021+0000 7f2833400640 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 0x55ef45bd8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2838a00640 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bcb5c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f282c009f00 tx=0x7f282c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f778fdf47357ea server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2829200640 1 -- 172.21.15.71:0/1096340088 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c004d20 con 0x55ef45bcb1e0 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2830c00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f28180a2da0 0x7f28180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2829200640 1 -- 172.21.15.71:0/1096340088 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282c004050 con 0x55ef45bcb1e0 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2829200640 1 -- 172.21.15.71:0/1096340088 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c005280 con 0x55ef45bcb1e0 2024-09-05T12:48:15.937 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2830c00640 1 -- 172.21.15.71:0/114364787 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f28200cfb80 con 0x7f2834109080 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.022+0000 7f2833e00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f28180a2da0 0x7f28180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 -- 172.21.15.71:0/1096340088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 msgr2=0x55ef45bcb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bcb5c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f282c009f00 tx=0x7f282c011810 comp rx=0 tx=0).stop 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 -- 172.21.15.71:0/1096340088 shutdown_connections 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bd63a0 0x55ef45bd8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.938 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 --2- 172.21.15.71:0/1096340088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bcb5c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 -- 172.21.15.71:0/1096340088 >> 172.21.15.71:0/1096340088 conn(0x55ef45bc7750 msgr2=0x55ef45bc7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f2833e00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f28180a2da0 0x7f28180a5260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2824000c00 tx=0x7f2824005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 -- 172.21.15.71:0/1096340088 shutdown_connections 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 -- 172.21.15.71:0/1096340088 wait complete. 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.023+0000 7f283d887740 1 -- start start 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 Processor -- start 2024-09-05T12:48:15.939 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bd3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ef45bd8fd0 con 0x55ef45bcb1e0 2024-09-05T12:48:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef45aba090 con 0x55ef45bcbb00 2024-09-05T12:48:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef45bd9150 con 0x55ef45bd63a0 2024-09-05T12:48:15.940 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f283d887740 10 client.0 _ll_register_callbacks cb 0x55ef45b0ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f2833e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f2833400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f2833e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39546/0 (socket says 172.21.15.71:39546) 2024-09-05T12:48:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f2833e00640 1 -- 172.21.15.71:0/3544517270 learned_addr learned my addr 172.21.15.71:0/3544517270 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.024+0000 7f2838a00640 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bd3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.941 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f2833400640 1 -- 172.21.15.71:0/3544517270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 msgr2=0x55ef45bd3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f2833400640 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bd3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f2833400640 1 -- 172.21.15.71:0/3544517270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcb1e0 msgr2=0x55ef45bd3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f2833400640 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f2833400640 1 -- 172.21.15.71:0/3544517270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef45910620 con 0x55ef45bcbb00 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f2833400640 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd3860 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f282009bca0 tx=0x7f2820004a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a94e854f9eecf256 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f281d600640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:48:15.942 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f281d600640 1 -- 172.21.15.71:0/3544517270 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282009bae0 con 0x55ef45bcbb00 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f282c004050 con 0x55ef45bcbb00 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f281d600640 1 -- 172.21.15.71:0/3544517270 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f28200cd1c0 con 0x55ef45bcbb00 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f281d600640 1 -- 172.21.15.71:0/3544517270 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f282009b670 con 0x55ef45bcbb00 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.025+0000 7f283d887740 10 client.27617 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ef45be4c20 con 0x55ef45bcbb00 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f281d600640 1 -- 172.21.15.71:0/3544517270 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282c004050 con 0x55ef45bcbb00 2024-09-05T12:48:15.943 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f281d600640 1 -- 172.21.15.71:0/3544517270 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f28200a44f0 con 0x55ef45bcbb00 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 10 client.27617 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 10 client.27617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 20 client.27617 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45be6090 0x55ef45be8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 4 client.27617 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef45b3e830 con 0x55ef45be6090 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.026+0000 7f2833e00640 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45be6090 0x55ef45be8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.944 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.027+0000 7f281d600640 10 client.27617 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.027+0000 7f2833e00640 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45be6090 0x55ef45be8480 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf2c1923539f400c server_cookie=f6c5b3187fcf1dd8 in_seq=0 out_seq=0 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f281d600640 1 -- 172.21.15.71:0/3544517270 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef45b3e830 con 0x55ef45be6090 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f281d600640 10 client.27617 handle_command_reply: tid=0 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 client.27617 shutdown 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 msgr2=0x55ef45bd3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd3860 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f282009bca0 tx=0x7f2820004a30 comp rx=0 tx=0).stop 2024-09-05T12:48:15.945 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 shutdown_connections 2024-09-05T12:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45be6090 0x55ef45be8480 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bd3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f281d600640 0 client.27617 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:15.946 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcbb00 0x55ef45bd3860 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.947 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 --2- 172.21.15.71:0/3544517270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 >> 172.21.15.71:0/3544517270 conn(0x55ef45bc7750 msgr2=0x55ef45bcd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 shutdown_connections 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 1 -- 172.21.15.71:0/3544517270 wait complete. 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.028+0000 7f283d887740 20 client.27617 trim_cache size 0 max 16384 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.043+0000 7f283d887740 1 Processor -- start 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.043+0000 7f283d887740 1 -- start start 2024-09-05T12:48:15.948 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f283d887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f283d887740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bedd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f283d887740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bee2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f283d887740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28341b10f0 con 0x55ef45bcbb00 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f283d887740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28341b2c30 con 0x55ef45bcb1e0 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f283d887740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28341b17c0 con 0x55ef45bd63a0 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.949 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bedd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39558/0 (socket says 172.21.15.71:39558) 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 -- 172.21.15.71:0/79852683 learned_addr learned my addr 172.21.15.71:0/79852683 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833e00640 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2838a00640 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bee2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 -- 172.21.15.71:0/79852683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 msgr2=0x55ef45bee2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bee2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 -- 172.21.15.71:0/79852683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcb1e0 msgr2=0x55ef45bd5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.950 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2833400640 1 -- 172.21.15.71:0/79852683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef45be8b00 con 0x55ef45bcbb00 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.044+0000 7f2838a00640 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f2833400640 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bedd90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f282009ba10 tx=0x7f2820004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e88270d7f8a59d server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f281fe00640 1 -- 172.21.15.71:0/79852683 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282009b850 con 0x55ef45bcbb00 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f281fe00640 1 -- 172.21.15.71:0/79852683 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282009c410 con 0x55ef45bcbb00 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f281fe00640 1 -- 172.21.15.71:0/79852683 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282009bae0 con 0x55ef45bcbb00 2024-09-05T12:48:15.951 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 -- 172.21.15.71:0/79852683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 msgr2=0x55ef45bedd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bedd90 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f282009ba10 tx=0x7f2820004aa0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 -- 172.21.15.71:0/79852683 shutdown_connections 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bd63a0 0x55ef45bee2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bedd90 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 --2- 172.21.15.71:0/79852683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bcb1e0 0x55ef45bd5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.045+0000 7f283d887740 1 -- 172.21.15.71:0/79852683 >> 172.21.15.71:0/79852683 conn(0x55ef45bc7750 msgr2=0x55ef45bd4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:15.952 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 -- 172.21.15.71:0/79852683 shutdown_connections 2024-09-05T12:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 -- 172.21.15.71:0/79852683 wait complete. 2024-09-05T12:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 -- start start 2024-09-05T12:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 Processor -- start 2024-09-05T12:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bda110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bda650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.953 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bd63a0 0x55ef45bdab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28341b2560 con 0x55ef45bcbb00 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28341b1e90 con 0x55ef45bd63a0 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28341af0f0 con 0x55ef45bcb1e0 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.046+0000 7f283d887740 10 client.0 _ll_register_callbacks cb 0x7f28200d33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bda650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bda650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39572/0 (socket says 172.21.15.71:39572) 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 -- 172.21.15.71:0/1000523273 learned_addr learned my addr 172.21.15.71:0/1000523273 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:15.954 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2838a00640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bd63a0 0x55ef45bdab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833e00640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bda110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 -- 172.21.15.71:0/1000523273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 msgr2=0x55ef45bda110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bda110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 -- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bd63a0 msgr2=0x55ef45bdab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bd63a0 0x55ef45bdab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 -- 172.21.15.71:0/1000523273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef45bc8800 con 0x55ef45bcbb00 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f2833400640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bda650 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f28200986e0 tx=0x7f282009b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b4a7550fda37fa server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:15.955 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f281d600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f281d600640 1 -- 172.21.15.71:0/1000523273 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282009bae0 con 0x55ef45bcbb00 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f281d600640 1 -- 172.21.15.71:0/1000523273 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9438+0+0 (secure 0 0 0) 0x7f282007ad30 con 0x55ef45bcbb00 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.047+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f282009c410 con 0x55ef45bcbb00 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f281d600640 1 -- 172.21.15.71:0/1000523273 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f28200a9050 con 0x55ef45bcbb00 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 10 client.17613 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ef45c50f10 con 0x55ef45bcbb00 2024-09-05T12:48:15.956 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f281d600640 1 -- 172.21.15.71:0/1000523273 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f282009c410 con 0x55ef45bcbb00 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f281d600640 1 -- 172.21.15.71:0/1000523273 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f282009abd0 con 0x55ef45bcbb00 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 10 client.17613 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 10 client.17613 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 20 client.17613 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45c512f0 0x55ef45c53710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 4 client.17613 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:48:15.957 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.048+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef45b38b80 con 0x55ef45c512f0 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.049+0000 7f2833e00640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45c512f0 0x55ef45c53710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.049+0000 7f281d600640 10 client.17613 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.049+0000 7f2833e00640 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45c512f0 0x55ef45c53710 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8842b0b983b9e36 server_cookie=46d55ea5a91d5b98 in_seq=0 out_seq=0 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.243+0000 7f281d600640 1 -- 172.21.15.71:0/1000523273 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef45b38b80 con 0x55ef45c512f0 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.243+0000 7f281d600640 10 client.17613 handle_command_reply: tid=0 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.243+0000 7f283d887740 1 client.17613 shutdown 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.243+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 msgr2=0x55ef45bda650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.958 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.243+0000 7f283d887740 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bda650 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f28200986e0 tx=0x7f282009b180 comp rx=0 tx=0).stop 2024-09-05T12:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:48:15.959 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:48:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 shutdown_connections 2024-09-05T12:48:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ef45c512f0 0x55ef45c53710 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef45bd63a0 0x55ef45bdab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ef45bcbb00 0x55ef45bda650 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 --2- 172.21.15.71:0/1000523273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef45bcb1e0 0x55ef45bda110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 >> 172.21.15.71:0/1000523273 conn(0x55ef45bc7750 msgr2=0x55ef45bcd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f281d600640 0 client.17613 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 shutdown_connections 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 1 -- 172.21.15.71:0/1000523273 wait complete. 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.244+0000 7f283d887740 20 client.17613 trim_cache size 0 max 16384 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.252+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f28180a2da0 msgr2=0x7f28180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.252+0000 7f283aa00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f28180a2da0 0x7f28180a5260 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2824000c00 tx=0x7f2824005b50 comp rx=0 tx=0).stop 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 msgr2=0x7f283419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 0x7f283419aaa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f282000e9c0 tx=0x7f282000ee80 comp rx=0 tx=0).stop 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 shutdown_connections 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f283419ee20 0x7f283419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f28180a2da0 0x7f28180a5260 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2834109080 0x7f283419aaa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 --2- 172.21.15.71:0/114364787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2834103f30 0x7f283419a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.253+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 >> 172.21.15.71:0/114364787 conn(0x7f28340ff980 msgr2=0x7f2834100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.254+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 shutdown_connections 2024-09-05T12:48:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:14.254+0000 7f283aa00640 1 -- 172.21.15.71:0/114364787 wait complete. 2024-09-05T12:48:15.965 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:14 smithi071 ceph-mon[29845]: pgmap v649: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.0 MiB/s wr, 471 op/s 2024-09-05T12:48:16.466 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:16.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:16 smithi071 ceph-mon[29845]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 467 op/s 2024-09-05T12:48:16.592 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:16 smithi071 ceph-mon[29845]: osdmap e128: 12 total, 12 up, 12 in 2024-09-05T12:48:16.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:16 smithi123 ceph-mon[34321]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 467 op/s 2024-09-05T12:48:16.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:16 smithi123 ceph-mon[34321]: osdmap e128: 12 total, 12 up, 12 in 2024-09-05T12:48:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:16 smithi174 ceph-mon[34313]: pgmap v650: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.0 MiB/s wr, 467 op/s 2024-09-05T12:48:16.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:16 smithi174 ceph-mon[34313]: osdmap e128: 12 total, 12 up, 12 in 2024-09-05T12:48:18.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.028+0000 7f4030e00640 1 -- 172.21.15.71:0/1974045240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 msgr2=0x7f402c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:18.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.028+0000 7f4030e00640 1 --2- 172.21.15.71:0/1974045240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 0x7f402c107da0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f401c009930 tx=0x7f401c02a460 comp rx=0 tx=0).stop 2024-09-05T12:48:18.242 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/1974045240 shutdown_connections 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 --2- 172.21.15.71:0/1974045240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c102360 0x7f402c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 --2- 172.21.15.71:0/1974045240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c1019c0 0x7f402c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 --2- 172.21.15.71:0/1974045240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 0x7f402c107da0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/1974045240 >> 172.21.15.71:0/1974045240 conn(0x7f402c0fd740 msgr2=0x7f402c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/1974045240 shutdown_connections 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/1974045240 wait complete. 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 Processor -- start 2024-09-05T12:48:18.243 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- start start 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c1019c0 0x7f402c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 0x7f402c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 0x7f402c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f402c111e00 con 0x7f402c1079c0 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f402c111c80 con 0x7f402c102360 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f402c111f80 con 0x7f402c1019c0 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 0x7f402c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:18.244 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c1019c0 0x7f402c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 0x7f402c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:45248/0 (socket says 172.21.15.71:45248) 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 -- 172.21.15.71:0/137739100 learned_addr learned my addr 172.21.15.71:0/137739100 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402be00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 0x7f402c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 -- 172.21.15.71:0/137739100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c1019c0 msgr2=0x7f402c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c1019c0 0x7f402c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 -- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 msgr2=0x7f402c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:18.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 0x7f402c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 -- 172.21.15.71:0/137739100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402c19f520 con 0x7f402c102360 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f402aa00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 0x7f402c19aab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f401800e9f0 tx=0x7f401800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401800cd80 con 0x7f402c102360 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402c19f810 con 0x7f402c102360 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4018010910 con 0x7f402c102360 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40180188f0 con 0x7f402c102360 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f402c1a70f0 con 0x7f402c102360 2024-09-05T12:48:18.246 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.029+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f402c101e20 con 0x7f402c102360 2024-09-05T12:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.035+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f4018010ac0 con 0x7f402c102360 2024-09-05T12:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.035+0000 7f4023e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40100a2cf0 0x7f40100a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.035+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f40180d0090 con 0x7f402c102360 2024-09-05T12:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.035+0000 7f402b400640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40100a2cf0 0x7f40100a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.035+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f401809c290 con 0x7f402c102360 2024-09-05T12:48:18.247 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.035+0000 7f402b400640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40100a2cf0 0x7f40100a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f401c009900 tx=0x7f401c0058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:18.258 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.257+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f402c1a7420 con 0x7f402c102360 2024-09-05T12:48:18.259 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.257+0000 7f4023e00640 1 -- 172.21.15.71:0/137739100 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7f40180bae20 con 0x7f402c102360 2024-09-05T12:48:18.259 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:48:18.260 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:48:18.260 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40100a2cf0 msgr2=0x7f40100a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40100a2cf0 0x7f40100a51b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f401c009900 tx=0x7f401c0058d0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 msgr2=0x7f402c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 0x7f402c19aab0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f401800e9f0 tx=0x7f401800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 shutdown_connections 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f40100a2cf0 0x7f40100a51b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.262 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f402c1079c0 0x7f402c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f402c102360 0x7f402c19aab0 secure :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f401800e9f0 tx=0x7f401800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 --2- 172.21.15.71:0/137739100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f402c1019c0 0x7f402c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:18.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 >> 172.21.15.71:0/137739100 conn(0x7f402c0fd740 msgr2=0x7f402c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:18.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 shutdown_connections 2024-09-05T12:48:18.263 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:18.258+0000 7f4030e00640 1 -- 172.21.15.71:0/137739100 wait complete. 2024-09-05T12:48:18.538 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:18 smithi071 ceph-mon[29845]: pgmap v652: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.8 MiB/s wr, 444 op/s 2024-09-05T12:48:18.538 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:18 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/137739100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:48:18.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:18 smithi123 ceph-mon[34321]: pgmap v652: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.8 MiB/s wr, 444 op/s 2024-09-05T12:48:18.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:18 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/137739100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:48:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:18 smithi174 ceph-mon[34313]: pgmap v652: 129 pgs: 129 active+clean; 1.2 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 7.8 MiB/s wr, 444 op/s 2024-09-05T12:48:18.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:18 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/137739100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:48:20.471 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:20 smithi071 ceph-mon[29845]: pgmap v653: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.6 MiB/s wr, 454 op/s 2024-09-05T12:48:20.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:20 smithi174 ceph-mon[34313]: pgmap v653: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.6 MiB/s wr, 454 op/s 2024-09-05T12:48:20.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:20 smithi123 ceph-mon[34321]: pgmap v653: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.6 MiB/s wr, 454 op/s 2024-09-05T12:48:21.834 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --cancel --all' 2024-09-05T12:48:21.834 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- timeout 105 ceph tell mds.24524 quiesce db --cancel --all 2024-09-05T12:48:22.313 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:22.675 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:22 smithi071 ceph-mon[29845]: pgmap v654: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 308 op/s 2024-09-05T12:48:22.731 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:22 smithi123 ceph-mon[34321]: pgmap v654: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 308 op/s 2024-09-05T12:48:22.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:22 smithi174 ceph-mon[34313]: pgmap v654: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.9 MiB/s wr, 308 op/s 2024-09-05T12:48:23.175 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-05T12:48:23.175 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.45 2024-09-05T12:48:23.176 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:48:24.729 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:24 smithi071 ceph-mon[29845]: pgmap v655: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 9.1 MiB/s wr, 300 op/s 2024-09-05T12:48:24.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:24 smithi174 ceph-mon[34313]: pgmap v655: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 9.1 MiB/s wr, 300 op/s 2024-09-05T12:48:24.857 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/1879703663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f4104010 msgr2=0x7ff8f41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- 172.21.15.71:0/1879703663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f4104010 0x7ff8f41043f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009900 tx=0x7ff8e802a410 comp rx=0 tx=0).stop 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/1879703663 shutdown_connections 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- 172.21.15.71:0/1879703663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f4109140 0x7ff8f4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- 172.21.15.71:0/1879703663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f41049c0 0x7ff8f4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- 172.21.15.71:0/1879703663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f4104010 0x7ff8f41043f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/1879703663 >> 172.21.15.71:0/1879703663 conn(0x7ff8f40ffa50 msgr2=0x7ff8f4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:24.858 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/1879703663 shutdown_connections 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/1879703663 wait complete. 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 Processor -- start 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- start start 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f4104010 0x7ff8f419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f41049c0 0x7ff8f419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f4109140 0x7ff8f419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41141c0 con 0x7ff8f41049c0 2024-09-05T12:48:24.859 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4114040 con 0x7ff8f4104010 2024-09-05T12:48:24.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4114340 con 0x7ff8f4109140 2024-09-05T12:48:24.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f4104010 0x7ff8f419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f41049c0 0x7ff8f419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f4104010 0x7ff8f419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58540/0 (socket says 172.21.15.71:58540) 2024-09-05T12:48:24.860 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9c00640 1 -- 172.21.15.71:0/3533648259 learned_addr learned my addr 172.21.15.71:0/3533648259 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fa600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f4109140 0x7ff8f419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 -- 172.21.15.71:0/3533648259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f4109140 msgr2=0x7ff8f419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f4109140 0x7ff8f419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 -- 172.21.15.71:0/3533648259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f4104010 msgr2=0x7ff8f419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f4104010 0x7ff8f419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 -- 172.21.15.71:0/3533648259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f419f680 con 0x7ff8f41049c0 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f9200640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f41049c0 0x7ff8f419ac10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9f0 tx=0x7ff8e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:24.861 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f2000640 1 -- 172.21.15.71:0/3533648259 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cd80 con 0x7ff8f41049c0 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f2000640 1 -- 172.21.15.71:0/3533648259 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e00035f0 con 0x7ff8f41049c0 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f419f970 con 0x7ff8f41049c0 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8f2000640 1 -- 172.21.15.71:0/3533648259 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0010690 con 0x7ff8f41049c0 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.810+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a7250 con 0x7ff8f41049c0 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 Processor -- start 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 -- start start 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad55ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad55ff0 0x55cbbad60370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.862 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8f2000640 1 -- 172.21.15.71:0/3533648259 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7ff8e0003b30 con 0x7ff8f41049c0 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad62ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cbbacbc340 con 0x55cbbad556d0 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cbbac63bf0 con 0x55cbbad608b0 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8ff072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cbbacb8300 con 0x55cbbad55ff0 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad55ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad55ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43866/0 (socket says 172.21.15.71:43866) 2024-09-05T12:48:24.863 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 -- 172.21.15.71:0/542058015 learned_addr learned my addr 172.21.15.71:0/542058015 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8f9200640 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad62ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 -- 172.21.15.71:0/542058015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad55ff0 msgr2=0x55cbbad60370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad55ff0 0x55cbbad60370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 -- 172.21.15.71:0/542058015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 msgr2=0x55cbbad62ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad62ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 -- 172.21.15.71:0/542058015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbaa2d620 con 0x55cbbad556d0 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8fa600640 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad55ab0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009f50 tx=0x7ff8ec0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf3d1e3c2e4d4db server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:24.864 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.816+0000 7ff8e6a00640 1 -- 172.21.15.71:0/542058015 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec004d20 con 0x55cbbad556d0 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8e6a00640 1 -- 172.21.15.71:0/542058015 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ec004050 con 0x55cbbad556d0 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8e6a00640 1 -- 172.21.15.71:0/542058015 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec005220 con 0x55cbbad556d0 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8f2000640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff8dc0a2cf0 0x7ff8dc0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8f2000640 1 -- 172.21.15.71:0/3533648259 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7ff8e00cfbb0 con 0x7ff8f41049c0 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff8dc0a2cf0 0x7ff8dc0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8ff072740 1 -- 172.21.15.71:0/542058015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 msgr2=0x55cbbad55ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.821+0000 7ff8ff072740 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad55ab0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009f50 tx=0x7ff8ec0118d0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/542058015 shutdown_connections 2024-09-05T12:48:24.865 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad62ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad55ff0 0x55cbbad60370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/542058015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad55ab0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/542058015 >> 172.21.15.71:0/542058015 conn(0x55cbbad51c60 msgr2=0x55cbbad52040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/542058015 shutdown_connections 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/542058015 wait complete. 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff8dc0a2cf0 0x7ff8dc0a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005ab0 tx=0x7ff8e8005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- start start 2024-09-05T12:48:24.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 Processor -- start 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad556d0 0x55cbbad5d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 0x55cbbad5dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad608b0 0x55cbbad5e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55cbbad63480 con 0x55cbbad55ff0 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cbbac77060 con 0x55cbbad556d0 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cbbad63600 con 0x55cbbad608b0 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 10 client.0 _ll_register_callbacks cb 0x55cbbac65af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 0x55cbbad5dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad556d0 0x55cbbad5d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 0x55cbbad5dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43878/0 (socket says 172.21.15.71:43878) 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 -- 172.21.15.71:0/3352790006 learned_addr learned my addr 172.21.15.71:0/3352790006 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8fa600640 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad608b0 0x55cbbad5e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 -- 172.21.15.71:0/3352790006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad608b0 msgr2=0x55cbbad5e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad608b0 0x55cbbad5e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 -- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad556d0 msgr2=0x55cbbad5d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad556d0 0x55cbbad5d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 -- 172.21.15.71:0/3352790006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbad6c950 con 0x55cbbad55ff0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9200640 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 0x55cbbad5dc00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff8e001f000 tx=0x7ff8e009b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97f187b88470140 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 1 -- 172.21.15.71:0/3352790006 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e009bd00 con 0x55cbbad55ff0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 1 -- 172.21.15.71:0/3352790006 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7ff8e00cd1b0 con 0x55cbbad55ff0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8ec004050 con 0x55cbbad55ff0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 1 -- 172.21.15.71:0/3352790006 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff8e00b9050 con 0x55cbbad55ff0 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 10 client.17631 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:24.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55cbbad6f2d0 con 0x55cbbad55ff0 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 1 -- 172.21.15.71:0/3352790006 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8ec004050 con 0x55cbbad55ff0 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 1 -- 172.21.15.71:0/3352790006 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7ff8e009b150 con 0x55cbbad55ff0 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 10 client.17631 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 10 client.17631 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 20 client.17631 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbad70790 0x55cbbad72ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 4 client.17631 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:48:24.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbbacc98b0 con 0x55cbbad70790 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbad70790 0x55cbbad72ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbad70790 0x55cbbad72ba0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e39f54be546f748a server_cookie=d5b7f1a6e7d7bafc in_seq=0 out_seq=0 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 10 client.17631 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 1 -- 172.21.15.71:0/3352790006 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbbacc98b0 con 0x55cbbad70790 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 10 client.17631 handle_command_reply: tid=0 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 client.17631 shutdown 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 msgr2=0x55cbbad5dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.871 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 0x55cbbad5dc00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff8e001f000 tx=0x7ff8e009b840 comp rx=0 tx=0).stop 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 shutdown_connections 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbad70790 0x55cbbad72ba0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad608b0 0x55cbbad5e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad55ff0 0x55cbbad5dc00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 --2- 172.21.15.71:0/3352790006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad556d0 0x55cbbad5d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8db400640 0 client.17631 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 >> 172.21.15.71:0/3352790006 conn(0x55cbbad51c60 msgr2=0x55cbbad57940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:24.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.822+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 shutdown_connections 2024-09-05T12:48:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.828+0000 7ff8ff072740 1 -- 172.21.15.71:0/3352790006 wait complete. 2024-09-05T12:48:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.828+0000 7ff8ff072740 20 client.17631 trim_cache size 0 max 16384 2024-09-05T12:48:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.843+0000 7ff8ff072740 1 Processor -- start 2024-09-05T12:48:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- start start 2024-09-05T12:48:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad5cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.873 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad5c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad788d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b1150 con 0x55cbbad556d0 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b2c90 con 0x55cbbad608b0 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b1820 con 0x55cbbad78470 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad5cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad5cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43888/0 (socket says 172.21.15.71:43888) 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1007387635 learned_addr learned my addr 172.21.15.71:0/1007387635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:24.874 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8fa600640 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad788d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1007387635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 msgr2=0x55cbbad788d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad788d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1007387635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 msgr2=0x55cbbad5c6a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad5c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1007387635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbbad55e20 con 0x55cbbad556d0 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad5cf80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000c00 tx=0x7ff8e803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9314bec9a1a2a528 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:24.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8e6000640 1 -- 172.21.15.71:0/1007387635 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8035050 con 0x55cbbad556d0 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8e6000640 1 -- 172.21.15.71:0/1007387635 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8056070 con 0x55cbbad556d0 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8e6000640 1 -- 172.21.15.71:0/1007387635 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8004e50 con 0x55cbbad556d0 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1007387635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 msgr2=0x55cbbad5cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad5cf80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8000c00 tx=0x7ff8e803c000 comp rx=0 tx=0).stop 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1007387635 shutdown_connections 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad788d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad5c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1007387635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad5cf80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1007387635 >> 172.21.15.71:0/1007387635 conn(0x55cbbad51c60 msgr2=0x55cbbad5bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1007387635 shutdown_connections 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1007387635 wait complete. 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- start start 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 Processor -- start 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad68f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad65290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b25c0 con 0x55cbbad556d0 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41b1ef0 con 0x55cbbad608b0 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41af150 con 0x55cbbad78470 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 10 client.0 _ll_register_callbacks cb 0x7ff8e00d3350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad68f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8fa600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad68f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43894/0 (socket says 172.21.15.71:43894) 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1425857205 learned_addr learned my addr 172.21.15.71:0/1425857205 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9200640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad65290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1425857205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 msgr2=0x55cbbad657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 msgr2=0x55cbbad65290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad65290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 -- 172.21.15.71:0/1425857205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbbad66000 con 0x55cbbad556d0 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9200640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad65290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:24.879 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8f9c00640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad68f50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8e802a450 tx=0x7ff8e8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b20c4275bdecae server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8db400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8db400640 1 -- 172.21.15.71:0/1425857205 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e805f060 con 0x55cbbad556d0 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8db400640 1 -- 172.21.15.71:0/1425857205 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7ff8e80046c0 con 0x55cbbad556d0 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8e8056070 con 0x55cbbad556d0 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8db400640 1 -- 172.21.15.71:0/1425857205 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff8e8035030 con 0x55cbbad556d0 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 10 client.17637 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:24.880 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55cbbaddc3a0 con 0x55cbbad556d0 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.846+0000 7ff8db400640 1 -- 172.21.15.71:0/1425857205 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e8056070 con 0x55cbbad556d0 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8db400640 1 -- 172.21.15.71:0/1425857205 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7ff8e8003b80 con 0x55cbbad556d0 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8ff072740 10 client.17637 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8ff072740 10 client.17637 resolve_mds: validated gid 24524 aka daemon mds.b 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8ff072740 20 client.17637 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbaddcaa0 0x55cbbaddeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8ff072740 4 client.17637 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-05T12:48:24.881 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.851+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55cbbacc3dc0 con 0x55cbbaddcaa0 2024-09-05T12:48:24.882 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.869+0000 7ff8f9200640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbaddcaa0 0x55cbbaddeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:24.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.928+0000 7ff8db400640 10 client.17637 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:24.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.928+0000 7ff8f9200640 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbaddcaa0 0x55cbbaddeec0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5041979fc38e0d2c server_cookie=22bc0da245567dc1 in_seq=0 out_seq=0 2024-09-05T12:48:24.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:24 smithi123 ceph-mon[34321]: pgmap v655: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 9.1 MiB/s wr, 300 op/s 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8db400640 1 -- 172.21.15.71:0/1425857205 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55cbbacc3dc0 con 0x55cbbaddcaa0 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8db400640 10 client.17637 handle_command_reply: tid=0 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 client.17637 shutdown 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 msgr2=0x55cbbad68f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad68f50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8e802a450 tx=0x7ff8e8005040 comp rx=0 tx=0).stop 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 shutdown_connections 2024-09-05T12:48:24.942 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55cbbaddcaa0 0x55cbbaddeec0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbbad78470 0x55cbbad657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbbad608b0 0x55cbbad65290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8db400640 0 client.17637 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 --2- 172.21.15.71:0/1425857205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cbbad556d0 0x55cbbad68f50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 >> 172.21.15.71:0/1425857205 conn(0x55cbbad51c60 msgr2=0x55cbbad5bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:24.943 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 shutdown_connections 2024-09-05T12:48:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 1 -- 172.21.15.71:0/1425857205 wait complete. 2024-09-05T12:48:24.944 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8ff072740 20 client.17637 trim_cache size 0 max 16384 2024-09-05T12:48:24.944 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:48:24.944 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 76, 2024-09-05T12:48:24.945 INFO:teuthology.orchestra.run.smithi071.stdout: "leader": 24524, 2024-09-05T12:48:24.945 INFO:teuthology.orchestra.run.smithi071.stdout: "set_version": 64, 2024-09-05T12:48:24.945 INFO:teuthology.orchestra.run.smithi071.stdout: "sets": {} 2024-09-05T12:48:24.945 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff8dc0a2cf0 msgr2=0x7ff8dc0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff8dc0a2cf0 0x7ff8dc0a51b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005ab0 tx=0x7ff8e8005b50 comp rx=0 tx=0).stop 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f41049c0 msgr2=0x7ff8f419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f41049c0 0x7ff8f419ac10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000e9f0 tx=0x7ff8e000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 shutdown_connections 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff8f4109140 0x7ff8f419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7ff8dc0a2cf0 0x7ff8dc0a51b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.946 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f41049c0 0x7ff8f419ac10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 --2- 172.21.15.71:0/3533648259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8f4104010 0x7ff8f419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:24.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 >> 172.21.15.71:0/3533648259 conn(0x7ff8f40ffa50 msgr2=0x7ff8f4100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:24.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 shutdown_connections 2024-09-05T12:48:24.947 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:24.941+0000 7ff8fc600640 1 -- 172.21.15.71:0/3533648259 wait complete. 2024-09-05T12:48:25.301 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:26.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1966152477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 msgr2=0x7fa484110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:26.487 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- 172.21.15.71:0/1966152477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa484110cc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009ca0 tx=0x7fa47c02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1966152477 shutdown_connections 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- 172.21.15.71:0/1966152477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa484110cc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- 172.21.15.71:0/1966152477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4841049c0 0x7fa484108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- 172.21.15.71:0/1966152477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa484104010 0x7fa4841043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1966152477 >> 172.21.15.71:0/1966152477 conn(0x7fa4840ffa50 msgr2=0x7fa484101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1966152477 shutdown_connections 2024-09-05T12:48:26.488 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1966152477 wait complete. 2024-09-05T12:48:26.489 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 Processor -- start 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- start start 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa484104010 0x7fa48419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4841049c0 0x7fa48419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa48419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841141c0 con 0x7fa484104010 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114040 con 0x7fa4841049c0 2024-09-05T12:48:26.490 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114340 con 0x7fa484109140 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa482a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4841049c0 0x7fa48419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa482a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4841049c0 0x7fa48419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58610/0 (socket says 172.21.15.71:58610) 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa482a00640 1 -- 172.21.15.71:0/1243200621 learned_addr learned my addr 172.21.15.71:0/1243200621 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa48419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483400640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa484104010 0x7fa48419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 -- 172.21.15.71:0/1243200621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4841049c0 msgr2=0x7fa48419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4841049c0 0x7fa48419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.491 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 -- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa484104010 msgr2=0x7fa48419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:26.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa484104010 0x7fa48419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 -- 172.21.15.71:0/1243200621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419f610 con 0x7fa484109140 2024-09-05T12:48:26.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa483e00640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa48419ef30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005e30 tx=0x7fa47c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:26.492 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c031650 con 0x7fa484109140 2024-09-05T12:48:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48419f8a0 con 0x7fa484109140 2024-09-05T12:48:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47c0317f0 con 0x7fa484109140 2024-09-05T12:48:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c02fa30 con 0x7fa484109140 2024-09-05T12:48:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a71e0 con 0x7fa484109140 2024-09-05T12:48:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.485+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4841043f0 con 0x7fa484109140 2024-09-05T12:48:26.493 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.490+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa47c02fbd0 con 0x7fa484109140 2024-09-05T12:48:26.494 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.492+0000 7fa47be00640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4680a2cf0 0x7fa4680a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:26.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.492+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7fa47c0f0a50 con 0x7fa484109140 2024-09-05T12:48:26.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.492+0000 7fa483400640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4680a2cf0 0x7fa4680a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:26.495 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.492+0000 7fa483400640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4680a2cf0 0x7fa4680a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa474005fd0 tx=0x7fa47400b390 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:26.502 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.498+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa47c0319a0 con 0x7fa484109140 2024-09-05T12:48:26.731 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:26 smithi071 ceph-mon[29845]: pgmap v656: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 9.1 MiB/s wr, 300 op/s 2024-09-05T12:48:26.731 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:26 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:48:26.731 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:26 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:26.732 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:26 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:26.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:26 smithi174 ceph-mon[34313]: pgmap v656: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 9.1 MiB/s wr, 300 op/s 2024-09-05T12:48:26.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:26 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:48:26.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:26 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:26.735 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:26 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:26.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.750+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa48410c3e0 con 0x7fa484109140 2024-09-05T12:48:26.753 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:48:26.753 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:48:26.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.750+0000 7fa47be00640 1 -- 172.21.15.71:0/1243200621 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7fa47c0bd3b0 con 0x7fa484109140 2024-09-05T12:48:26.754 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:48:26.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4680a2cf0 msgr2=0x7fa4680a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:26.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4680a2cf0 0x7fa4680a51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa474005fd0 tx=0x7fa47400b390 comp rx=0 tx=0).stop 2024-09-05T12:48:26.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 msgr2=0x7fa48419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa48419ef30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005e30 tx=0x7fa47c003f10 comp rx=0 tx=0).stop 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 shutdown_connections 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484109140 0x7fa48419ef30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4841049c0 0x7fa48419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4680a2cf0 0x7fa4680a51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 --2- 172.21.15.71:0/1243200621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa484104010 0x7fa48419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 >> 172.21.15.71:0/1243200621 conn(0x7fa4840ffa50 msgr2=0x7fa484100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:26.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 shutdown_connections 2024-09-05T12:48:26.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:26.753+0000 7fa48a000640 1 -- 172.21.15.71:0/1243200621 wait complete. 2024-09-05T12:48:26.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:26 smithi123 ceph-mon[34321]: pgmap v656: 129 pgs: 129 active+clean; 1.2 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 717 KiB/s rd, 9.1 MiB/s wr, 300 op/s 2024-09-05T12:48:26.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:26 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-05T12:48:26.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:26 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:26.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:26 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/3036570603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b41048e0 msgr2=0x7f84b4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- 172.21.15.71:0/3036570603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b41048e0 0x7f84b4108920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84a0009900 tx=0x7f84a002a410 comp rx=0 tx=0).stop 2024-09-05T12:48:27.392 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/3036570603 shutdown_connections 2024-09-05T12:48:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- 172.21.15.71:0/3036570603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b4109080 0x7f84b4110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- 172.21.15.71:0/3036570603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b41048e0 0x7f84b4108920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- 172.21.15.71:0/3036570603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b4103f30 0x7f84b4104310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/3036570603 >> 172.21.15.71:0/3036570603 conn(0x7f84b40ff980 msgr2=0x7f84b4101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/3036570603 shutdown_connections 2024-09-05T12:48:27.393 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/3036570603 wait complete. 2024-09-05T12:48:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 Processor -- start 2024-09-05T12:48:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- start start 2024-09-05T12:48:27.395 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b4103f30 0x7f84b419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4109080 0x7f84b419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b419ee50 0x7f84b419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4114080 con 0x7f84b419ee50 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4113f00 con 0x7f84b4109080 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84b4114200 con 0x7f84b4103f30 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b419ee50 0x7f84b419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.396 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4109080 0x7f84b419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4109080 0x7f84b419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58626/0 (socket says 172.21.15.71:58626) 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b3e00640 1 -- 172.21.15.71:0/2725815644 learned_addr learned my addr 172.21.15.71:0/2725815644 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 -- 172.21.15.71:0/2725815644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b4103f30 msgr2=0x7f84b419a590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b4103f30 0x7f84b419a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 -- 172.21.15.71:0/2725815644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4109080 msgr2=0x7f84b419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4109080 0x7f84b419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 -- 172.21.15.71:0/2725815644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84b419f940 con 0x7f84b419ee50 2024-09-05T12:48:27.397 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b9600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b419ee50 0x7f84b419f300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009fd0 tx=0x7f84ac00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b1600640 1 -- 172.21.15.71:0/2725815644 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac00ce20 con 0x7f84b419ee50 2024-09-05T12:48:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b1600640 1 -- 172.21.15.71:0/2725815644 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84ac005020 con 0x7f84b419ee50 2024-09-05T12:48:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84b419fc30 con 0x7f84b419ee50 2024-09-05T12:48:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84b1600640 1 -- 172.21.15.71:0/2725815644 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac010720 con 0x7f84b419ee50 2024-09-05T12:48:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.389+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84b41a73d0 con 0x7f84b419ee50 2024-09-05T12:48:27.398 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 Processor -- start 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b1600640 1 -- 172.21.15.71:0/2725815644 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f84ac005540 con 0x7f84b419ee50 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- start start 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da61b00 0x55ee0da6be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da6c3a0 0x55ee0da6e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ee0d9c7100 con 0x55ee0da611e0 2024-09-05T12:48:27.400 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee0d96f1b0 con 0x55ee0da6c3a0 2024-09-05T12:48:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee0d9c30c0 con 0x55ee0da61b00 2024-09-05T12:48:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43936/0 (socket says 172.21.15.71:43936) 2024-09-05T12:48:27.401 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 -- 172.21.15.71:0/967087320 learned_addr learned my addr 172.21.15.71:0/967087320 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da6c3a0 0x55ee0da6e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 -- 172.21.15.71:0/967087320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da61b00 msgr2=0x55ee0da6be60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-05T12:48:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da61b00 0x55ee0da6be60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 -- 172.21.15.71:0/967087320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da6c3a0 msgr2=0x55ee0da6e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.405 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da6c3a0 0x55ee0da6e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 -- 172.21.15.71:0/967087320 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0d6da210 con 0x55ee0da611e0 2024-09-05T12:48:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b3e00640 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da615c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f84a0031550 tx=0x7f84a00314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bb66148f69ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b1600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f849c0a2da0 0x7f849c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84a5c00640 1 -- 172.21.15.71:0/967087320 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0035030 con 0x55ee0da611e0 2024-09-05T12:48:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84a5c00640 1 -- 172.21.15.71:0/967087320 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a00035f0 con 0x55ee0da611e0 2024-09-05T12:48:27.406 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84a5c00640 1 -- 172.21.15.71:0/967087320 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a003d650 con 0x55ee0da611e0 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b1600640 1 -- 172.21.15.71:0/2725815644 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f84ac0cfb70 con 0x7f84b419ee50 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b8c00640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f849c0a2da0 0x7f849c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- 172.21.15.71:0/967087320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 msgr2=0x55ee0da615c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da615c0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f84a0031550 tx=0x7f84a00314e0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- 172.21.15.71:0/967087320 shutdown_connections 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da6c3a0 0x55ee0da6e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da61b00 0x55ee0da6be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.407 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- 172.21.15.71:0/967087320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da615c0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- 172.21.15.71:0/967087320 >> 172.21.15.71:0/967087320 conn(0x55ee0da5d750 msgr2=0x55ee0da5db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84b8c00640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f849c0a2da0 0x7f849c0a5260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009a60 tx=0x7f84a8005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- 172.21.15.71:0/967087320 shutdown_connections 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- 172.21.15.71:0/967087320 wait complete. 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- start start 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 Processor -- start 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da611e0 0x55ee0da692f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.408 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 0x55ee0da69830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da69d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ee0da6ef70 con 0x55ee0da61b00 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee0d950090 con 0x55ee0da611e0 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee0da6f0f0 con 0x55ee0da6c3a0 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.396+0000 7f84be436740 10 client.0 _ll_register_callbacks cb 0x55ee0d9a4f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 0x55ee0da69830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b9600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da69d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.409 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 0x55ee0da69830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43944/0 (socket says 172.21.15.71:43944) 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 -- 172.21.15.71:0/2617531964 learned_addr learned my addr 172.21.15.71:0/2617531964 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b8c00640 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da611e0 0x55ee0da692f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 -- 172.21.15.71:0/2617531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 msgr2=0x55ee0da69d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da69d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 -- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da611e0 msgr2=0x55ee0da692f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da611e0 0x55ee0da692f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 -- 172.21.15.71:0/2617531964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0d7a6620 con 0x55ee0da61b00 2024-09-05T12:48:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84b3e00640 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 0x55ee0da69830 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f84a002a450 tx=0x7f84a00315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4703e81f8725409 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f849a000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f849a000640 1 -- 172.21.15.71:0/2617531964 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0035050 con 0x55ee0da61b00 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a00035f0 con 0x55ee0da61b00 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f849a000640 1 -- 172.21.15.71:0/2617531964 <== mon.0 v2:172.21.15.71:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f84a0003b10 con 0x55ee0da61b00 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f849a000640 1 -- 172.21.15.71:0/2617531964 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f84a0058050 con 0x55ee0da61b00 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84be436740 10 client.17655 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.401+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ee0da7ab70 con 0x55ee0da61b00 2024-09-05T12:48:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f849a000640 1 -- 172.21.15.71:0/2617531964 <== mon.0 v2:172.21.15.71:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a00035f0 con 0x55ee0da61b00 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f849a000640 1 -- 172.21.15.71:0/2617531964 <== mon.0 v2:172.21.15.71:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f84a0005430 con 0x55ee0da61b00 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 10 client.17655 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 10 client.17655 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 20 client.17655 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0da7bec0 0x55ee0da7e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 4 client.17655 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0d9d4830 con 0x55ee0da7bec0 2024-09-05T12:48:27.412 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84b8c00640 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0da7bec0 0x55ee0da7e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84b8c00640 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0da7bec0 0x55ee0da7e2b0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cdde944a6847eae server_cookie=c454f5223a029bb4 in_seq=0 out_seq=0 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f849a000640 10 client.17655 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f849a000640 1 -- 172.21.15.71:0/2617531964 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee0d9d4830 con 0x55ee0da7bec0 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f849a000640 10 client.17655 handle_command_reply: tid=0 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 client.17655 shutdown 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 msgr2=0x55ee0da69830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 0x55ee0da69830 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f84a002a450 tx=0x7f84a00315b0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 shutdown_connections 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0da7bec0 0x55ee0da7e2b0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da69d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da61b00 0x55ee0da69830 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f849a000640 0 client.17655 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 --2- 172.21.15.71:0/2617531964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da611e0 0x55ee0da692f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 >> 172.21.15.71:0/2617531964 conn(0x55ee0da5d750 msgr2=0x55ee0da634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 shutdown_connections 2024-09-05T12:48:27.415 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 1 -- 172.21.15.71:0/2617531964 wait complete. 2024-09-05T12:48:27.416 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.403+0000 7f84be436740 20 client.17655 trim_cache size 0 max 16384 2024-09-05T12:48:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 Processor -- start 2024-09-05T12:48:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- start start 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 0x55ee0da812d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da81810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da6c3a0 0x55ee0da66530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b1050 con 0x55ee0da6c3a0 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b2b90 con 0x55ee0da61b00 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b1720 con 0x55ee0da611e0 2024-09-05T12:48:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 0x55ee0da812d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 0x55ee0da812d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55018/0 (socket says 172.21.15.71:55018) 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 -- 172.21.15.71:0/1556455056 learned_addr learned my addr 172.21.15.71:0/1556455056 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b9600640 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da6c3a0 0x55ee0da66530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b3e00640 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da81810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 -- 172.21.15.71:0/1556455056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 msgr2=0x55ee0da81810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da81810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 -- 172.21.15.71:0/1556455056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da6c3a0 msgr2=0x55ee0da66530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da6c3a0 0x55ee0da66530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 -- 172.21.15.71:0/1556455056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee0da61930 con 0x55ee0da611e0 2024-09-05T12:48:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84b8c00640 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 0x55ee0da812d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55ee0da6f100 tx=0x7f84a8015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca50ebec642197e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:27.430 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84a5200640 1 -- 172.21.15.71:0/1556455056 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8005020 con 0x55ee0da611e0 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84a5200640 1 -- 172.21.15.71:0/1556455056 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a80047a0 con 0x55ee0da611e0 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84a5200640 1 -- 172.21.15.71:0/1556455056 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80255b0 con 0x55ee0da611e0 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- 172.21.15.71:0/1556455056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 msgr2=0x55ee0da812d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 0x55ee0da812d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55ee0da6f100 tx=0x7f84a8015000 comp rx=0 tx=0).stop 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- 172.21.15.71:0/1556455056 shutdown_connections 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da6c3a0 0x55ee0da66530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da81810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.431 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 --2- 172.21.15.71:0/1556455056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da611e0 0x55ee0da812d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- 172.21.15.71:0/1556455056 >> 172.21.15.71:0/1556455056 conn(0x55ee0da5d750 msgr2=0x55ee0da77b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- 172.21.15.71:0/1556455056 shutdown_connections 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.420+0000 7f84be436740 1 -- 172.21.15.71:0/1556455056 wait complete. 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 -- start start 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 Processor -- start 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da81bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.432 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da7ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da7f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b24c0 con 0x55ee0da611e0 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41b1df0 con 0x55ee0da61b00 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84b41af050 con 0x55ee0da6c3a0 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84be436740 10 client.0 _ll_register_callbacks cb 0x7f84a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84b8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da81bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84b9600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da7f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.433 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84b9600640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da7f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55020/0 (socket says 172.21.15.71:55020) 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84b9600640 1 -- 172.21.15.71:0/1057622758 learned_addr learned my addr 172.21.15.71:0/1057622758 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.425+0000 7f84b3e00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da7ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b3e00640 1 -- 172.21.15.71:0/1057622758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 msgr2=0x55ee0da7f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b3e00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da7f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b3e00640 1 -- 172.21.15.71:0/1057622758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 msgr2=0x55ee0da81bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b3e00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da81bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b3e00640 1 -- 172.21.15.71:0/1057622758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee0dae5800 con 0x55ee0da61b00 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b8c00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da81bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:27.434 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b3e00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da7ef10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000c00 tx=0x7f84a003d6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c822063ac3091a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 1 -- 172.21.15.71:0/1057622758 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a0035040 con 0x55ee0da61b00 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 1 -- 172.21.15.71:0/1057622758 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f84a003dae0 con 0x55ee0da61b00 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a80047a0 con 0x55ee0da61b00 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 1 -- 172.21.15.71:0/1057622758 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f84a0005d50 con 0x55ee0da61b00 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 10 client.27668 fetch_fsmap learned FSMap version 76 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55ee0dae69b0 con 0x55ee0da61b00 2024-09-05T12:48:27.435 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 1 -- 172.21.15.71:0/1057622758 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84a80047a0 con 0x55ee0da61b00 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 1 -- 172.21.15.71:0/1057622758 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 76) ==== 8370+0+0 (secure 0 0 0) 0x7f84a004aad0 con 0x55ee0da61b00 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 10 client.27668 fetch_fsmap finished waiting for FSMap version 76 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 10 client.27668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 20 client.27668 populate_metadata read hostname 'smithi071' 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0dae70d0 0x55ee0dae94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 4 client.27668 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 --> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee0d9ceb80 con 0x55ee0dae70d0 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b8c00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0dae70d0 0x55ee0dae94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f84b8c00640 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0dae70d0 0x55ee0dae94f0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8df57a31267ec252 server_cookie=33f474819c48aa86 in_seq=0 out_seq=0 2024-09-05T12:48:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.426+0000 7f849a000640 10 client.27668 ms_handle_connect on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:27.493 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f849a000640 1 -- 172.21.15.71:0/1057622758 <== mds.0 v2:172.21.15.123:6838/782979893 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee0d9ceb80 con 0x55ee0dae70d0 2024-09-05T12:48:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f849a000640 10 client.27668 handle_command_reply: tid=0 2024-09-05T12:48:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 client.27668 shutdown 2024-09-05T12:48:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 msgr2=0x55ee0da7ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da7ef10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84a0000c00 tx=0x7f84a003d6e0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 shutdown_connections 2024-09-05T12:48:27.494 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:6838/782979893,v1:172.21.15.123:6839/782979893] conn(0x55ee0dae70d0 0x55ee0dae94f0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee0da6c3a0 0x55ee0da7f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee0da61b00 0x55ee0da7ef10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 --2- 172.21.15.71:0/1057622758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ee0da611e0 0x55ee0da81bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f849a000640 0 client.27668 ms_handle_reset on v2:172.21.15.123:6838/782979893 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 >> 172.21.15.71:0/1057622758 conn(0x55ee0da5d750 msgr2=0x55ee0da62c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 shutdown_connections 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 1 -- 172.21.15.71:0/1057622758 wait complete. 2024-09-05T12:48:27.495 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84be436740 20 client.27668 trim_cache size 0 max 16384 2024-09-05T12:48:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:{ 2024-09-05T12:48:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "message": "", 2024-09-05T12:48:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout: "return_code": 0 2024-09-05T12:48:27.496 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stdout:} 2024-09-05T12:48:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f849c0a2da0 msgr2=0x7f849c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f849c0a2da0 0x7f849c0a5260 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009a60 tx=0x7f84a8005f90 comp rx=0 tx=0).stop 2024-09-05T12:48:27.497 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b419ee50 msgr2=0x7f84b419f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b419ee50 0x7f84b419f300 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f84ac009fd0 tx=0x7f84ac00ef10 comp rx=0 tx=0).stop 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 shutdown_connections 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f849c0a2da0 0x7f849c0a5260 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84b419ee50 0x7f84b419f300 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84b4109080 0x7f84b419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 --2- 172.21.15.71:0/2725815644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84b4103f30 0x7f84b419a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 >> 172.21.15.71:0/2725815644 conn(0x7f84b40ff980 msgr2=0x7f84b4100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 shutdown_connections 2024-09-05T12:48:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi071.front.sepia.ceph.com.smithi071.stderr:2024-09-05T12:48:27.491+0000 7f84bb600640 1 -- 172.21.15.71:0/2725815644 wait complete. 2024-09-05T12:48:27.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:27 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1243200621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:48:27.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:27 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.594 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:27 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:27 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1243200621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:48:27.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:27 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:27 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.832 INFO:tasks.quiescer:done joining Quiescers 2024-09-05T12:48:27.833 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-05T12:48:27.843 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:48:27.858 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-05T12:48:27.867 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-05T12:48:27.867 INFO:teuthology.task.exec:Executing custom commands... 2024-09-05T12:48:27.867 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:48:27.867 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-05T12:48:27.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:27 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1243200621' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:48:27.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:27 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:27 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:27.960 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-05T12:48:28.020 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-05T12:48:28.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 -- 172.21.15.71:0/3571639777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103f50 msgr2=0x7f6b9c104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 --2- 172.21.15.71:0/3571639777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103f50 0x7f6b9c104330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009900 tx=0x7f6b8c02a420 comp rx=0 tx=0).stop 2024-09-05T12:48:28.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 -- 172.21.15.71:0/3571639777 shutdown_connections 2024-09-05T12:48:28.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 --2- 172.21.15.71:0/3571639777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 --2- 172.21.15.71:0/3571639777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b9c104900 0x7f6b9c108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 --2- 172.21.15.71:0/3571639777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103f50 0x7f6b9c104330 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 -- 172.21.15.71:0/3571639777 >> 172.21.15.71:0/3571639777 conn(0x7f6b9c0ff9e0 msgr2=0x7f6b9c101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:28.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 -- 172.21.15.71:0/3571639777 shutdown_connections 2024-09-05T12:48:28.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.171+0000 7f6ba2000640 1 -- 172.21.15.71:0/3571639777 wait complete. 2024-09-05T12:48:28.179 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 Processor -- start 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- start start 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b9c103f50 0x7f6b9c19a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c19ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b9c19f000 0x7f6b9c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c114190 con 0x7f6b9c1090a0 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c114010 con 0x7f6b9c103f50 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c114310 con 0x7f6b9c19f000 2024-09-05T12:48:28.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c19ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b9c103f50 0x7f6b9c19a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c19ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:43982/0 (socket says 172.21.15.71:43982) 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 -- 172.21.15.71:0/4098748294 learned_addr learned my addr 172.21.15.71:0/4098748294 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 -- 172.21.15.71:0/4098748294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b9c19f000 msgr2=0x7f6b9c19f4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b9c19f000 0x7f6b9c19f4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 -- 172.21.15.71:0/4098748294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b9c103f50 msgr2=0x7f6b9c19a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b9c103f50 0x7f6b9c19a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.181 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 -- 172.21.15.71:0/4098748294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c19faf0 con 0x7f6b9c1090a0 2024-09-05T12:48:28.184 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b9aa00640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c19ac80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800b470 tx=0x7f6b8800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b88015020 con 0x7f6b9c1090a0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b88011d70 con 0x7f6b9c1090a0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800f340 con 0x7f6b9c1090a0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c19fde0 con 0x7f6b9c1090a0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1a7580 con 0x7f6b9c1090a0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.176+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b9c104330 con 0x7f6b9c1090a0 2024-09-05T12:48:28.185 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.183+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6b88018020 con 0x7f6b9c1090a0 2024-09-05T12:48:28.186 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.183+0000 7f6b93e00640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b800a2e20 0x7f6b800a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.183+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f6b880cf5e0 con 0x7f6b9c1090a0 2024-09-05T12:48:28.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.183+0000 7f6b9b400640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b800a2e20 0x7f6b800a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.183+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b8801f020 con 0x7f6b9c1090a0 2024-09-05T12:48:28.187 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.183+0000 7f6b9b400640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b800a2e20 0x7f6b800a52e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c000c00 tx=0x7f6b8c005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:28.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.410+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f6b9c10c2b0 con 0x7f6b800a2e20 2024-09-05T12:48:28.423 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.421+0000 7f6b93e00640 1 -- 172.21.15.71:0/4098748294 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f6b9c10c2b0 con 0x7f6b800a2e20 2024-09-05T12:48:28.423 INFO:teuthology.orchestra.run.smithi071.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-05T00:00:00", "created": "2024-09-05T12:27:45", "first": "2024-09-05T12:28:00", "last": "2024-09-05T12:48:00", "last_pruned": "2024-09-05T12:48:01", "created_count": 21, "pruned_count": 15, "active": true} 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.423+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b800a2e20 msgr2=0x7f6b800a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.423+0000 7f6ba2000640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b800a2e20 0x7f6b800a52e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c000c00 tx=0x7f6b8c005b50 comp rx=0 tx=0).stop 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.423+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 msgr2=0x7f6b9c19ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.423+0000 7f6ba2000640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c19ac80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800b470 tx=0x7f6b8800b930 comp rx=0 tx=0).stop 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 shutdown_connections 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b9c19f000 0x7f6b9c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6b800a2e20 0x7f6b800a52e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c1090a0 0x7f6b9c19ac80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 --2- 172.21.15.71:0/4098748294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b9c103f50 0x7f6b9c19a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 >> 172.21.15.71:0/4098748294 conn(0x7f6b9c0ff9e0 msgr2=0x7f6b9c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 shutdown_connections 2024-09-05T12:48:28.427 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.424+0000 7f6ba2000640 1 -- 172.21.15.71:0/4098748294 wait complete. 2024-09-05T12:48:28.443 DEBUG:teuthology.orchestra.run.smithi071:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-05T12:48:28.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.596+0000 7f156b200640 1 -- 172.21.15.71:0/467750523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1564103b00 msgr2=0x7f1564103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.596+0000 7f156b200640 1 --2- 172.21.15.71:0/467750523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1564103b00 0x7f1564103f60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1554009930 tx=0x7f155402a420 comp rx=0 tx=0).stop 2024-09-05T12:48:28.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.596+0000 7f156b200640 1 -- 172.21.15.71:0/467750523 shutdown_connections 2024-09-05T12:48:28.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.596+0000 7f156b200640 1 --2- 172.21.15.71:0/467750523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15641044a0 0x7f15641109d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.597+0000 7f156b200640 1 --2- 172.21.15.71:0/467750523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1564103b00 0x7f1564103f60 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.597+0000 7f156b200640 1 --2- 172.21.15.71:0/467750523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564109ba0 0x7f1564109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.597+0000 7f156b200640 1 -- 172.21.15.71:0/467750523 >> 172.21.15.71:0/467750523 conn(0x7f15640ff7c0 msgr2=0x7f1564101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.597+0000 7f156b200640 1 -- 172.21.15.71:0/467750523 shutdown_connections 2024-09-05T12:48:28.600 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.597+0000 7f156b200640 1 -- 172.21.15.71:0/467750523 wait complete. 2024-09-05T12:48:28.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 Processor -- start 2024-09-05T12:48:28.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 -- start start 2024-09-05T12:48:28.602 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 0x7f156419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15641044a0 0x7f156419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1564109ba0 0x7f156419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1564113f60 con 0x7f15641044a0 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1564113de0 con 0x7f1564103b00 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156b200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15641140e0 con 0x7f1564109ba0 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f1569e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 0x7f156419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f1569400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15641044a0 0x7f156419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f1569e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 0x7f156419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58700/0 (socket says 172.21.15.71:58700) 2024-09-05T12:48:28.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f1569e00640 1 -- 172.21.15.71:0/3994928808 learned_addr learned my addr 172.21.15.71:0/3994928808 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.598+0000 7f156a800640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1564109ba0 0x7f156419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569e00640 1 -- 172.21.15.71:0/3994928808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1564109ba0 msgr2=0x7f156419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569e00640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1564109ba0 0x7f156419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569e00640 1 -- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15641044a0 msgr2=0x7f156419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569e00640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15641044a0 0x7f156419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569e00640 1 -- 172.21.15.71:0/3994928808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156419f460 con 0x7f1564103b00 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569400640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15641044a0 0x7f156419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f1569e00640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 0x7f156419a470 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f155800e9f0 tx=0x7f155800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:28.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155800cd80 con 0x7f1564103b00 2024-09-05T12:48:28.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1558010910 con 0x7f1564103b00 2024-09-05T12:48:28.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156419f750 con 0x7f1564103b00 2024-09-05T12:48:28.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.599+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558018990 con 0x7f1564103b00 2024-09-05T12:48:28.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.600+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a6fe0 con 0x7f1564103b00 2024-09-05T12:48:28.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.601+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564103f60 con 0x7f1564103b00 2024-09-05T12:48:28.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.602+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f1558010ac0 con 0x7f1564103b00 2024-09-05T12:48:28.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.603+0000 7f155ea00640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f154c0a2d70 0x7f154c0a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:28.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.604+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f15580cf9f0 con 0x7f1564103b00 2024-09-05T12:48:28.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.604+0000 7f1569400640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f154c0a2d70 0x7f154c0a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:28.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.604+0000 7f1569400640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f154c0a2d70 0x7f154c0a5230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f1554031510 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:28.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.607+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f155809bbf0 con 0x7f1564103b00 2024-09-05T12:48:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 8.2 MiB/s wr, 322 op/s 2024-09-05T12:48:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:28 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.816+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 --> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1564062630 con 0x7f154c0a2d70 2024-09-05T12:48:28.829 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.827+0000 7f155ea00640 1 -- 172.21.15.71:0/3994928808 <== mgr.15165 v2:172.21.15.71:6800/1162352135 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1564062630 con 0x7f154c0a2d70 2024-09-05T12:48:28.829 INFO:teuthology.orchestra.run.smithi071.stdout:/ 1m 6m3h 2024-09-05T12:48:28.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f154c0a2d70 msgr2=0x7f154c0a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f154c0a2d70 0x7f154c0a5230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1554006010 tx=0x7f1554031510 comp rx=0 tx=0).stop 2024-09-05T12:48:28.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 msgr2=0x7f156419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 0x7f156419a470 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f155800e9f0 tx=0x7f155800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 shutdown_connections 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1564109ba0 0x7f156419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f154c0a2d70 0x7f154c0a5230 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15641044a0 0x7f156419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 --2- 172.21.15.71:0/3994928808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1564103b00 0x7f156419a470 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 >> 172.21.15.71:0/3994928808 conn(0x7f15640ff7c0 msgr2=0x7f15640ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 shutdown_connections 2024-09-05T12:48:28.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:28.829+0000 7f156b200640 1 -- 172.21.15.71:0/3994928808 wait complete. 2024-09-05T12:48:28.851 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 8.2 MiB/s wr, 322 op/s 2024-09-05T12:48:28.851 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.851 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.851 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.851 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.852 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:28 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.853 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-05T12:48:28.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 8.2 MiB/s wr, 322 op/s 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-05T12:48:28.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi174 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-09-05T12:48:28.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:28.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:28.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:28.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:28 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:29.029 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 -- 172.21.15.71:0/3131260407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a341018e0 msgr2=0x7f6a34101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3131260407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a341018e0 0x7f6a34101d40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6a20009860 tx=0x7f6a2002a340 comp rx=0 tx=0).stop 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 -- 172.21.15.71:0/3131260407 shutdown_connections 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3131260407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a34102280 0x7f6a3410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3131260407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a341018e0 0x7f6a34101d40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 --2- 172.21.15.71:0/3131260407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a341078f0 0x7f6a34107cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 -- 172.21.15.71:0/3131260407 >> 172.21.15.71:0/3131260407 conn(0x7f6a340fd5e0 msgr2=0x7f6a340ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 -- 172.21.15.71:0/3131260407 shutdown_connections 2024-09-05T12:48:29.030 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.027+0000 7f6a33e00640 1 -- 172.21.15.71:0/3131260407 wait complete. 2024-09-05T12:48:29.032 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.028+0000 7f6a33e00640 1 Processor -- start 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.028+0000 7f6a33e00640 1 -- start start 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 0x7f6a340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34102280 0x7f6a34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a341078f0 0x7f6a34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34111d30 con 0x7f6a341018e0 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34111bb0 con 0x7f6a34102280 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34111eb0 con 0x7f6a341078f0 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 0x7f6a340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.033 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 0x7f6a340744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44018/0 (socket says 172.21.15.71:44018) 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 -- 172.21.15.71:0/946975680 learned_addr learned my addr 172.21.15.71:0/946975680 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32000640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34102280 0x7f6a34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a33400640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a341078f0 0x7f6a34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 -- 172.21.15.71:0/946975680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a341078f0 msgr2=0x7f6a34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a341078f0 0x7f6a34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 -- 172.21.15.71:0/946975680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34102280 msgr2=0x7f6a34074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34102280 0x7f6a34074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.029+0000 7f6a32a00640 1 -- 172.21.15.71:0/946975680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a341a6740 con 0x7f6a341018e0 2024-09-05T12:48:29.034 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.030+0000 7f6a32a00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 0x7f6a340744f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e9f0 tx=0x7f6a2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:29.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.030+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400cd80 con 0x7f6a341018e0 2024-09-05T12:48:29.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.030+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a24005500 con 0x7f6a341018e0 2024-09-05T12:48:29.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.030+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24010690 con 0x7f6a341018e0 2024-09-05T12:48:29.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.031+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a341a6a30 con 0x7f6a341018e0 2024-09-05T12:48:29.035 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.031+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a6f00 con 0x7f6a341018e0 2024-09-05T12:48:29.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.032+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a34101d40 con 0x7f6a341018e0 2024-09-05T12:48:29.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.034+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f6a240046c0 con 0x7f6a341018e0 2024-09-05T12:48:29.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.035+0000 7f6a38a00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a180a2e50 0x7f6a180a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.035+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f6a240cfa90 con 0x7f6a341018e0 2024-09-05T12:48:29.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.035+0000 7f6a32000640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a180a2e50 0x7f6a180a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.036+0000 7f6a32000640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a180a2e50 0x7f6a180a5310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a2002a930 tx=0x7f6a200058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:29.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.039+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a2409c2e0 con 0x7f6a341018e0 2024-09-05T12:48:29.233 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.231+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a3402d7c0 con 0x7f6a341018e0 2024-09-05T12:48:29.235 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.233+0000 7f6a38a00640 1 -- 172.21.15.71:0/946975680 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+25237 (secure 0 0 0) 0x7f6a2409bbe0 con 0x7f6a341018e0 2024-09-05T12:48:29.235 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:48:29.235 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":128,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:48:15.393851+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/2607784636":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/4208428214":"2024-09-06T12:27:44.498834+0000","172.21.15.71:6801/2050574497":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/2050574497":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/3957197147":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/372010080":"2024-09-06T12:27:44.498834+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:48:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a180a2e50 msgr2=0x7f6a180a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a180a2e50 0x7f6a180a5310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a2002a930 tx=0x7f6a200058d0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.237 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 msgr2=0x7f6a340744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 0x7f6a340744f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e9f0 tx=0x7f6a2400edf0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 shutdown_connections 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a341078f0 0x7f6a34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a34102280 0x7f6a34074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f6a180a2e50 0x7f6a180a5310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 --2- 172.21.15.71:0/946975680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a341018e0 0x7f6a340744f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 >> 172.21.15.71:0/946975680 conn(0x7f6a340fd5e0 msgr2=0x7f6a340fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 shutdown_connections 2024-09-05T12:48:29.238 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.235+0000 7f6a33e00640 1 -- 172.21.15.71:0/946975680 wait complete. 2024-09-05T12:48:29.255 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-05T12:19:53.639902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-05T12:24:44.687716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-05T12:24:47.492194+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-05T12:24:57.061844+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 37, 'snap_epoch': 126, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-05T12:48:29.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-05T12:48:29.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 -- 172.21.15.71:0/1859710387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0898102be0 msgr2=0x7f0898102fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 --2- 172.21.15.71:0/1859710387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0898102be0 0x7f0898102fc0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0888009900 tx=0x7f088802a480 comp rx=0 tx=0).stop 2024-09-05T12:48:29.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 -- 172.21.15.71:0/1859710387 shutdown_connections 2024-09-05T12:48:29.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 --2- 172.21.15.71:0/1859710387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f089810bfb0 0x7f089810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 --2- 172.21.15.71:0/1859710387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089810b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 --2- 172.21.15.71:0/1859710387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0898102be0 0x7f0898102fc0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 -- 172.21.15.71:0/1859710387 >> 172.21.15.71:0/1859710387 conn(0x7f08980778c0 msgr2=0x7f08980fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:29.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 -- 172.21.15.71:0/1859710387 shutdown_connections 2024-09-05T12:48:29.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.426+0000 7f089d000640 1 -- 172.21.15.71:0/1859710387 wait complete. 2024-09-05T12:48:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.427+0000 7f089d000640 1 Processor -- start 2024-09-05T12:48:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 -- start start 2024-09-05T12:48:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0898102be0 0x7f089819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.432 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f089810bfb0 0x7f089819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111d80 con 0x7f089810bfb0 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111c00 con 0x7f0898102be0 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f089d000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0898111f00 con 0x7f0898103500 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f0897400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0898102be0 0x7f089819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f0896a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f0897400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0898102be0 0x7f089819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:58736/0 (socket says 172.21.15.71:58736) 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.428+0000 7f0896a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:55100/0 (socket says 172.21.15.71:55100) 2024-09-05T12:48:29.433 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0897400640 1 -- 172.21.15.71:0/1390225703 learned_addr learned my addr 172.21.15.71:0/1390225703 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0897e00640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f089810bfb0 0x7f089819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0896a00640 1 -- 172.21.15.71:0/1390225703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0898102be0 msgr2=0x7f089819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0896a00640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0898102be0 0x7f089819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0896a00640 1 -- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f089810bfb0 msgr2=0x7f089819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0896a00640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f089810bfb0 0x7f089819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0896a00640 1 -- 172.21.15.71:0/1390225703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089819f4a0 con 0x7f0898103500 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f0896a00640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089819a9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9f0 tx=0x7f088400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:29.434 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.429+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088400ccc0 con 0x7f0898103500 2024-09-05T12:48:29.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.430+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0884010910 con 0x7f0898103500 2024-09-05T12:48:29.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.430+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089819f790 con 0x7f0898103500 2024-09-05T12:48:29.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.430+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884018960 con 0x7f0898103500 2024-09-05T12:48:29.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.430+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08981a7020 con 0x7f0898103500 2024-09-05T12:48:29.435 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.431+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0898102fc0 con 0x7f0898103500 2024-09-05T12:48:29.436 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.434+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0884010ac0 con 0x7f0898103500 2024-09-05T12:48:29.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.435+0000 7f088fe00640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f087c0a2d00 0x7f087c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.436+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f08840d0080 con 0x7f0898103500 2024-09-05T12:48:29.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.436+0000 7f0897400640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f087c0a2d00 0x7f087c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.436+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f088409c2e0 con 0x7f0898103500 2024-09-05T12:48:29.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.436+0000 7f0897400640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f087c0a2d00 0x7f087c0a51c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08880096c0 tx=0x7f0888005a20 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:29.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.627+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08981a7220 con 0x7f0898103500 2024-09-05T12:48:29.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.628+0000 7f088fe00640 1 -- 172.21.15.71:0/1390225703 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f08840a05e0 con 0x7f0898103500 2024-09-05T12:48:29.630 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 1 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.630+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f087c0a2d00 msgr2=0x7f087c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.630+0000 7f089d000640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f087c0a2d00 0x7f087c0a51c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f08880096c0 tx=0x7f0888005a20 comp rx=0 tx=0).stop 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.630+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 msgr2=0x7f089819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.630+0000 7f089d000640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089819a9f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f088400e9f0 tx=0x7f088400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 shutdown_connections 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f087c0a2d00 0x7f087c0a51c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f089810bfb0 0x7f089819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0898103500 0x7f089819a9f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 --2- 172.21.15.71:0/1390225703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0898102be0 0x7f089819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 >> 172.21.15.71:0/1390225703 conn(0x7f08980778c0 msgr2=0x7f08980fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:29.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 shutdown_connections 2024-09-05T12:48:29.634 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.631+0000 7f089d000640 1 -- 172.21.15.71:0/1390225703 wait complete. 2024-09-05T12:48:29.647 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:29 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:48:29.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:29 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:48:29.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:29 smithi071 ceph-mon[29845]: from='client.17667 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:48:29.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:29 smithi071 ceph-mon[29845]: from='client.27677 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:48:29.648 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:29 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/946975680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:48:29.650 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-05T12:48:29.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:29 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:48:29.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:29 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:48:29.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:29 smithi174 ceph-mon[34313]: from='client.17667 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:48:29.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:29 smithi174 ceph-mon[34313]: from='client.27677 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:48:29.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:29 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/946975680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:48:29.813 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 -- 172.21.15.71:0/1479499767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06701079a0 msgr2=0x7f0670107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 --2- 172.21.15.71:0/1479499767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06701079a0 0x7f0670107d80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f0664009860 tx=0x7f066402a340 comp rx=0 tx=0).stop 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 -- 172.21.15.71:0/1479499767 shutdown_connections 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 --2- 172.21.15.71:0/1479499767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 0x7f067010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 --2- 172.21.15.71:0/1479499767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0670101990 0x7f0670101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 --2- 172.21.15.71:0/1479499767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06701079a0 0x7f0670107d80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 -- 172.21.15.71:0/1479499767 >> 172.21.15.71:0/1479499767 conn(0x7f06700fd670 msgr2=0x7f06700ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:29.814 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 -- 172.21.15.71:0/1479499767 shutdown_connections 2024-09-05T12:48:29.815 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.811+0000 7f0676000640 1 -- 172.21.15.71:0/1479499767 wait complete. 2024-09-05T12:48:29.816 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 Processor -- start 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 -- start start 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670101990 0x7f06700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 0x7f0670074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06701079a0 0x7f0670074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0670111de0 con 0x7f0670102330 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0670111c60 con 0x7f0670101990 2024-09-05T12:48:29.817 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.812+0000 7f0676000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0670111f60 con 0x7f06701079a0 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f066fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 0x7f0670074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670101990 0x7f06700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f066fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 0x7f0670074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44064/0 (socket says 172.21.15.71:44064) 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f066fe00640 1 -- 172.21.15.71:0/3431420432 learned_addr learned my addr 172.21.15.71:0/3431420432 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0675600640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06701079a0 0x7f0670074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 -- 172.21.15.71:0/3431420432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06701079a0 msgr2=0x7f0670074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06701079a0 0x7f0670074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.818 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 -- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 msgr2=0x7f0670074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 0x7f0670074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 -- 172.21.15.71:0/3431420432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06701a6810 con 0x7f0670101990 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0675600640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06701079a0 0x7f0670074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.813+0000 7f0674c00640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670101990 0x7f06700744f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0664035040 tx=0x7f0664033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.814+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066403f020 con 0x7f0670101990 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.814+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0664004040 con 0x7f0670101990 2024-09-05T12:48:29.819 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.814+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0664037040 con 0x7f0670101990 2024-09-05T12:48:29.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.814+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06701a6aa0 con 0x7f0670101990 2024-09-05T12:48:29.820 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.814+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06701a6fe0 con 0x7f0670101990 2024-09-05T12:48:29.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.815+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0670101df0 con 0x7f0670101990 2024-09-05T12:48:29.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.817+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f0664042020 con 0x7f0670101990 2024-09-05T12:48:29.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.819+0000 7f066d600640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06580a2d80 0x7f06580a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:29.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.819+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f06640ef760 con 0x7f0670101990 2024-09-05T12:48:29.821 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.819+0000 7f066fe00640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06580a2d80 0x7f06580a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:29.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.820+0000 7f066fe00640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06580a2d80 0x7f06580a5240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0670075f80 tx=0x7f065c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:29.827 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:29.823+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0664049020 con 0x7f0670101990 2024-09-05T12:48:29.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:29 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-05T12:48:29.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:29 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi123 to 184588185: error parsing value: Value '184588185' is below minimum 939524096 2024-09-05T12:48:29.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:29 smithi123 ceph-mon[34321]: from='client.17667 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:48:29.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:29 smithi123 ceph-mon[34321]: from='client.27677 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-05T12:48:29.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:29 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/946975680' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:48:30.009 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.007+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0670109f00 con 0x7f0670101990 2024-09-05T12:48:30.010 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.008+0000 7f066d600640 1 -- 172.21.15.71:0/3431420432 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) ==== 104+0+11 (secure 0 0 0) 0x7f06640bc0c0 con 0x7f0670101990 2024-09-05T12:48:30.010 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-09-05T12:48:30.012 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06580a2d80 msgr2=0x7f06580a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06580a2d80 0x7f06580a5240 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0670075f80 tx=0x7f065c005f90 comp rx=0 tx=0).stop 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670101990 msgr2=0x7f06700744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670101990 0x7f06700744f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0664035040 tx=0x7f0664033b60 comp rx=0 tx=0).stop 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 shutdown_connections 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06701079a0 0x7f0670074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f06580a2d80 0x7f06580a5240 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0670102330 0x7f0670074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.013 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 --2- 172.21.15.71:0/3431420432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0670101990 0x7f06700744f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.010+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 >> 172.21.15.71:0/3431420432 conn(0x7f06700fd670 msgr2=0x7f06700fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:30.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.011+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 shutdown_connections 2024-09-05T12:48:30.014 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.011+0000 7f0676000640 1 -- 172.21.15.71:0/3431420432 wait complete. 2024-09-05T12:48:30.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-05T12:48:30.193 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 -- 172.21.15.71:0/2932400167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c074ee0 msgr2=0x7f344c10dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 --2- 172.21.15.71:0/2932400167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c074ee0 0x7f344c10dde0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3438009900 tx=0x7f343802a480 comp rx=0 tx=0).stop 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 -- 172.21.15.71:0/2932400167 shutdown_connections 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 --2- 172.21.15.71:0/2932400167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f344c10e320 0x7f344c110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 --2- 172.21.15.71:0/2932400167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c074ee0 0x7f344c10dde0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 --2- 172.21.15.71:0/2932400167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f344c0745c0 0x7f344c0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 -- 172.21.15.71:0/2932400167 >> 172.21.15.71:0/2932400167 conn(0x7f344c0fd7c0 msgr2=0x7f344c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 -- 172.21.15.71:0/2932400167 shutdown_connections 2024-09-05T12:48:30.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.191+0000 7f3450e00640 1 -- 172.21.15.71:0/2932400167 wait complete. 2024-09-05T12:48:30.196 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.192+0000 7f3450e00640 1 Processor -- start 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.192+0000 7f3450e00640 1 -- start start 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.192+0000 7f3450e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 0x7f344c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.192+0000 7f3450e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f344c074ee0 0x7f344c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3450e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f344c10e320 0x7f344c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3450e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f344c1140b0 con 0x7f344c0745c0 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3450e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f344c113f30 con 0x7f344c074ee0 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3450e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f344c114230 con 0x7f344c10e320 2024-09-05T12:48:30.197 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 0x7f344c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 0x7f344c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44088/0 (socket says 172.21.15.71:44088) 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 -- 172.21.15.71:0/933620716 learned_addr learned my addr 172.21.15.71:0/933620716 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449400640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f344c074ee0 0x7f344c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f344a800640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f344c10e320 0x7f344c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 -- 172.21.15.71:0/933620716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f344c10e320 msgr2=0x7f344c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f344c10e320 0x7f344c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 -- 172.21.15.71:0/933620716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f344c074ee0 msgr2=0x7f344c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f344c074ee0 0x7f344c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.198 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.193+0000 7f3449e00640 1 -- 172.21.15.71:0/933620716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344c19f6a0 con 0x7f344c0745c0 2024-09-05T12:48:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.194+0000 7f3449e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 0x7f344c19a6b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f343c00e9f0 tx=0x7f343c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.194+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c00cd80 con 0x7f344c0745c0 2024-09-05T12:48:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.194+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f343c005500 con 0x7f344c0745c0 2024-09-05T12:48:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.194+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c010690 con 0x7f344c0745c0 2024-09-05T12:48:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.194+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344c19f990 con 0x7f344c0745c0 2024-09-05T12:48:30.199 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.194+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344c1a7220 con 0x7f344c0745c0 2024-09-05T12:48:30.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.195+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344c0749a0 con 0x7f344c0745c0 2024-09-05T12:48:30.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.197+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f343c0046c0 con 0x7f344c0745c0 2024-09-05T12:48:30.200 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.198+0000 7f3442a00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34300a2d80 0x7f34300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.198+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f343c0cfba0 con 0x7f344c0745c0 2024-09-05T12:48:30.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.199+0000 7f3449400640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34300a2d80 0x7f34300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.201 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.199+0000 7f3449400640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34300a2d80 0x7f34300a5240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3438009230 tx=0x7f34380058d0 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:30.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.202+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f343c09c3f0 con 0x7f344c0745c0 2024-09-05T12:48:30.425 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.423+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f344c1094a0 con 0x7f344c0745c0 2024-09-05T12:48:30.426 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.424+0000 7f3442a00640 1 -- 172.21.15.71:0/933620716 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) ==== 100+0+11 (secure 0 0 0) 0x7f343c09bcf0 con 0x7f344c0745c0 2024-09-05T12:48:30.426 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 64 2024-09-05T12:48:30.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34300a2d80 msgr2=0x7f34300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34300a2d80 0x7f34300a5240 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3438009230 tx=0x7f34380058d0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.428 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 msgr2=0x7f344c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 0x7f344c19a6b0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f343c00e9f0 tx=0x7f343c00edf0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 shutdown_connections 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f344c10e320 0x7f344c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f344c074ee0 0x7f344c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f34300a2d80 0x7f34300a5240 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 --2- 172.21.15.71:0/933620716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f344c0745c0 0x7f344c19a6b0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 >> 172.21.15.71:0/933620716 conn(0x7f344c0fd7c0 msgr2=0x7f344c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:30.429 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 shutdown_connections 2024-09-05T12:48:30.430 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.426+0000 7f3450e00640 1 -- 172.21.15.71:0/933620716 wait complete. 2024-09-05T12:48:30.445 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-05T12:48:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 -- 172.21.15.71:0/2705824843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f4109060 msgr2=0x7fa4f4110bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2705824843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f4109060 0x7fa4f4110bb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec009ca0 tx=0x7fa4ec02a8c0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 -- 172.21.15.71:0/2705824843 shutdown_connections 2024-09-05T12:48:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2705824843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f4109060 0x7fa4f4110bb0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2705824843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4f41048c0 0x7fa4f4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2705824843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 0x7fa4f41042f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 -- 172.21.15.71:0/2705824843 >> 172.21.15.71:0/2705824843 conn(0x7fa4f40ff980 msgr2=0x7fa4f4101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:30.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 -- 172.21.15.71:0/2705824843 shutdown_connections 2024-09-05T12:48:30.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.602+0000 7fa4faa00640 1 -- 172.21.15.71:0/2705824843 wait complete. 2024-09-05T12:48:30.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 Processor -- start 2024-09-05T12:48:30.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 -- start start 2024-09-05T12:48:30.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 0x7fa4f419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4f4109060 0x7fa4f419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 0x7fa4f419f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f41140e0 con 0x7fa4f419ee70 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4113f60 con 0x7fa4f4109060 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4faa00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f4114260 con 0x7fa4f4103f10 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4f8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 0x7fa4f419f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4f3400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4f4109060 0x7fa4f419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4f3e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 0x7fa4f419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4f8a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 0x7fa4f419f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44102/0 (socket says 172.21.15.71:44102) 2024-09-05T12:48:30.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.603+0000 7fa4f8a00640 1 -- 172.21.15.71:0/2292621738 learned_addr learned my addr 172.21.15.71:0/2292621738 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f8a00640 1 -- 172.21.15.71:0/2292621738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 msgr2=0x7fa4f419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f8a00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 0x7fa4f419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f8a00640 1 -- 172.21.15.71:0/2292621738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4f4109060 msgr2=0x7fa4f419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f8a00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4f4109060 0x7fa4f419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f8a00640 1 -- 172.21.15.71:0/2292621738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f419f960 con 0x7fa4f419ee70 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f3e00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 0x7fa4f419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f8a00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 0x7fa4f419f320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec005e30 tx=0x7fa4ec003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec031650 con 0x7fa4f419ee70 2024-09-05T12:48:30.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4ec0317f0 con 0x7fa4f419ee70 2024-09-05T12:48:30.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f419fbf0 con 0x7fa4f419ee70 2024-09-05T12:48:30.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.604+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec02f9d0 con 0x7fa4f419ee70 2024-09-05T12:48:30.610 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.605+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f41a7410 con 0x7fa4f419ee70 2024-09-05T12:48:30.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.606+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f41042f0 con 0x7fa4f419ee70 2024-09-05T12:48:30.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.607+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fa4ec02fb70 con 0x7fa4f419ee70 2024-09-05T12:48:30.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.609+0000 7fa4f0c00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4d80a2d00 0x7fa4d80a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:30.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.609+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7fa4ec0f0bd0 con 0x7fa4f419ee70 2024-09-05T12:48:30.611 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.609+0000 7fa4f3e00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4d80a2d00 0x7fa4d80a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:30.612 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.610+0000 7fa4f3e00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4d80a2d00 0x7fa4d80a51c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40095f0 tx=0x7fa4e4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:30.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.611+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4ec0bd530 con 0x7fa4f419ee70 2024-09-05T12:48:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-09-05T12:48:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.760 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1390225703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3431420432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:48:30.761 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:48:30.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.762 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:30 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/933620716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.787 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.785+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa4f41048c0 con 0x7fa4f419ee70 2024-09-05T12:48:30.875 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.786+0000 7fa4f0c00640 1 -- 172.21.15.71:0/2292621738 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v128) ==== 103+0+11 (secure 0 0 0) 0x7fa4ec0bce30 con 0x7fa4f419ee70 2024-09-05T12:48:30.875 INFO:teuthology.orchestra.run.smithi071.stdout:pg_num: 32 2024-09-05T12:48:30.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4d80a2d00 msgr2=0x7fa4d80a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4d80a2d00 0x7fa4d80a51c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40095f0 tx=0x7fa4e4005f90 comp rx=0 tx=0).stop 2024-09-05T12:48:30.876 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 msgr2=0x7fa4f419f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 0x7fa4f419f320 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec005e30 tx=0x7fa4ec003d00 comp rx=0 tx=0).stop 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 shutdown_connections 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fa4d80a2d00 0x7fa4d80a51c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa4f419ee70 0x7fa4f419f320 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4f4109060 0x7fa4f419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 --2- 172.21.15.71:0/2292621738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4f4103f10 0x7fa4f419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 >> 172.21.15.71:0/2292621738 conn(0x7fa4f40ff980 msgr2=0x7fa4f4100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:30.877 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 shutdown_connections 2024-09-05T12:48:30.878 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:30.788+0000 7fa4faa00640 1 -- 172.21.15.71:0/2292621738 wait complete. 2024-09-05T12:48:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-09-05T12:48:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1390225703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3431420432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.878 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.879 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:30 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/933620716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.880 DEBUG:teuthology.orchestra.run.smithi071:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-05T12:48:30.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 4.9 MiB/s wr, 220 op/s 2024-09-05T12:48:30.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1390225703' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3431420432' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-05T12:48:30.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' 2024-09-05T12:48:30.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:30 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/933620716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-05T12:48:31.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 -- 172.21.15.71:0/3824714845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4109b70 msgr2=0x7f99b4109f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:31.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 --2- 172.21.15.71:0/3824714845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4109b70 0x7f99b4109f50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f99a8009900 tx=0x7f99a802a3f0 comp rx=0 tx=0).stop 2024-09-05T12:48:31.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 -- 172.21.15.71:0/3824714845 shutdown_connections 2024-09-05T12:48:31.037 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 --2- 172.21.15.71:0/3824714845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4104470 0x7f99b41109a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 --2- 172.21.15.71:0/3824714845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4103ad0 0x7f99b4103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 --2- 172.21.15.71:0/3824714845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4109b70 0x7f99b4109f50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 -- 172.21.15.71:0/3824714845 >> 172.21.15.71:0/3824714845 conn(0x7f99b40ff7d0 msgr2=0x7f99b4101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 -- 172.21.15.71:0/3824714845 shutdown_connections 2024-09-05T12:48:31.038 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.035+0000 7f99bb200640 1 -- 172.21.15.71:0/3824714845 wait complete. 2024-09-05T12:48:31.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 Processor -- start 2024-09-05T12:48:31.040 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 -- start start 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 0x7f99b419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4104470 0x7f99b419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4109b70 0x7f99b419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f99b4113f20 con 0x7f99b4103ad0 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f99b4113da0 con 0x7f99b4104470 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99bb200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99b41140a0 con 0x7f99b4109b70 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99b9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 0x7f99b419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:31.041 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99b9e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 0x7f99b419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:44110/0 (socket says 172.21.15.71:44110) 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99b9e00640 1 -- 172.21.15.71:0/1440245163 learned_addr learned my addr 172.21.15.71:0/1440245163 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.036+0000 7f99b9400640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4104470 0x7f99b419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99ba800640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4109b70 0x7f99b419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99b9e00640 1 -- 172.21.15.71:0/1440245163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4109b70 msgr2=0x7f99b419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99b9e00640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4109b70 0x7f99b419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99b9e00640 1 -- 172.21.15.71:0/1440245163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4104470 msgr2=0x7f99b419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99b9e00640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4104470 0x7f99b419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:31.042 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99b9e00640 1 -- 172.21.15.71:0/1440245163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b419f4a0 con 0x7f99b4103ad0 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99ba800640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4109b70 0x7f99b419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99b9e00640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 0x7f99b419a4b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006010 tx=0x7f99a8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a803f070 con 0x7f99b4103ad0 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99a8031c30 con 0x7f99b4103ad0 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a8035030 con 0x7f99b4103ad0 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.037+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b419f730 con 0x7f99b4103ad0 2024-09-05T12:48:31.043 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.038+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b41a7020 con 0x7f99b4103ad0 2024-09-05T12:48:31.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.039+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99b4103f30 con 0x7f99b4103ad0 2024-09-05T12:48:31.044 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.040+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f99a8042070 con 0x7f99b4103ad0 2024-09-05T12:48:31.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.042+0000 7f99aea00640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f999c0a2d00 0x7f999c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:31.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.043+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f99a80efbd0 con 0x7f99b4103ad0 2024-09-05T12:48:31.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.043+0000 7f99b9400640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f999c0a2d00 0x7f999c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:31.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.043+0000 7f99b9400640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f999c0a2d00 0x7f999c0a51c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f99b4101de0 tx=0x7f99a4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:31.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.047+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99a80bc420 con 0x7f99b4103ad0 2024-09-05T12:48:31.269 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.267+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f99b410c040 con 0x7f99b4103ad0 2024-09-05T12:48:31.279 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.277+0000 7f99aea00640 1 -- 172.21.15.71:0/1440245163 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+740900 (secure 0 0 0) 0x7f99a80bbd20 con 0x7f99b4103ad0 2024-09-05T12:48:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:48:31.289 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:48:31.289 INFO:teuthology.orchestra.run.smithi071.stdout: "epoch": 26, 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "active_gid": 15165, 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "active_name": "x", 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "active_addrs": { 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:31.290 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6800", 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1162352135 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v1", 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:6801", 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1162352135 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:31.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "active_addr": "172.21.15.71:6801/1162352135", 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "active_change": "2024-09-05T12:27:44.499160+0000", 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "active_mgr_features": 4540701547738038271, 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "available": true, 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "standbys": [ 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 25207, 2024-09-05T12:48:31.292 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "z", 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_features": 4540701547738038271, 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "available_modules": [ 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "alerts", 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:31.293 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.294 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.295 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.296 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.297 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.298 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.299 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.300 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.301 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.302 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.303 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_destination": { 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_destination", 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.304 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email address to send alerts to", 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_from_name": { 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_from_name", 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "Ceph", 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.306 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email From: name", 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_host": { 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_host", 2024-09-05T12:48:31.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.308 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP server", 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_password": { 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_password", 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.309 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Password to authenticate with", 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.310 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_port": { 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_port", 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.311 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "465", 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP port", 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_sender": { 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_sender", 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.313 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP envelope sender", 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_ssl": { 2024-09-05T12:48:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_ssl", 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.315 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_user": { 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_user", 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.316 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "User to authenticate as", 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.317 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "balancer", 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.318 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "active": { 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "active", 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.319 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_time": { 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_time", 2024-09-05T12:48:31.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0000", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:48:31.321 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_weekday": { 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_weekday", 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_max_iterations": { 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_max_iterations", 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "25", 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:31.324 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "250", 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_metrics": { 2024-09-05T12:48:31.325 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_metrics", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T12:48:31.326 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_step": { 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_step", 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.327 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.5", 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0.001", 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "0.999", 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "aggressiveness of optimization", 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "end_time": { 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_time", 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2359", 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.329 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "end_weekday": { 2024-09-05T12:48:31.330 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_weekday", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T12:48:31.331 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.332 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.333 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.334 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.335 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.336 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.337 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.338 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.339 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.340 INFO:teuthology.orchestra.run.smithi071.stdout: "min_score": { 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_score", 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.341 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.342 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "upmap", 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-compat", 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "none", 2024-09-05T12:48:31.343 INFO:teuthology.orchestra.run.smithi071.stdout: "read", 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap", 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap-read" 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Balancer mode", 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_ids": { 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_ids", 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.345 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.346 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.346 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T12:48:31.346 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.346 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.346 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.347 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.348 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_deviation": { 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_deviation", 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.349 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_optimizations": { 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_optimizations", 2024-09-05T12:48:31.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T12:48:31.351 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm", 2024-09-05T12:48:31.352 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_down_multiplier": { 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_down_multiplier", 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.353 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.354 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_refresh_rate": { 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_refresh_rate", 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "20", 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_starting_port": { 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_starting_port", 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.356 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4721", 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.357 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_ptrace": { 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_ptrace", 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.358 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_interval": { 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_interval", 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.360 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_memory_target_ratio": { 2024-09-05T12:48:31.361 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.7", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T12:48:31.362 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_log_destination": { 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm_log_destination", 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.363 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "file", 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "file,syslog", 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "syslog" 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.364 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "cgroups_split": { 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cgroups_split", 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.365 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "config_checks_enabled": { 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_checks_enabled", 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.367 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "config_dashboard": { 2024-09-05T12:48:31.368 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_dashboard", 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.369 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_alertmanager": { 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_alertmanager", 2024-09-05T12:48:31.370 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.371 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_base": { 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_base", 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T12:48:31.372 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Container image name, without the tag", 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.373 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_elasticsearch": { 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_elasticsearch", 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.374 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "elasticsearch container image", 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_grafana": { 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_grafana", 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.375 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.376 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_haproxy": { 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_haproxy", 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.377 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "HAproxy container image", 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.378 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_agent": { 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_agent", 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger agent container image", 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_collector": { 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_collector", 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.380 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger collector container image", 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.381 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_query": { 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_query", 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.382 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger query container image", 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_keepalived": { 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_keepalived", 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.384 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Keepalived container image", 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_loki": { 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_loki", 2024-09-05T12:48:31.385 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Loki container image", 2024-09-05T12:48:31.386 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_node_exporter": { 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_node_exporter", 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.387 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.388 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_nvmeof": { 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_nvmeof", 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T12:48:31.389 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Nvme-of container image", 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.390 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_prometheus": { 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_prometheus", 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.391 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_promtail": { 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_promtail", 2024-09-05T12:48:31.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Promtail container image", 2024-09-05T12:48:31.393 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_samba": { 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_samba", 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.394 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Samba/SMB container image", 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_snmp_gateway": { 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_snmp_gateway", 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.396 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SNMP Gateway container image", 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "container_init": { 2024-09-05T12:48:31.397 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_init", 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.398 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "daemon_cache_timeout": { 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "daemon_cache_timeout", 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.399 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.400 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "default_cephadm_command_timeout": { 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.401 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "900", 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.402 INFO:teuthology.orchestra.run.smithi071.stdout: "default_registry": { 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_registry", 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io", 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.403 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "device_cache_timeout": { 2024-09-05T12:48:31.404 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_cache_timeout", 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1800", 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache device inventory", 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "device_enhanced_scan": { 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_enhanced_scan", 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.406 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.407 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "facts_cache_timeout": { 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "facts_cache_timeout", 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:31.408 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache host facts data", 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.409 INFO:teuthology.orchestra.run.smithi071.stdout: "grafana_dashboards_path": { 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "grafana_dashboards_path", 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.410 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "host_check_interval": { 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "host_check_interval", 2024-09-05T12:48:31.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to perform a host check", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.412 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "hw_monitoring": { 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hw_monitoring", 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.413 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "inventory_list_all": { 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "inventory_list_all", 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.415 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.416 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.417 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.418 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "log_refresh_metadata": { 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_refresh_metadata", 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.419 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.420 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.421 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.422 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.423 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.424 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.426 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.427 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "*", 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.429 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "max_count_per_host": { 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_count_per_host", 2024-09-05T12:48:31.430 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.431 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of daemons per service per host", 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd_draining_count": { 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_osd_draining_count", 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.432 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.433 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "migration_current": { 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "migration_current", 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.434 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "internal - do not modify", 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "root", 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.436 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm-package", 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: "root" 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "oob_default_addr": { 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "oob_default_addr", 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "169.254.1.1", 2024-09-05T12:48:31.438 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.439 INFO:teuthology.orchestra.run.smithi071.stdout: "prometheus_alerts_path": { 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus_alerts_path", 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.440 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_insecure": { 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_insecure", 2024-09-05T12:48:31.441 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.442 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_password": { 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_password", 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.443 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.444 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_url": { 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_url", 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.445 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_username": { 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_username", 2024-09-05T12:48:31.446 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.447 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "secure_monitoring_stack": { 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "secure_monitoring_stack", 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.448 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "service_discovery_port": { 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "service_discovery_port", 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8765", 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.450 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "cephadm service discovery port", 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "ssh_config_file": { 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssh_config_file", 2024-09-05T12:48:31.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T12:48:31.452 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "use_agent": { 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_agent", 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.453 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.454 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "use_repo_digest": { 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_repo_digest", 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.455 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_failed_host_check": { 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_failed_host_check", 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.456 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.457 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_daemons": { 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_daemons", 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.458 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.459 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_hosts": { 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_hosts", 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.460 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.461 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crash", 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.462 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.463 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.465 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.468 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.469 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.470 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.471 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "retain_interval": { 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retain_interval", 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.472 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "31536000", 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_recent_interval": { 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_recent_interval", 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.474 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1209600", 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.475 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dashboard", 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T12:48:31.476 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.477 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.478 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.479 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.480 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_ENABLED": { 2024-09-05T12:48:31.481 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.482 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.483 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.485 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.487 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T12:48:31.488 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.489 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.490 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.491 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.492 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.494 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T12:48:31.495 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.496 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.498 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.499 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.500 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.501 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T12:48:31.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.503 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_URL": { 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_URL", 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.504 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.505 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:31.506 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.507 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.508 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T12:48:31.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.510 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.512 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.513 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.514 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.515 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.516 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.517 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.518 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.519 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.520 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T12:48:31.521 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.522 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.523 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.524 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.525 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.526 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.527 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.528 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.529 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.530 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T12:48:31.531 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.532 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.533 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.534 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8", 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.535 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T12:48:31.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "45", 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.537 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.538 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.539 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.540 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T12:48:31.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.542 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.543 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.544 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.545 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T12:48:31.546 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.547 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.548 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.549 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "cross_origin_url": { 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cross_origin_url", 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.550 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "crt_file": { 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crt_file", 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.552 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "debug": { 2024-09-05T12:48:31.553 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "debug", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable/disable debug options", 2024-09-05T12:48:31.554 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: "jwt_token_ttl": { 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "jwt_token_ttl", 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.555 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "28800", 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.556 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.557 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.558 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.559 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.560 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.561 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.562 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.563 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.564 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.565 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.566 INFO:teuthology.orchestra.run.smithi071.stdout: "motd": { 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "motd", 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.567 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "The message of the day", 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect_resolve_ip_addr": { 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.568 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.569 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-09-05T12:48:31.570 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:31.571 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8080", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.572 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.573 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl_server_port": { 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl_server_port", 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8443", 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.575 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-09-05T12:48:31.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "redirect", 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.577 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect" 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-09-05T12:48:31.578 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.579 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "url_prefix": { 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url_prefix", 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.580 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.581 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_monitoring": { 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_monitoring", 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.582 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "monitor device health metrics", 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.583 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.584 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.585 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.586 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.587 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.588 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.589 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.590 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.591 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "mark_out_threshold": { 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mark_out_threshold", 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.592 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2419200", 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.593 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": { 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_name", 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "device_health_metrics", 2024-09-05T12:48:31.594 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.595 INFO:teuthology.orchestra.run.smithi071.stdout: "retention_period": { 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retention_period", 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15552000", 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain device health metrics", 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_frequency": { 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_frequency", 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.597 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T12:48:31.598 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "self_heal": { 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "self_heal", 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.599 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.600 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.601 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_threshold": { 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_threshold", 2024-09-05T12:48:31.602 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7257600", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.603 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "diskprediction_local", 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.604 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.605 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.606 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.608 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.609 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.610 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.612 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "predict_interval": { 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predict_interval", 2024-09-05T12:48:31.613 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.614 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "predictor_model": { 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predictor_model", 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.615 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "prophetstor", 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.617 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.618 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "influx", 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": false, 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "influxdb python module not found", 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "batch_size": { 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "batch_size", 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5000", 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.620 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "database": { 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "database", 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "ceph", 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.621 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": { 2024-09-05T12:48:31.622 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hostname", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server hostname", 2024-09-05T12:48:31.623 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.624 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "30", 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "5", 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.625 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.626 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.627 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.628 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.629 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.630 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.631 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.632 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.633 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "password": { 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "password", 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.634 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "password of InfluxDB server user", 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "port": { 2024-09-05T12:48:31.635 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "port", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8086", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server port", 2024-09-05T12:48:31.636 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.637 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "false", 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.638 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "threads": { 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threads", 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:31.639 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "32", 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "username": { 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "username", 2024-09-05T12:48:31.640 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "username of InfluxDB server user", 2024-09-05T12:48:31.641 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "verify_ssl": { 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "verify_ssl", 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.642 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "true", 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.643 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "insights", 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.644 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.645 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.646 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.647 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.648 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.649 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.651 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.652 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "iostat", 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.653 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.654 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.655 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.656 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.657 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.658 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.659 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.660 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.661 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "k8sevents", 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_event_retention_days": { 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ceph_event_retention_days", 2024-09-05T12:48:31.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7", 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T12:48:31.663 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "config_check_secs": { 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_check_secs", 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.664 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "10", 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.665 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.666 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.667 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.668 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.669 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.670 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.671 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.672 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.673 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "localpool", 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.674 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "failure_domain": { 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "failure_domain", 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "host", 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.675 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "failure domain for any created local pool", 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.677 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.678 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.679 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.680 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.681 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.682 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.683 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": { 2024-09-05T12:48:31.684 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_size", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default min_size for any created local pool", 2024-09-05T12:48:31.685 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "num_rep": { 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "num_rep", 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.686 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3", 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default replica count for any created local pool", 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.687 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": { 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_num", 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "128", 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.688 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "prefix": { 2024-09-05T12:48:31.689 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prefix", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name prefix for any created local pool", 2024-09-05T12:48:31.690 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "subtree": { 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subtree", 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.691 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "rack", 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.692 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mds_autoscaler", 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.693 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.694 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.695 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.696 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.697 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.698 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.699 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.701 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirroring", 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.703 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.704 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.705 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.706 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.707 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.708 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.709 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.710 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.711 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "nfs", 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.712 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.713 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.714 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.715 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.716 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.717 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.718 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.719 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.720 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "fail_fs": { 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "fail_fs", 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.721 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.722 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.723 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.724 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.726 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.727 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.728 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.729 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.730 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator": { 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.731 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "rook", 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "test_orchestrator" 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.732 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Orchestrator backend", 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.733 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_perf_query", 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.734 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.735 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.736 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.737 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.738 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.739 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.741 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_support", 2024-09-05T12:48:31.742 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.743 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.744 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.746 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.747 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.748 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.749 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.750 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.751 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_autoscaler", 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.752 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.753 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.754 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.755 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.756 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.757 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.758 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.759 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.760 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.761 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.762 INFO:teuthology.orchestra.run.smithi071.stdout: "threshold": { 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threshold", 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1.0", 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.763 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "scaling threshold", 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.764 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "progress", 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_pg_recovery_event": { 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_pg_recovery_event", 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.765 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.766 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.767 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.768 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.769 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.770 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.771 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.772 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.773 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.774 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.775 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.776 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "max_completed_events": { 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_completed_events", 2024-09-05T12:48:31.777 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "50", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "number of past completed events to remember", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.778 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.779 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long the module is going to sleep", 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.780 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus", 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.781 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "cache": { 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cache", 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.782 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "exclude_perf_counters": { 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "exclude_perf_counters", 2024-09-05T12:48:31.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T12:48:31.784 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.785 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.786 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.787 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.788 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.789 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.790 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.791 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.792 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools": { 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools", 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.794 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T12:48:31.795 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "300", 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.796 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_interval": { 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_interval", 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.797 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15.0", 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.798 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.799 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.800 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "9283", 2024-09-05T12:48:31.801 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.802 INFO:teuthology.orchestra.run.smithi071.stdout: "stale_cache_strategy": { 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stale_cache_strategy", 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "log", 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.803 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-09-05T12:48:31.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "default", 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "default", 2024-09-05T12:48:31.805 INFO:teuthology.orchestra.run.smithi071.stdout: "error" 2024-09-05T12:48:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.806 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.808 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.809 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.810 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.811 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.812 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.813 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.814 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.815 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.816 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.817 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.818 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_snap_create": { 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_snap_create", 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.819 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.820 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "mirror_snapshot_schedule": { 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.821 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.822 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "trash_purge_schedule": { 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "trash_purge_schedule", 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.823 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.824 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "restful", 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_auth": { 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_auth", 2024-09-05T12:48:31.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.826 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.827 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.828 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.829 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.830 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.831 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.832 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.833 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.834 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.835 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.836 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.837 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.838 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:31.839 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.840 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.841 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rgw", 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.842 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.843 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.845 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.847 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.848 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.849 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.850 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rook", 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.851 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.852 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.853 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.854 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.855 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.856 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.857 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.858 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.859 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "storage_class": { 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "storage_class", 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.860 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "local", 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.861 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "selftest", 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.862 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.863 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.864 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.865 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.866 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.868 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.869 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.870 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "roption1": { 2024-09-05T12:48:31.871 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption1", 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.872 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "roption2": { 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption2", 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.873 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "xyz", 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.874 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption1": { 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption1", 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.875 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption2": { 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption2", 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.877 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption3": { 2024-09-05T12:48:31.878 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption3", 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.879 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption4": { 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption4", 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.880 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.881 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption5": { 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption5", 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.882 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.883 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption6": { 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption6", 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.884 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption7": { 2024-09-05T12:48:31.885 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption7", 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "42", 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.886 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "testkey": { 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testkey", 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.887 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.888 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "testlkey": { 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testlkey", 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.889 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.890 INFO:teuthology.orchestra.run.smithi071.stdout: "testnewline": { 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testnewline", 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.891 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_m_granularity": { 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_m_granularity", 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.893 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.894 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "dump_on_update": { 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dump_on_update", 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.895 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "dump database to debug log on update", 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.896 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.897 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.898 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.899 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.900 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.901 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.902 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.903 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.904 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.905 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stats", 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.906 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.907 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.908 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.909 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.910 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.911 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.912 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.913 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.914 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "status", 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.915 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.916 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.917 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.919 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.920 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.921 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.922 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.923 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.924 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telegraf", 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "address": { 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "address", 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.925 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.926 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.927 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15", 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.928 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.929 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.930 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.931 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.932 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.933 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.934 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.936 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telemetry", 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_basic": { 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_basic", 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.938 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.939 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_crash": { 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_crash", 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.940 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.941 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_device": { 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_device", 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.942 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_ident": { 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_ident", 2024-09-05T12:48:31.943 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T12:48:31.944 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_perf": { 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_perf", 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.945 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.946 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "contact": { 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "contact", 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.947 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "description": { 2024-09-05T12:48:31.948 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "description", 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.949 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "device_url": { 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_url", 2024-09-05T12:48:31.950 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.951 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.953 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "24", 2024-09-05T12:48:31.954 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "8", 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "last_opt_revision": { 2024-09-05T12:48:31.955 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "last_opt_revision", 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1", 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.956 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard": { 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard", 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.957 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.958 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard_description": { 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard_description", 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.959 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.961 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.962 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.963 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.964 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.965 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.966 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.967 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.968 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "organization": { 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "organization", 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.969 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.970 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "proxy": { 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "proxy", 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.971 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "url": { 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url", 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.973 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.974 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "test_orchestrator", 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.975 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.976 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.977 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.978 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.979 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.980 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.981 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.993 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_clones": { 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_clones", 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4", 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.994 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "periodic_async_work": { 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "periodic_async_work", 2024-09-05T12:48:31.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Periodically check for async work", 2024-09-05T12:48:31.996 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_delay": { 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_delay", 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:31.997 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:31.998 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_no_wait": { 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:31.999 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.000 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix", 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.001 INFO:teuthology.orchestra.run.smithi071.stdout: "discovery_interval": { 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "discovery_interval", 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "100", 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.002 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "identifier": { 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "identifier", 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.003 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.004 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.005 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.007 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.008 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.010 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.011 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.012 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.013 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.014 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_host": { 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_host", 2024-09-05T12:48:32.015 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.016 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_port": { 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_port", 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.017 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10051", 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.018 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_sender": { 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_sender", 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.019 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:32.020 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "gid": 25208, 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "y", 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "mgr_features": 4540701547738038271, 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "available_modules": [ 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "alerts", 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.021 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.022 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.023 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.024 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.025 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.026 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.028 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.029 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.030 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.031 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_destination": { 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_destination", 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.032 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email address to send alerts to", 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_from_name": { 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_from_name", 2024-09-05T12:48:32.033 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "Ceph", 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email From: name", 2024-09-05T12:48:32.034 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_host": { 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_host", 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.035 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP server", 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_password": { 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_password", 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.037 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Password to authenticate with", 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_port": { 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_port", 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.038 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "465", 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP port", 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.039 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_sender": { 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_sender", 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.040 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP envelope sender", 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_ssl": { 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_ssl", 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.042 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_user": { 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_user", 2024-09-05T12:48:32.043 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "User to authenticate as", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.044 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "balancer", 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.045 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "active": { 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "active", 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.046 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_time": { 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_time", 2024-09-05T12:48:32.047 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0000", 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T12:48:32.048 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_weekday": { 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_weekday", 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.049 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.050 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_max_iterations": { 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_max_iterations", 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "25", 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:32.051 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "250", 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_metrics": { 2024-09-05T12:48:32.052 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_metrics", 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.053 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_step": { 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_step", 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.054 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.5", 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0.001", 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "0.999", 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "aggressiveness of optimization", 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.055 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "end_time": { 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_time", 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2359", 2024-09-05T12:48:32.056 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.057 INFO:teuthology.orchestra.run.smithi071.stdout: "end_weekday": { 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_weekday", 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-09-05T12:48:32.058 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.059 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.060 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.061 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.062 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.063 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.064 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.065 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.066 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.067 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "min_score": { 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_score", 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.068 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.069 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "upmap", 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.070 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-compat", 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "none", 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "read", 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap", 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap-read" 2024-09-05T12:48:32.071 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Balancer mode", 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_ids": { 2024-09-05T12:48:32.072 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_ids", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T12:48:32.073 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.074 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.075 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_deviation": { 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_deviation", 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.076 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_optimizations": { 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_optimizations", 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.077 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.078 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm", 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.079 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_down_multiplier": { 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_down_multiplier", 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.080 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_refresh_rate": { 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_refresh_rate", 2024-09-05T12:48:32.081 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "20", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.082 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_starting_port": { 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_starting_port", 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4721", 2024-09-05T12:48:32.083 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_ptrace": { 2024-09-05T12:48:32.084 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_ptrace", 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.085 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_interval": { 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_interval", 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.086 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.087 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_memory_target_ratio": { 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.7", 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.088 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_log_destination": { 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm_log_destination", 2024-09-05T12:48:32.089 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "file", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "file,syslog", 2024-09-05T12:48:32.090 INFO:teuthology.orchestra.run.smithi071.stdout: "syslog" 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "cgroups_split": { 2024-09-05T12:48:32.091 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cgroups_split", 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.092 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "config_checks_enabled": { 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_checks_enabled", 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.093 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.094 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "config_dashboard": { 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_dashboard", 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.095 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_alertmanager": { 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_alertmanager", 2024-09-05T12:48:32.096 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.097 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_base": { 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_base", 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.098 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Container image name, without the tag", 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.099 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_elasticsearch": { 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_elasticsearch", 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.100 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "elasticsearch container image", 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.101 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_grafana": { 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_grafana", 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.102 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.103 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_haproxy": { 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_haproxy", 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.104 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "HAproxy container image", 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.105 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_agent": { 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_agent", 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T12:48:32.106 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger agent container image", 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_collector": { 2024-09-05T12:48:32.107 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_collector", 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.108 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger collector container image", 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_query": { 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_query", 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.109 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger query container image", 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_keepalived": { 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_keepalived", 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.111 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Keepalived container image", 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_loki": { 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_loki", 2024-09-05T12:48:32.112 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.113 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Loki container image", 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_node_exporter": { 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_node_exporter", 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.114 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.115 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_nvmeof": { 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_nvmeof", 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.116 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Nvme-of container image", 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_prometheus": { 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_prometheus", 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.117 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.118 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_promtail": { 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_promtail", 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.119 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Promtail container image", 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_samba": { 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_samba", 2024-09-05T12:48:32.120 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Samba/SMB container image", 2024-09-05T12:48:32.121 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_snmp_gateway": { 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_snmp_gateway", 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T12:48:32.122 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SNMP Gateway container image", 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "container_init": { 2024-09-05T12:48:32.123 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_init", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T12:48:32.124 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "daemon_cache_timeout": { 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "daemon_cache_timeout", 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.125 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "default_cephadm_command_timeout": { 2024-09-05T12:48:32.126 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "900", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T12:48:32.127 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "default_registry": { 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_registry", 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.128 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io", 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.129 INFO:teuthology.orchestra.run.smithi071.stdout: "device_cache_timeout": { 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_cache_timeout", 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1800", 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.130 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache device inventory", 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "device_enhanced_scan": { 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_enhanced_scan", 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.131 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "facts_cache_timeout": { 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "facts_cache_timeout", 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.133 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache host facts data", 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "grafana_dashboards_path": { 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "grafana_dashboards_path", 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.134 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.135 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "host_check_interval": { 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "host_check_interval", 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.136 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to perform a host check", 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "hw_monitoring": { 2024-09-05T12:48:32.137 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hw_monitoring", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T12:48:32.138 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "inventory_list_all": { 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "inventory_list_all", 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.139 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.140 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.141 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.142 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "log_refresh_metadata": { 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_refresh_metadata", 2024-09-05T12:48:32.143 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T12:48:32.144 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.145 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.147 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.148 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.149 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.151 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T12:48:32.152 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "*", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.153 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "max_count_per_host": { 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_count_per_host", 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.154 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of daemons per service per host", 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd_draining_count": { 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_osd_draining_count", 2024-09-05T12:48:32.155 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T12:48:32.156 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "migration_current": { 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "migration_current", 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.157 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "internal - do not modify", 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.158 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "root", 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.159 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm-package", 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "root" 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "oob_default_addr": { 2024-09-05T12:48:32.160 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "oob_default_addr", 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "169.254.1.1", 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.161 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "prometheus_alerts_path": { 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus_alerts_path", 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.162 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.163 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_insecure": { 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_insecure", 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.164 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_password": { 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_password", 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.165 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.166 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_url": { 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_url", 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.167 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_username": { 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_username", 2024-09-05T12:48:32.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.169 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "secure_monitoring_stack": { 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "secure_monitoring_stack", 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.170 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.171 INFO:teuthology.orchestra.run.smithi071.stdout: "service_discovery_port": { 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "service_discovery_port", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8765", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "cephadm service discovery port", 2024-09-05T12:48:32.172 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "ssh_config_file": { 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssh_config_file", 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.173 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.174 INFO:teuthology.orchestra.run.smithi071.stdout: "use_agent": { 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_agent", 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.175 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "use_repo_digest": { 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_repo_digest", 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.176 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.177 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_failed_host_check": { 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_failed_host_check", 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.178 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_daemons": { 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_daemons", 2024-09-05T12:48:32.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.180 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_hosts": { 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_hosts", 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.181 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.182 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crash", 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.183 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.184 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.185 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.186 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.187 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.188 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.189 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.190 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "retain_interval": { 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retain_interval", 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.191 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "31536000", 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_recent_interval": { 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_recent_interval", 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1209600", 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.193 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.194 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dashboard", 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.195 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.197 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T12:48:32.198 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.199 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_ENABLED": { 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.200 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.201 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.202 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.203 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.204 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.205 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.207 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T12:48:32.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.209 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.211 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.212 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T12:48:32.213 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.214 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.215 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.216 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.217 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.218 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.219 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.220 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.221 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_URL": { 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_URL", 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.222 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T12:48:32.223 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.224 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.225 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T12:48:32.226 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.227 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.228 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.229 INFO:teuthology.orchestra.run.smithi071.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.230 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.231 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.232 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.233 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.234 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.235 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.236 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T12:48:32.237 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.238 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.239 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T12:48:32.240 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.241 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.242 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.243 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.244 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.245 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.246 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.247 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T12:48:32.248 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.249 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8", 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.250 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T12:48:32.251 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "45", 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.252 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.253 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.254 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.255 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.256 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.257 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.258 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T12:48:32.259 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.260 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.261 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.262 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.263 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "cross_origin_url": { 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cross_origin_url", 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.264 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.265 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "crt_file": { 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crt_file", 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.266 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "debug": { 2024-09-05T12:48:32.267 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "debug", 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.268 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable/disable debug options", 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "jwt_token_ttl": { 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "jwt_token_ttl", 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.269 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "28800", 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.270 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.271 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.272 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.273 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.274 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.275 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.276 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.277 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.278 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.279 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "motd": { 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "motd", 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.281 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "The message of the day", 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect_resolve_ip_addr": { 2024-09-05T12:48:32.282 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.283 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.284 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8080", 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.286 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.287 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl_server_port": { 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl_server_port", 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8443", 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.289 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.290 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "redirect", 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect" 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.291 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "url_prefix": { 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url_prefix", 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.294 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.295 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_monitoring": { 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_monitoring", 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.296 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "monitor device health metrics", 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.297 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.298 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.299 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.300 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.301 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.302 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.303 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.304 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "mark_out_threshold": { 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mark_out_threshold", 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2419200", 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.306 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": { 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_name", 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.307 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "device_health_metrics", 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.308 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "retention_period": { 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retention_period", 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15552000", 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.309 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain device health metrics", 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_frequency": { 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_frequency", 2024-09-05T12:48:32.310 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.311 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "self_heal": { 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "self_heal", 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.312 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:32.313 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.314 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_threshold": { 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_threshold", 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.315 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7257600", 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.316 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "diskprediction_local", 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.317 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.318 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.319 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.320 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.321 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.322 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.323 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.324 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.325 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "predict_interval": { 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predict_interval", 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.326 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "predictor_model": { 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predictor_model", 2024-09-05T12:48:32.327 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "prophetstor", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.328 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.329 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "influx", 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": false, 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "influxdb python module not found", 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "batch_size": { 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "batch_size", 2024-09-05T12:48:32.331 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5000", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.332 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "database": { 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "database", 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "ceph", 2024-09-05T12:48:32.333 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": { 2024-09-05T12:48:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hostname", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server hostname", 2024-09-05T12:48:32.335 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.336 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "30", 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "5", 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.337 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.338 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.339 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.340 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.341 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.342 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.343 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.344 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "password": { 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "password", 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.345 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "password of InfluxDB server user", 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.346 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "port": { 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "port", 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8086", 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.347 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server port", 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.348 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "false", 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.349 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "threads": { 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threads", 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:32.350 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "32", 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "username": { 2024-09-05T12:48:32.351 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "username", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "username of InfluxDB server user", 2024-09-05T12:48:32.352 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "verify_ssl": { 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "verify_ssl", 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.353 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "true", 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.354 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "insights", 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.356 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.358 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.359 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.360 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.361 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.362 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "iostat", 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.363 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.364 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.365 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.366 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.367 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.367 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.367 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.367 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.367 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.368 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.369 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.370 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.371 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "k8sevents", 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_event_retention_days": { 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ceph_event_retention_days", 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.373 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7", 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.374 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "config_check_secs": { 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_check_secs", 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.375 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "10", 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.376 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.377 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.378 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.379 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.380 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.381 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.382 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.383 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.384 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "localpool", 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "failure_domain": { 2024-09-05T12:48:32.385 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "failure_domain", 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "host", 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.386 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "failure domain for any created local pool", 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.387 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.388 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.389 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.390 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.391 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.394 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": { 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_size", 2024-09-05T12:48:32.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default min_size for any created local pool", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T12:48:32.396 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "num_rep": { 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "num_rep", 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3", 2024-09-05T12:48:32.397 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default replica count for any created local pool", 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": { 2024-09-05T12:48:32.398 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_num", 2024-09-05T12:48:32.399 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.399 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.399 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.399 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "128", 2024-09-05T12:48:32.399 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.399 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "prefix": { 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prefix", 2024-09-05T12:48:32.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name prefix for any created local pool", 2024-09-05T12:48:32.401 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "subtree": { 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subtree", 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.402 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "rack", 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.403 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mds_autoscaler", 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.404 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.405 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.406 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.407 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.408 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.409 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.410 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.411 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirroring", 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.414 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.415 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.416 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.417 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.418 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.419 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.420 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.421 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "nfs", 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.422 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.423 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.424 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.425 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.427 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.428 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.430 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.431 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "fail_fs": { 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "fail_fs", 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.432 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.433 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.434 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.435 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.436 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.437 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.438 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.439 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.440 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.441 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator": { 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.442 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: "rook", 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: "test_orchestrator" 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Orchestrator backend", 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.443 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_perf_query", 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.444 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.445 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.446 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.447 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.448 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.450 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.451 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.452 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_support", 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.453 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.454 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.455 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.456 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.457 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.458 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.459 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.460 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.461 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_autoscaler", 2024-09-05T12:48:32.462 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.463 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.464 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.465 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.466 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.467 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.468 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.469 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.470 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.471 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "threshold": { 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threshold", 2024-09-05T12:48:32.472 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1.0", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "scaling threshold", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T12:48:32.473 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "progress", 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.474 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_pg_recovery_event": { 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_pg_recovery_event", 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.475 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.476 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.477 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.478 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.479 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.480 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.481 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.482 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.483 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.484 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.485 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "max_completed_events": { 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_completed_events", 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "50", 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.486 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "number of past completed events to remember", 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:32.487 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long the module is going to sleep", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.488 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus", 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.489 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "cache": { 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cache", 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.490 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "exclude_perf_counters": { 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "exclude_perf_counters", 2024-09-05T12:48:32.491 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T12:48:32.492 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.493 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.494 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.495 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.496 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.497 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.498 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.499 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.500 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.501 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools": { 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.502 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.503 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "300", 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.504 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_interval": { 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_interval", 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15.0", 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.505 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.506 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.507 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "9283", 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.508 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "stale_cache_strategy": { 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stale_cache_strategy", 2024-09-05T12:48:32.509 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "log", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.510 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.511 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "default", 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "default", 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "error" 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.512 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-09-05T12:48:32.513 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.514 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.515 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.516 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.517 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.518 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.519 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.520 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.522 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_snap_create": { 2024-09-05T12:48:32.523 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_snap_create", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.524 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: "mirror_snapshot_schedule": { 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.525 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.526 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.526 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.526 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.526 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.526 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.526 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: "trash_purge_schedule": { 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "trash_purge_schedule", 2024-09-05T12:48:32.527 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.528 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.529 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "restful", 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_auth": { 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_auth", 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.530 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.531 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.531 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.531 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.531 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.531 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.531 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-09-05T12:48:32.532 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.533 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.534 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.535 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.536 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.537 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.538 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.539 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.540 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.541 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:32.542 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.543 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.544 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.545 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rgw", 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.546 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.547 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.548 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.549 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.550 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.551 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.552 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.553 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.554 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rook", 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.555 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.556 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.557 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.558 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.559 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.560 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.561 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.562 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.563 INFO:teuthology.orchestra.run.smithi071.stdout: "storage_class": { 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "storage_class", 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "local", 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.564 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "selftest", 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.566 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.567 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.568 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.569 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.570 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.571 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.572 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.573 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "roption1": { 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption1", 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.574 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.575 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "roption2": { 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption2", 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "xyz", 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.576 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption1": { 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption1", 2024-09-05T12:48:32.577 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.578 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption2": { 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption2", 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.579 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.580 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption3": { 2024-09-05T12:48:32.581 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption3", 2024-09-05T12:48:32.581 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.581 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.581 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.581 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.581 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.582 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption4": { 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption4", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.583 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption5": { 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption5", 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.584 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption6": { 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption6", 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.586 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption7": { 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption7", 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.587 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "42", 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "testkey": { 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testkey", 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.589 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "testlkey": { 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testlkey", 2024-09-05T12:48:32.590 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.591 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "testnewline": { 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testnewline", 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.592 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.593 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.594 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_m_granularity": { 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_m_granularity", 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.595 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "dump_on_update": { 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dump_on_update", 2024-09-05T12:48:32.596 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "dump database to debug log on update", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.597 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.598 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.599 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.600 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.601 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.602 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.603 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.604 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.605 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.606 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stats", 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.607 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.608 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.609 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.610 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.611 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.612 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.613 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.614 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.615 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "status", 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.616 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.617 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.618 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.619 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.620 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.621 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.622 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.623 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.624 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.624 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.624 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.624 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.624 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.624 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.625 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telegraf", 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "address": { 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "address", 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.626 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.627 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15", 2024-09-05T12:48:32.628 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.629 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.630 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.631 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.632 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.633 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.634 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.635 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.636 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.637 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telemetry", 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_basic": { 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_basic", 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.638 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.639 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_crash": { 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_crash", 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.640 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_device": { 2024-09-05T12:48:32.641 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_device", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T12:48:32.642 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_ident": { 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_ident", 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.643 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.644 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_perf": { 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_perf", 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.645 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "contact": { 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "contact", 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.646 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.647 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "description": { 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "description", 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.648 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "device_url": { 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_url", 2024-09-05T12:48:32.649 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.650 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.651 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.652 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "24", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "8", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.653 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "last_opt_revision": { 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "last_opt_revision", 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.654 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1", 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.655 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard": { 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard", 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.656 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.657 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard_description": { 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard_description", 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.658 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.659 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.660 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.661 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.662 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.663 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.664 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.665 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.666 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "organization": { 2024-09-05T12:48:32.667 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "organization", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.668 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "proxy": { 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "proxy", 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.669 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.670 INFO:teuthology.orchestra.run.smithi071.stdout: "url": { 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url", 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.671 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.672 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "test_orchestrator", 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.673 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.674 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.675 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.676 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.677 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.678 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.679 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.680 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.681 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.682 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.683 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.684 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.685 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.686 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.687 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.688 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.689 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_clones": { 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_clones", 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4", 2024-09-05T12:48:32.690 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "periodic_async_work": { 2024-09-05T12:48:32.691 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "periodic_async_work", 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.692 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Periodically check for async work", 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_delay": { 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_delay", 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.693 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.694 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_no_wait": { 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.695 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.696 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix", 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "discovery_interval": { 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "discovery_interval", 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.697 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "100", 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.698 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "identifier": { 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "identifier", 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.699 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.700 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.701 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.702 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.703 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.704 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.705 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.706 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.707 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.709 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_host": { 2024-09-05T12:48:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_host", 2024-09-05T12:48:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.711 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.712 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_port": { 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_port", 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.713 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10051", 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.714 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_sender": { 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_sender", 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.715 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.716 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "modules": [ 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "dashboard", 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "iostat", 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "nfs", 2024-09-05T12:48:32.717 INFO:teuthology.orchestra.run.smithi071.stdout: "restful", 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "snap_schedule" 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "available_modules": [ 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "alerts", 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.718 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.719 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How frequently to reexamine health status", 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.720 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.721 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.722 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.723 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.724 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.725 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.726 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.727 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.728 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_destination": { 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_destination", 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.729 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email address to send alerts to", 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_from_name": { 2024-09-05T12:48:32.730 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_from_name", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "Ceph", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Email From: name", 2024-09-05T12:48:32.731 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_host": { 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_host", 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.732 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP server", 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.733 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_password": { 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_password", 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.734 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Password to authenticate with", 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_port": { 2024-09-05T12:48:32.735 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_port", 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "465", 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.736 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP port", 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_sender": { 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_sender", 2024-09-05T12:48:32.737 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SMTP envelope sender", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.738 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_ssl": { 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_ssl", 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "smtp_user": { 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "smtp_user", 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.741 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "User to authenticate as", 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.742 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "balancer", 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "active": { 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "active", 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.743 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically balance PGs across cluster", 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.744 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_time": { 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_time", 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0000", 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.745 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "beginning time of day to automatically balance", 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "begin_weekday": { 2024-09-05T12:48:32.746 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "begin_weekday", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-05T12:48:32.747 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_max_iterations": { 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_max_iterations", 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.748 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "25", 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "250", 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.749 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_metrics": { 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_metrics", 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "pgs,objects,bytes", 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.750 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "crush_compat_step": { 2024-09-05T12:48:32.751 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crush_compat_step", 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.5", 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0.001", 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "0.999", 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.752 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "aggressiveness of optimization", 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "end_time": { 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_time", 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.753 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2359", 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ending time of day to automatically balance", 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.754 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "end_weekday": { 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "end_weekday", 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.755 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "0", 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "6", 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.756 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.757 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.758 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.759 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.760 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.761 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.762 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.763 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.764 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "min_score": { 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_score", 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.765 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-09-05T12:48:32.766 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "upmap", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "crush-compat", 2024-09-05T12:48:32.767 INFO:teuthology.orchestra.run.smithi071.stdout: "none", 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "read", 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap", 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap-read" 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Balancer mode", 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.768 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_ids": { 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_ids", 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.769 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.770 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.771 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_deviation": { 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_deviation", 2024-09-05T12:48:32.772 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-05T12:48:32.773 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "upmap_max_optimizations": { 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "upmap_max_optimizations", 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.774 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.775 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm", 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_down_multiplier": { 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_down_multiplier", 2024-09-05T12:48:32.776 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.777 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_refresh_rate": { 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_refresh_rate", 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "20", 2024-09-05T12:48:32.778 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "agent_starting_port": { 2024-09-05T12:48:32.779 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "agent_starting_port", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4721", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-05T12:48:32.780 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_ptrace": { 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_ptrace", 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.781 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_interval": { 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_interval", 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.783 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "autotune_memory_target_ratio": { 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "autotune_memory_target_ratio", 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:32.784 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0.7", 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.785 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm_log_destination": { 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cephadm_log_destination", 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.786 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "file", 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "file,syslog", 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "syslog" 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.787 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "cgroups_split": { 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cgroups_split", 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.788 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "config_checks_enabled": { 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_checks_enabled", 2024-09-05T12:48:32.789 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.790 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "config_dashboard": { 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_dashboard", 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.791 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.792 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_alertmanager": { 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_alertmanager", 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.793 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_base": { 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_base", 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.794 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Container image name, without the tag", 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.795 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_elasticsearch": { 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_elasticsearch", 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-05T12:48:32.796 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "elasticsearch container image", 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_grafana": { 2024-09-05T12:48:32.797 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_grafana", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.798 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_haproxy": { 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_haproxy", 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.799 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "HAproxy container image", 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.800 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_agent": { 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_agent", 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.801 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger agent container image", 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_collector": { 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_collector", 2024-09-05T12:48:32.802 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger collector container image", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.803 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_jaeger_query": { 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_jaeger_query", 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-05T12:48:32.804 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Jaeger query container image", 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.805 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_keepalived": { 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_keepalived", 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.806 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Keepalived container image", 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_loki": { 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_loki", 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.807 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Loki container image", 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.808 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_node_exporter": { 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_node_exporter", 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.809 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_nvmeof": { 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_nvmeof", 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.810 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Nvme-of container image", 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.811 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_prometheus": { 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_prometheus", 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-05T12:48:32.812 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Prometheus container image", 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_promtail": { 2024-09-05T12:48:32.813 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_promtail", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Promtail container image", 2024-09-05T12:48:32.814 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_samba": { 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_samba", 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.815 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Samba/SMB container image", 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.816 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "container_image_snmp_gateway": { 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_image_snmp_gateway", 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.817 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "SNMP Gateway container image", 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "container_init": { 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "container_init", 2024-09-05T12:48:32.818 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Run podman/docker with `--init`", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.819 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "daemon_cache_timeout": { 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "daemon_cache_timeout", 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.820 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.821 INFO:teuthology.orchestra.run.smithi071.stdout: "default_cephadm_command_timeout": { 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_cephadm_command_timeout", 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "900", 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.822 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "default_registry": { 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "default_registry", 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.823 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "docker.io", 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.824 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "device_cache_timeout": { 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_cache_timeout", 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1800", 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.825 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache device inventory", 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "device_enhanced_scan": { 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_enhanced_scan", 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.826 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.827 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "facts_cache_timeout": { 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "facts_cache_timeout", 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:32.828 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "seconds to cache host facts data", 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.829 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "grafana_dashboards_path": { 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "grafana_dashboards_path", 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.830 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "host_check_interval": { 2024-09-05T12:48:32.831 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "host_check_interval", 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.832 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to perform a host check", 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "hw_monitoring": { 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hw_monitoring", 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.833 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.834 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "inventory_list_all": { 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "inventory_list_all", 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.835 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.836 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.837 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.838 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "log_refresh_metadata": { 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_refresh_metadata", 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.839 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.840 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.841 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.842 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.843 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.844 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.845 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-05T12:48:32.846 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.847 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.848 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "*", 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.849 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "max_count_per_host": { 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_count_per_host", 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.850 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of daemons per service per host", 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "max_osd_draining_count": { 2024-09-05T12:48:32.851 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_osd_draining_count", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-05T12:48:32.852 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "migration_current": { 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "migration_current", 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.853 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "internal - do not modify", 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.854 INFO:teuthology.orchestra.run.smithi071.stdout: "mode": { 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mode", 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "root", 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.855 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm-package", 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: "root" 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "mode for remote execution of cephadm", 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.856 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "oob_default_addr": { 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "oob_default_addr", 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "169.254.1.1", 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.857 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "prometheus_alerts_path": { 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus_alerts_path", 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.858 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.859 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_insecure": { 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_insecure", 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.860 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.861 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_password": { 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_password", 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.862 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_url": { 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_url", 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.863 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.864 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "registry_username": { 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "registry_username", 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.865 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "secure_monitoring_stack": { 2024-09-05T12:48:32.866 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "secure_monitoring_stack", 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.867 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "service_discovery_port": { 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "service_discovery_port", 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.868 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8765", 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "cephadm service discovery port", 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.869 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "ssh_config_file": { 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssh_config_file", 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.870 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "use_agent": { 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_agent", 2024-09-05T12:48:32.871 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-05T12:48:32.872 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "use_repo_digest": { 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "use_repo_digest", 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.873 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.874 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_failed_host_check": { 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_failed_host_check", 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.875 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if the host check fails", 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_daemons": { 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_daemons", 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.876 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.877 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_on_stray_hosts": { 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_on_stray_hosts", 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.878 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.879 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crash", 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.880 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.881 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.882 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.883 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.884 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.885 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.886 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.887 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "retain_interval": { 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retain_interval", 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.888 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "31536000", 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.889 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_recent_interval": { 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_recent_interval", 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1209600", 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.890 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.891 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dashboard", 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-05T12:48:32.892 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.893 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.895 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_HOST": { 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.896 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.897 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.898 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_ENABLED": { 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_ENABLED", 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.899 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.900 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.901 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "ENABLE_BROWSABLE_API": { 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-05T12:48:32.902 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.903 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.904 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.905 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.906 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.907 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.908 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.909 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.910 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.911 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.912 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.913 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-05T12:48:32.914 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.915 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.916 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.917 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_PASSWORD": { 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.918 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-05T12:48:32.919 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.920 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_URL": { 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_URL", 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.921 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.922 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_API_USERNAME": { 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.923 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-05T12:48:32.924 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.925 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.926 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-05T12:48:32.927 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.928 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.929 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.930 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_HOST": { 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.931 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-05T12:48:32.932 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.933 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.934 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-05T12:48:32.935 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.936 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.937 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.938 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.939 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-05T12:48:32.940 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.941 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.942 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.943 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.944 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_ENABLED": { 2024-09-05T12:48:32.945 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.946 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.947 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.948 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.949 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-05T12:48:32.950 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8", 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.951 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.952 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "45", 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.953 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ACCESS_KEY": { 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.954 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.955 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "admin", 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.956 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SECRET_KEY": { 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.957 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "RGW_API_SSL_VERIFY": { 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.958 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.959 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.960 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.961 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.962 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.963 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "cross_origin_url": { 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cross_origin_url", 2024-09-05T12:48:32.964 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "crt_file": { 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "crt_file", 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.966 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.967 INFO:teuthology.orchestra.run.smithi071.stdout: "debug": { 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "debug", 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.968 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Enable/disable debug options", 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "jwt_token_ttl": { 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "jwt_token_ttl", 2024-09-05T12:48:32.969 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "28800", 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.970 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.971 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.972 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.973 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.974 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.975 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:32.976 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:32.977 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.979 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "motd": { 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "motd", 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.980 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "The message of the day", 2024-09-05T12:48:32.981 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect_resolve_ip_addr": { 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "redirect_resolve_ip_addr", 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.982 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.983 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.984 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.985 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8080", 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.986 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.987 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.988 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl_server_port": { 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl_server_port", 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.989 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8443", 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.990 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "redirect", 2024-09-05T12:48:32.991 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "redirect" 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.992 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.993 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.994 INFO:teuthology.orchestra.run.smithi071.stdout: "url_prefix": { 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url_prefix", 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.995 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:32.996 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_monitoring": { 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_monitoring", 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.997 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "monitor device health metrics", 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:32.998 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:32.999 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:32.999 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:32.999 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:32.999 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:32.999 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:32.999 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.000 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.001 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.002 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.003 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.004 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.005 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.006 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.007 INFO:teuthology.orchestra.run.smithi071.stdout: "mark_out_threshold": { 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mark_out_threshold", 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "2419200", 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.008 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "pool_name": { 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pool_name", 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.009 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "device_health_metrics", 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.010 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "retention_period": { 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "retention_period", 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15552000", 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.011 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long to retain device health metrics", 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_frequency": { 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_frequency", 2024-09-05T12:48:33.012 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-05T12:48:33.013 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "self_heal": { 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "self_heal", 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.014 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.015 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.016 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how frequently to wake up and check device health", 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "warn_threshold": { 2024-09-05T12:48:33.017 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "warn_threshold", 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7257600", 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.018 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.019 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "diskprediction_local", 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.020 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.021 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.022 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.023 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.024 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.025 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.026 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.027 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.028 INFO:teuthology.orchestra.run.smithi071.stdout: "predict_interval": { 2024-09-05T12:48:33.029 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predict_interval", 2024-09-05T12:48:33.029 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.029 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.029 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.029 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "86400", 2024-09-05T12:48:33.029 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.030 INFO:teuthology.orchestra.run.smithi071.stdout: "predictor_model": { 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "predictor_model", 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "prophetstor", 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.031 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:33.032 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "600", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.033 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "influx", 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": false, 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "influxdb python module not found", 2024-09-05T12:48:33.034 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "batch_size": { 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "batch_size", 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5000", 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.035 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "database": { 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "database", 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.036 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "ceph", 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.037 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "hostname": { 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "hostname", 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.038 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server hostname", 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.039 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "30", 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "5", 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.040 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.041 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.042 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.043 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.044 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.045 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.046 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.047 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.048 INFO:teuthology.orchestra.run.smithi071.stdout: "password": { 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "password", 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.049 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "password of InfluxDB server user", 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "port": { 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "port", 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.050 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "8086", 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "InfluxDB server port", 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.051 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "ssl": { 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ssl", 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "false", 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.052 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "threads": { 2024-09-05T12:48:33.053 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threads", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "32", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-05T12:48:33.054 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "username": { 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "username", 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.055 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "username of InfluxDB server user", 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.056 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "verify_ssl": { 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "verify_ssl", 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "true", 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.057 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.058 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "insights", 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.059 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.060 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.061 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.062 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.063 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.064 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.065 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.066 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "iostat", 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.067 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.068 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.069 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.070 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.071 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.072 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.073 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.074 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.075 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "k8sevents", 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_event_retention_days": { 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "ceph_event_retention_days", 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.076 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "7", 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.077 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "config_check_secs": { 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "config_check_secs", 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "10", 2024-09-05T12:48:33.078 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.079 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.080 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.081 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.082 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.083 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.084 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.085 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.086 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "localpool", 2024-09-05T12:48:33.087 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "failure_domain": { 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "failure_domain", 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.088 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "host", 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "failure domain for any created local pool", 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.089 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.090 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.091 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.092 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.093 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.094 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.095 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.096 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "min_size": { 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "min_size", 2024-09-05T12:48:33.097 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.099 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default min_size for any created local pool", 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.100 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "num_rep": { 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "num_rep", 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3", 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.101 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default replica count for any created local pool", 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_num": { 2024-09-05T12:48:33.102 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_num", 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "128", 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.103 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "default pg_num for any created local pool", 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "prefix": { 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prefix", 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.104 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "name prefix for any created local pool", 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.105 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "subtree": { 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "subtree", 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "rack", 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.106 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.107 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mds_autoscaler", 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.108 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.109 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.110 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.111 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.112 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.113 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.114 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.115 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirroring", 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.116 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.117 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.118 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.119 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.120 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.121 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.122 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.123 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.124 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "nfs", 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.125 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.126 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.127 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.128 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.129 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.130 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.131 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.132 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.133 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.134 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "fail_fs": { 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "fail_fs", 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.135 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.136 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.137 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.138 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.139 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.140 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.141 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.142 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.143 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.144 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator": { 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "orchestrator", 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.145 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "cephadm", 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "rook", 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "test_orchestrator" 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Orchestrator backend", 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.146 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_perf_query", 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.147 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.148 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.149 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.150 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.151 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.152 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.153 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.154 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.155 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "osd_support", 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.156 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.157 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.158 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.159 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.160 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.161 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.162 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.163 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.164 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "pg_autoscaler", 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.165 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.166 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.167 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.168 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.169 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.170 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.171 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.172 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.173 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.174 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "threshold": { 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "threshold", 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "3.0", 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1.0", 2024-09-05T12:48:33.175 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "scaling threshold", 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.176 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "progress", 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_pg_recovery_event": { 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_pg_recovery_event", 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.177 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.178 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.179 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.180 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.181 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.182 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.183 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.184 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.185 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.186 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.187 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.188 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "max_completed_events": { 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_completed_events", 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "50", 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.189 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "number of past completed events to remember", 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "sleep_interval": { 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "sleep_interval", 2024-09-05T12:48:33.190 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "5", 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "how long the module is going to sleep", 2024-09-05T12:48:33.191 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "prometheus", 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.192 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "cache": { 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "cache", 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.193 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "exclude_perf_counters": { 2024-09-05T12:48:33.194 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "exclude_perf_counters", 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.195 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.196 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.197 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.199 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.200 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.201 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.202 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.203 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.204 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools": { 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools", 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.205 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.206 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "300", 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.207 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "scrape_interval": { 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "scrape_interval", 2024-09-05T12:48:33.208 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15.0", 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.209 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.210 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "::", 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.211 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "9283", 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.212 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "stale_cache_strategy": { 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stale_cache_strategy", 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.213 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "log", 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.214 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_behaviour": { 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_behaviour", 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "default", 2024-09-05T12:48:33.215 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "default", 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "error" 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "standby_error_status_code": { 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "standby_error_status_code", 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "500", 2024-09-05T12:48:33.217 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "400", 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "599", 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.218 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.219 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.220 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.221 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.222 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.223 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.224 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.225 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.227 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_snap_create": { 2024-09-05T12:48:33.228 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_snap_create", 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10", 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.229 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "mirror_snapshot_schedule": { 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "mirror_snapshot_schedule", 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.230 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.231 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "trash_purge_schedule": { 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "trash_purge_schedule", 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.232 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.233 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "restful", 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "enable_auth": { 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enable_auth", 2024-09-05T12:48:33.234 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.235 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "key_file": { 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "key_file", 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.236 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.237 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.238 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.239 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.240 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.241 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.242 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.243 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.244 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.245 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "server_addr": { 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_addr", 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.246 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "server_port": { 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "server_port", 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.247 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.248 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rgw", 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.249 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.250 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.251 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.252 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.253 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.254 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.255 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.256 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.257 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rook", 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.258 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.259 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.260 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.261 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.262 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.263 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.264 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.265 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "storage_class": { 2024-09-05T12:48:33.266 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "storage_class", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "local", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.267 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "selftest", 2024-09-05T12:48:33.268 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.269 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.270 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.271 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.272 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.273 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.274 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.275 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.276 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "roption1": { 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption1", 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.277 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.278 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "roption2": { 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "roption2", 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "xyz", 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.279 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption1": { 2024-09-05T12:48:33.280 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption1", 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.281 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption2": { 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption2", 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.282 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.283 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption3": { 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption3", 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "float", 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.284 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption4": { 2024-09-05T12:48:33.285 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption4", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.286 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption5": { 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption5", 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.287 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.288 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption6": { 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption6", 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.289 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "rwoption7": { 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rwoption7", 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.290 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "1", 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "42", 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.291 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "testkey": { 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testkey", 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.292 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "testlkey": { 2024-09-05T12:48:33.293 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testlkey", 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.294 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "testnewline": { 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "testnewline", 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.295 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.296 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.297 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "allow_m_granularity": { 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "allow_m_granularity", 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.298 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "allow minute scheduled snapshots", 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "dump_on_update": { 2024-09-05T12:48:33.299 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "dump_on_update", 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.300 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "dump database to debug log on update", 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.301 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.302 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.302 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.302 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.302 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.302 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.303 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.305 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.306 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.307 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.308 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.309 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.310 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "stats", 2024-09-05T12:48:33.311 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.312 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.313 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.314 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.315 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.316 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.317 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.319 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.320 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "status", 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.321 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.322 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.323 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.324 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.325 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.326 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.327 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.328 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.329 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telegraf", 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.330 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "address": { 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "address", 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.331 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:33.332 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "15", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.333 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.334 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.335 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.336 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.337 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.338 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.339 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.341 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "telemetry", 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_basic": { 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_basic", 2024-09-05T12:48:33.342 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-05T12:48:33.343 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_crash": { 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_crash", 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.344 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.345 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_device": { 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_device", 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.346 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_ident": { 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_ident", 2024-09-05T12:48:33.347 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-05T12:48:33.348 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "channel_perf": { 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "channel_perf", 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.349 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "contact": { 2024-09-05T12:48:33.350 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "contact", 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.351 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "description": { 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "description", 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.352 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.353 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "device_url": { 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "device_url", 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.354 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "enabled": { 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "enabled", 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.355 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.356 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "24", 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "8", 2024-09-05T12:48:33.357 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "last_opt_revision": { 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "last_opt_revision", 2024-09-05T12:48:33.358 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "1", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.359 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard": { 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard", 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.360 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: "leaderboard_description": { 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "leaderboard_description", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.362 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.363 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.364 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.365 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.366 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.367 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.368 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.369 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.370 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "organization": { 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "organization", 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.371 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "proxy": { 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "proxy", 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.372 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.373 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "url": { 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "url", 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.374 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.375 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "test_orchestrator", 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.376 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.377 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.378 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.379 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.380 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.381 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.382 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.383 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.384 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.385 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.386 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.387 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.388 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.389 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.390 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.391 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.392 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.393 INFO:teuthology.orchestra.run.smithi071.stdout: "max_concurrent_clones": { 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "max_concurrent_clones", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "4", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-05T12:48:33.394 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "periodic_async_work": { 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "periodic_async_work", 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.395 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Periodically check for async work", 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.396 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_delay": { 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_delay", 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "0", 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.397 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "snapshot_clone_no_wait": { 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snapshot_clone_no_wait", 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.398 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "True", 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.399 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix", 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "can_run": true, 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "error_string": "", 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "module_options": { 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "discovery_interval": { 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "discovery_interval", 2024-09-05T12:48:33.400 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "uint", 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "100", 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.401 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "identifier": { 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "identifier", 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.402 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.403 INFO:teuthology.orchestra.run.smithi071.stdout: "interval": { 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "interval", 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "secs", 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "60", 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.404 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "log_level": { 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_level", 2024-09-05T12:48:33.405 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.406 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster": { 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster", 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.408 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_cluster_level": { 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_cluster_level", 2024-09-05T12:48:33.409 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "info", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [ 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "critical", 2024-09-05T12:48:33.410 INFO:teuthology.orchestra.run.smithi071.stdout: "debug", 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "error", 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "info", 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "warning" 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "log_to_file": { 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "log_to_file", 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "bool", 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 1, 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "False", 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.412 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_host": { 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_host", 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.413 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "", 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.414 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_port": { 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_port", 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "int", 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "10051", 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.415 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "zabbix_sender": { 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "zabbix_sender", 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "str", 2024-09-05T12:48:33.416 INFO:teuthology.orchestra.run.smithi071.stdout: "level": "advanced", 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "flags": 0, 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "min": "", 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "max": "", 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "enum_allowed": [], 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "desc": "", 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "long_desc": "", 2024-09-05T12:48:33.417 INFO:teuthology.orchestra.run.smithi071.stdout: "tags": [], 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: "see_also": [] 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: "services": { 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: "dashboard": "https://172.21.15.71:8443/" 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: "always_on_modules": { 2024-09-05T12:48:33.418 INFO:teuthology.orchestra.run.smithi071.stdout: "octopus": [ 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-09-05T12:48:33.419 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "pacific": [ 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-09-05T12:48:33.420 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.421 INFO:teuthology.orchestra.run.smithi071.stdout: "quincy": [ 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-09-05T12:48:33.422 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "reef": [ 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-09-05T12:48:33.423 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: ], 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "squid": [ 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "balancer", 2024-09-05T12:48:33.424 INFO:teuthology.orchestra.run.smithi071.stdout: "crash", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "devicehealth", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "orchestrator", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "pg_autoscaler", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "progress", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "rbd_support", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "status", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "telemetry", 2024-09-05T12:48:33.425 INFO:teuthology.orchestra.run.smithi071.stdout: "volumes" 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: "last_failure_osd_epoch": 95, 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: "active_clients": [ 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "devicehealth", 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:33.426 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 1057326283 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "libcephsqlite", 2024-09-05T12:48:33.427 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 106173894 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.428 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "rbd_support", 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 3154422061 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.429 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2764563035 2024-09-05T12:48:33.430 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "snap_schedule", 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:33.431 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 2077907524 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: "name": "volumes", 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: "addrvec": [ 2024-09-05T12:48:33.432 INFO:teuthology.orchestra.run.smithi071.stdout: { 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: "type": "v2", 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: "addr": "172.21.15.71:0", 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: "nonce": 84693321 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-09-05T12:48:33.433 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:48:33.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:31 smithi123 ceph-mon[34321]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:48:33.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:31 smithi123 ceph-mon[34321]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:48:33.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:31 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2292621738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T12:48:33.435 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:31 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1440245163' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T12:48:33.436 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:32 smithi123 ceph-mon[34321]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 4.0 MiB/s wr, 207 op/s 2024-09-05T12:48:33.436 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:31 smithi174 ceph-mon[34313]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:48:33.436 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:31 smithi174 ceph-mon[34313]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:48:33.436 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:31 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2292621738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T12:48:33.436 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:31 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1440245163' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T12:48:33.436 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:32 smithi174 ceph-mon[34313]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 4.0 MiB/s wr, 207 op/s 2024-09-05T12:48:33.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f999c0a2d00 msgr2=0x7f999c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:33.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f999c0a2d00 0x7f999c0a51c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f99b4101de0 tx=0x7f99a4005f90 comp rx=0 tx=0).stop 2024-09-05T12:48:33.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 msgr2=0x7f99b419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:33.437 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 0x7f99b419a4b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f99a8006010 tx=0x7f99a8003c10 comp rx=0 tx=0).stop 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 shutdown_connections 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99b4109b70 0x7f99b419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f99b4104470 0x7f99b419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f999c0a2d00 0x7f999c0a51c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 --2- 172.21.15.71:0/1440245163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f99b4103ad0 0x7f99b419a4b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 >> 172.21.15.71:0/1440245163 conn(0x7f99b40ff7d0 msgr2=0x7f99b40fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 shutdown_connections 2024-09-05T12:48:33.438 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:31.284+0000 7f99bb200640 1 -- 172.21.15.71:0/1440245163 wait complete. 2024-09-05T12:48:33.446 DEBUG:tasks.check_counter:Getting stats from a 2024-09-05T12:48:33.446 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-05T12:48:33.446 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:31 smithi071 ceph-mon[29845]: Adjusting osd_memory_target on smithi071 to 176.4M 2024-09-05T12:48:33.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:31 smithi071 ceph-mon[29845]: Unable to set osd_memory_target on smithi071 to 185021849: error parsing value: Value '185021849' is below minimum 939524096 2024-09-05T12:48:33.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:31 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2292621738' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-05T12:48:33.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:31 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1440245163' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-05T12:48:33.447 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:32 smithi071 ceph-mon[29845]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 4.0 MiB/s wr, 207 op/s 2024-09-05T12:48:33.816 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:34.548 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:48:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1830, 2024-09-05T12:48:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1830, 2024-09-05T12:48:34.548 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 76076461, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 581117, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 9, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.393615306, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.106098003, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.194268473, 2024-09-05T12:48:34.549 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.068160169, 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1829, 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.159133260, 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000087005 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.550 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2831, 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2772, 2024-09-05T12:48:34.551 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 57044774, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 991067, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.426790441, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.097143239, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.259836273, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.071291306, 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:34.552 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2771, 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.152075658, 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000054881 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.553 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1999216, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 243392 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 390, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 680, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 58820666, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 791651, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 9, 2024-09-05T12:48:34.554 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.213888544, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.037591868, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.141222908, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.025907871, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 677, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.056192860, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000083002 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.555 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 668256 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-09-05T12:48:34.556 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7538, 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.212760497, 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000028225 2024-09-05T12:48:34.557 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.558 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.559 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 9175, 2024-09-05T12:48:34.560 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 263, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 8912, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 8799, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 23, 2024-09-05T12:48:34.561 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-09-05T12:48:34.562 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-09-05T12:48:34.563 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-09-05T12:48:34.564 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-09-05T12:48:34.565 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:34.566 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-09-05T12:48:34.567 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-09-05T12:48:34.568 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 48034, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-09-05T12:48:34.569 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 66, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 5, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-09-05T12:48:34.570 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 192858546, 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 192841016, 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 48034 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.571 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 1395, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 9192, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 7797, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 80, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 99, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 19, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 9175, 2024-09-05T12:48:34.572 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 9378, 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 203, 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 63004, 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 215328 2024-09-05T12:48:34.573 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.574 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:34.575 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.576 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.577 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-09-05T12:48:34.578 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.579 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.580 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-09-05T12:48:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.581 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-09-05T12:48:34.582 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.583 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.584 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.585 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:34.586 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.587 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.588 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.589 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.590 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.591 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.592 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 3, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 1, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 3539, 2024-09-05T12:48:34.593 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:34.594 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:34.595 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:34.596 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-09-05T12:48:34.597 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 299594, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 1685, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-09-05T12:48:34.598 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 187, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:34.599 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 10100890, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 72951, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.600 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 4565, 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 21, 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 3, 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 4565, 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-09-05T12:48:34.601 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4565, 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 14.720742590, 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003224697 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4565, 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4566 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.602 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 4565, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 4565, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 2734, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 1829, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-09-05T12:48:34.603 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:34.604 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 1, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-09-05T12:48:34.605 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-09-05T12:48:34.606 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-09-05T12:48:34.607 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 128, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 34, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 11, 2024-09-05T12:48:34.608 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 13, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 2, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-09-05T12:48:34.609 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-09-05T12:48:34.610 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 0, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.611 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 5047, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 191460309, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 5047, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:34.612 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5047, 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 191460309, 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.613 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:34.614 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 4565, 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 188666433, 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 4565, 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 188666433, 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.615 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:34.616 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 4565, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 4565, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 4565, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 4565, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.617 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:34.618 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.619 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x56176daacbe0": { 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:34.620 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:34.621 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:34.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.622 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:34.622 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:48:34.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:34 smithi071 ceph-mon[29845]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-05T12:48:34.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:34 smithi123 ceph-mon[34321]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-05T12:48:35.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:34 smithi174 ceph-mon[34313]: pgmap v660: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 1.4 MiB/s wr, 129 op/s 2024-09-05T12:48:35.397 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-05T12:48:35.397 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-05T12:48:35.398 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-09-05T12:48:35.398 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:35.398 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:35.398 DEBUG:tasks.check_counter:Getting stats from d 2024-09-05T12:48:35.398 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-05T12:48:35.770 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:36.428 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:48:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 403, 2024-09-05T12:48:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 390, 2024-09-05T12:48:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1634672, 2024-09-05T12:48:36.428 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 240560, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.081985700, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.037790968, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.055839625, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 390, 2024-09-05T12:48:36.429 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.036120594, 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000092616 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.430 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1634672, 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 240560 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 3, 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 298, 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 480, 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 288416, 2024-09-05T12:48:36.431 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.034524937, 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.025456173, 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.003513915, 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:36.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 295, 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.042042756, 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000142517 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:36.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 288416 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-09-05T12:48:36.434 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.000621248, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000474862, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:36.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:36.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-09-05T12:48:36.437 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:36.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:36.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:36.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:36.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148714, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 30, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:36.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-09-05T12:48:36.443 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:36.444 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 402, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1575336, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 402, 2024-09-05T12:48:36.445 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 402, 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1575336, 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:36.446 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:36.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:36.447 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:48:36.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:36 smithi071 ceph-mon[29845]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 88 op/s 2024-09-05T12:48:36.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:36 smithi123 ceph-mon[34321]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 88 op/s 2024-09-05T12:48:37.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:36 smithi174 ceph-mon[34313]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 458 KiB/s wr, 88 op/s 2024-09-05T12:48:37.260 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-05T12:48:37.260 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-05T12:48:37.260 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-05T12:48:37.260 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-05T12:48:37.261 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-05T12:48:37.261 DEBUG:tasks.check_counter:Getting stats from g 2024-09-05T12:48:37.261 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-05T12:48:37.630 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:38.256 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:48:38.256 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 36083, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 32579, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 14624101, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 74354138, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 11, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 4.242464798, 2024-09-05T12:48:38.257 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 1.742881409, 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 2.039761179, 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.650598571, 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 32579, 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5.839056180, 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000179227 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:38.258 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 37543, 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.007827622, 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000208 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2031424, 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 788896 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:38.259 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 77354, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 74107, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 24207941, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 102427610, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 15, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 8, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 7.845909001, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 3.327269102, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 4.174430375, 2024-09-05T12:48:38.260 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.930294792, 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 74104, 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 6.360370611, 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000085830 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 118127, 2024-09-05T12:48:38.261 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.024606436, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000208 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 136592 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 142930, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 209158, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 47887275, 2024-09-05T12:48:38.262 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 229333387, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 13, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 33.999447390, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 26.622803144, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 13.097306437, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.872403673, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 209156, 2024-09-05T12:48:38.263 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 40.230193837, 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000192345 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 171920, 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.082212829, 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000478 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T12:48:38.264 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-09-05T12:48:38.265 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 34122, 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 48.248127569, 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001413988 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 24783, 2024-09-05T12:48:38.266 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 3.603644521, 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000145407 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 81253, 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 77573, 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 77573, 2024-09-05T12:48:38.267 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 617.461156109, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007959743 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 3680, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 36, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 3, 2024-09-05T12:48:38.268 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 1, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 1384, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 263, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 1064, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 57, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 1008, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 293, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 142, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 142, 2024-09-05T12:48:38.269 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 27, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 135875, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 70518, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 3678, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 42, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 894, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 21469, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-09-05T12:48:38.270 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 3, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 228, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 7, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 426, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 703, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 214584438, 2024-09-05T12:48:38.271 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:38.272 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 10, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 6664, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 54647, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 10, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 27377, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 8204, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 1512, 2024-09-05T12:48:38.273 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 61047 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 57, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 49, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 37, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 64, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 114, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-09-05T12:48:38.274 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 13152, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 13152, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:38.275 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 16, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 1327, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 4, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 4, 2024-09-05T12:48:38.276 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 64813, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 64813, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-09-05T12:48:38.277 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 89, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 89, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 7, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-09-05T12:48:38.278 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 231652555, 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 18554, 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 145.329819758, 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007832802 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-09-05T12:48:38.279 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 1046, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 14390, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 13344, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 140, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 190, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 50, 2024-09-05T12:48:38.280 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 1384, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 24094, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 22710, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 142, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 14643, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 14501, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 71112, 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 223520 2024-09-05T12:48:38.281 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 121300, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 59, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 81253, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 1161, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 10976, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 13121, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 252.603302162, 2024-09-05T12:48:38.282 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.019251833 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2221, 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 31.275849662, 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.014081877 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.283 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-09-05T12:48:38.284 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 18619, 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 33.509603673, 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001799753 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.285 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.286 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.287 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.288 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.288 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.288 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-09-05T12:48:38.288 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.288 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.288 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-09-05T12:48:38.289 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.290 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001173305, 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000586652 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 27958, 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 103.525790859, 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003702904 2024-09-05T12:48:38.291 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2194, 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 7.747570152, 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003531253 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.292 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.293 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.294 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 46, 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 32.594657868, 2024-09-05T12:48:38.295 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.708579518 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 260, 2024-09-05T12:48:38.296 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.295885928, 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001138022 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-09-05T12:48:38.297 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.298 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 13152, 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 155.907322500, 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.011854267 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:38.299 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 1, 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 2, 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 1, 2024-09-05T12:48:38.300 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 1, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 10851, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 10851, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 1252, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:38.301 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:38.302 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:38.303 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:38.304 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:38.305 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 314393, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 502, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:38.306 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 187, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 3877807, 2024-09-05T12:48:38.307 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 30367, 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-09-05T12:48:38.308 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 47356, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 230493467, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 30, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 47356, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 47356, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 387.327230215, 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.008179052 2024-09-05T12:48:38.309 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 47356, 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 48294 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 47356, 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 692, 2024-09-05T12:48:38.310 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 46664, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 10, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 692, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 19246, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 417, 2024-09-05T12:48:38.311 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 3, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 26378, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-09-05T12:48:38.312 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 20, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:38.313 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-09-05T12:48:38.314 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-09-05T12:48:38.315 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 128, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 34, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-09-05T12:48:38.316 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 1101, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 427 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-09-05T12:48:38.317 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 1315 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T12:48:38.318 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 5, 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 26304, 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 13152, 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:38.319 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 256363, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 65475526, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 256363, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:38.320 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 256363, 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 65475526, 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.321 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:38.322 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 47356, 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 231929227, 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 47356, 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 231929227, 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.323 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:38.324 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 47356, 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 47356, 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 47356, 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 47356, 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.325 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 13152, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1435760, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:38.326 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 13152, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 692, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1435760, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.327 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55d685948960": { 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 1148, 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 64813, 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 227458251, 2024-09-05T12:48:38.328 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 64813, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 18554, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 227457103, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:38.329 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:38.330 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:38.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.330 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:38.330 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:48:38.837 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:38 smithi071 ceph-mon[29845]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 103 op/s 2024-09-05T12:48:38.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:38 smithi123 ceph-mon[34321]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 103 op/s 2024-09-05T12:48:39.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:38 smithi174 ceph-mon[34313]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 586 KiB/s wr, 103 op/s 2024-09-05T12:48:39.466 INFO:tasks.check_counter:Daemon mds.g mds.exported=3 2024-09-05T12:48:39.466 INFO:tasks.check_counter:Daemon mds.g mds.imported=7 2024-09-05T12:48:39.466 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=6 2024-09-05T12:48:39.467 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:39.467 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:39.467 DEBUG:tasks.check_counter:Getting stats from j 2024-09-05T12:48:39.467 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-05T12:48:39.835 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:40.440 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-09-05T12:48:40.440 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:40.440 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 4, 2024-09-05T12:48:40.440 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 4, 2024-09-05T12:48:40.440 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2992, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 448, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.006125015, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000072008, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.003650476, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T12:48:40.441 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.006522180, 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002174060 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:40.442 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 3, 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 299, 2024-09-05T12:48:40.443 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 480, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 288592, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.037156263, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.026219636, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.006061635, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:40.444 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 296, 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.032519878, 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000109864 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:40.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 288592 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 401, 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 390, 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1632224, 2024-09-05T12:48:40.446 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 241392, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.079278487, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.038928664, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.054408382, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:40.447 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 389, 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.036154907, 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000092943 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:40.448 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1632224, 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 241392 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-09-05T12:48:40.449 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:40.450 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:40.451 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:40.452 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:40.453 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148714, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 30, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:40.454 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-09-05T12:48:40.455 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:40.456 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 404, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1575596, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:40.457 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 404, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 404, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1575596, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-09-05T12:48:40.458 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-09-05T12:48:40.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:40.459 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-09-05T12:48:40.459 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-09-05T12:48:40.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:40 smithi071 ceph-mon[29845]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 224 KiB/s wr, 45 op/s 2024-09-05T12:48:40.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:40 smithi123 ceph-mon[34321]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 224 KiB/s wr, 45 op/s 2024-09-05T12:48:41.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:40 smithi174 ceph-mon[34313]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 224 KiB/s wr, 45 op/s 2024-09-05T12:48:41.257 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-05T12:48:41.257 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-05T12:48:41.257 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-05T12:48:41.257 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-05T12:48:41.257 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-05T12:48:41.258 DEBUG:tasks.check_counter:Getting stats from b 2024-09-05T12:48:41.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-05T12:48:41.576 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 95043, 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 108014, 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 26392761, 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 167916213, 2024-09-05T12:48:42.183 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 249, 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 6.227979490, 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 2.362390140, 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 3.117770751, 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.996084185, 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:42.184 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 108011, 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.121928795, 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000093712 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 143688, 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.023025392, 2024-09-05T12:48:42.185 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000160 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 764352, 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 252112 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 251570, 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 335998, 2024-09-05T12:48:42.186 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 79379776, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 368751494, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 34, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 14.430925012, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 6.757624263, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 8.311126091, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 1.076222107, 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:42.187 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 335993, 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 23.399691135, 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000069643 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 301433, 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.070905855, 2024-09-05T12:48:42.188 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000235 2024-09-05T12:48:42.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.189 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-05T12:48:42.189 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 150304 2024-09-05T12:48:42.189 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.189 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:42.189 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 137334, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 156026, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 36751517, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 132876524, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 7.828739457, 2024-09-05T12:48:42.190 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 3.090962086, 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 4.794200539, 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.756125793, 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 156029, 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 9.806182015, 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000062848 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.191 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 247655, 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.036961356, 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000149 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2048048, 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 804448 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.192 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 73705, 2024-09-05T12:48:42.193 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 60.647621247, 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000822842 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:42.194 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 26481, 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3.736254894, 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000141091 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 108421, 2024-09-05T12:48:42.195 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 95931, 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 95931, 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 764.068201973, 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.007964768 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-05T12:48:42.196 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 12490, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 20, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 2340, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 8, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 6, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 843, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 474, 2024-09-05T12:48:42.197 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 357, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 12, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 138, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 2678, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 127, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 128, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 20, 2024-09-05T12:48:42.198 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 150968, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 103522, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 12462, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 76, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 2, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 433, 2024-09-05T12:48:42.199 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 25226, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 26, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 1830, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 13, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 565, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:42.200 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 703, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 218185743, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 6, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:42.201 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 21, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 25150, 2024-09-05T12:48:42.202 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 62096, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 52783, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 11607, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 3198, 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 51733 2024-09-05T12:48:42.203 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 21, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 37, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 24, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 92, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 223, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-05T12:48:42.204 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 13444, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 13444, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:42.205 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 16, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 1266, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 26, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-05T12:48:42.206 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 14, 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 83926, 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 83898, 2024-09-05T12:48:42.207 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 83898, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 28, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 551, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 198, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 197, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 197, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 1, 2024-09-05T12:48:42.208 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 1, 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 1, 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 354717870, 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 355178323, 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-05T12:48:42.209 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 38343, 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 225.689649720, 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005886071 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-05T12:48:42.210 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 712, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 15848, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 15136, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 140, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 340, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 200, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 843, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 32651, 2024-09-05T12:48:42.211 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 31808, 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 128, 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 15172, 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 15044, 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 86872, 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 231712 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-05T12:48:42.212 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 146671, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 28027, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 108421, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 1844, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 55681, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 14222, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 179.337111666, 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.012609837 2024-09-05T12:48:42.213 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 18001, 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 115.397905167, 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006410638 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.214 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.215 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 18095, 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 104.186471286, 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005757749 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.217 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-05T12:48:42.217 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-09-05T12:48:42.217 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000275800, 2024-09-05T12:48:42.217 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000137900 2024-09-05T12:48:42.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.217 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 21, 2024-09-05T12:48:42.219 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.005547963, 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000264188 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 119, 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.598337217, 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005028043 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.220 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 21, 2024-09-05T12:48:42.221 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 38.621546478, 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 1.839121260 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10, 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.023199780, 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002319978 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-05T12:48:42.222 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 28803, 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 158.748103740, 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005511512 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2694, 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 45.289032496, 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.016811073 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.223 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.224 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1, 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001430209, 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001430209 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 15, 2024-09-05T12:48:42.225 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 8.382468515, 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.558831234 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 39, 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 32.570523105, 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.835141618 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.227 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 436, 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.736491563, 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001689200 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-05T12:48:42.228 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.229 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.229 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-05T12:48:42.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8, 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.019671823, 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002458977 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 13444, 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 80.150085165, 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005961773 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:42.231 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-09-05T12:48:42.232 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 5, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 3, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 9185, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 4592, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1250, 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:42.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 480, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 480, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:42.234 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:42.235 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:42.236 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:42.237 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 317519, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 469, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 88, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 1, 2024-09-05T12:48:42.238 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 187, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:42.239 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 2752733, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 23909, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-05T12:48:42.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 85542, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 372396850, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 36, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 85542, 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-05T12:48:42.241 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 85542, 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2562.412656118, 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.029955023 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 85542, 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 113708 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.242 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 85542, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 565, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 84977, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2139, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 14701, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 498, 2024-09-05T12:48:42.243 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 38841, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 680, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 8, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 27614, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:42.244 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 6, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 22259, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-05T12:48:42.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-05T12:48:42.246 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-05T12:48:42.247 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 128, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 43, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-09-05T12:48:42.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 5702, 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 87, 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 1173 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-09-05T12:48:42.249 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 1298 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:42.250 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 5, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 26888, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 13444, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:42.251 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 483943, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 101546457, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 483943, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 483943, 2024-09-05T12:48:42.252 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 101546457, 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:42.253 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 85542, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 373862633, 2024-09-05T12:48:42.254 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 85542, 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 373862633, 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:42.255 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 85542, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 85542, 2024-09-05T12:48:42.256 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 85542, 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 85542, 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.257 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 13444, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1467468, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 13444, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:42.258 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 498, 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1467468, 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x556f16dca960": { 2024-09-05T12:48:42.259 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 83926, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 350984019, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 83926, 2024-09-05T12:48:42.260 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 38343, 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 350984019, 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:42.261 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.262 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:42.262 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-05T12:48:42.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:42 smithi071 ceph-mon[29845]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 237 KiB/s wr, 42 op/s 2024-09-05T12:48:42.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:42 smithi123 ceph-mon[34321]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 237 KiB/s wr, 42 op/s 2024-09-05T12:48:43.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:42 smithi174 ceph-mon[34313]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 237 KiB/s wr, 42 op/s 2024-09-05T12:48:43.085 INFO:tasks.check_counter:Daemon mds.b mds.exported=26 2024-09-05T12:48:43.085 INFO:tasks.check_counter:Daemon mds.b mds.imported=13 2024-09-05T12:48:43.086 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-05T12:48:43.086 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=21 2024-09-05T12:48:43.086 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=15 2024-09-05T12:48:43.086 DEBUG:tasks.check_counter:Getting stats from e 2024-09-05T12:48:43.086 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-05T12:48:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:48:43.904 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-05T12:48:43.904 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 3240, 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 3242, 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 81628813, 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1030327, 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-09-05T12:48:43.905 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.302107198, 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.083030073, 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.117005179, 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.080427949, 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3240, 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.109208343, 2024-09-05T12:48:43.906 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000033706 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T12:48:43.907 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T12:48:43.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.908 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:43.908 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1295, 2024-09-05T12:48:43.908 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1234, 2024-09-05T12:48:43.908 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 73515007, 2024-09-05T12:48:43.908 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 503750, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.167606433, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.043865946, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.115926744, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.020302387, 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:43.909 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1234, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.050188004, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000040670 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2004736, 2024-09-05T12:48:43.910 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 248016 2024-09-05T12:48:43.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.911 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:43.911 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 702, 2024-09-05T12:48:43.911 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 998, 2024-09-05T12:48:43.911 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 75692722, 2024-09-05T12:48:43.911 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 896760, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.110860218, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.025293687, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.060446583, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.017920587, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 995, 2024-09-05T12:48:43.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.041971693, 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000042182 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.913 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 674160 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:43.914 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7948, 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.169320453, 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000021303 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:43.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-09-05T12:48:43.916 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:43.917 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 14313, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 14, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 14299, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-05T12:48:43.918 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 14293, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 6, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-09-05T12:48:43.919 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-05T12:48:43.920 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-05T12:48:43.921 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-05T12:48:43.922 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-09-05T12:48:43.923 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:43.924 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:43.925 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-05T12:48:43.926 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-05T12:48:43.927 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 64813, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 89, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 7, 2024-09-05T12:48:43.928 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 231652555, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 231651981, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 64813 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 1013, 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 14324, 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 13311, 2024-09-05T12:48:43.930 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 61, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 76, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 15, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 14313, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 14550, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 237, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-05T12:48:43.931 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 67404, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215328 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-05T12:48:43.932 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-05T12:48:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-05T12:48:43.933 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.934 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-05T12:48:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.935 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.936 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.937 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:43.939 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-05T12:48:43.940 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.941 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.942 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-05T12:48:43.943 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.944 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:43.945 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.946 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.947 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.948 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:43.949 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.950 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.951 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:43.952 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.953 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-09-05T12:48:43.954 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 3999, 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:43.955 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:43.956 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:43.957 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:43.958 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 285686, 2024-09-05T12:48:43.959 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 1252, 2024-09-05T12:48:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-05T12:48:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-05T12:48:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:43.960 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 187, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-05T12:48:43.961 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 12783145, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 84417, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-05T12:48:43.962 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 4744, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 4744, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4744, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 7.232821393, 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001524625 2024-09-05T12:48:43.963 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4744, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4745 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 4744, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 4744, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-09-05T12:48:43.964 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2761, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1981, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-09-05T12:48:43.965 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-09-05T12:48:43.966 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-05T12:48:43.967 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-05T12:48:43.968 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-05T12:48:43.969 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 128, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 11, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 1, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-09-05T12:48:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-09-05T12:48:43.971 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-05T12:48:43.972 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 5233, 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 230337859, 2024-09-05T12:48:43.973 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 5233, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 5233, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 230337859, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.974 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:43.975 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 4744, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 227511910, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 4744, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 227511910, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.976 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:43.977 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 4744, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 4744, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 4744, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 4744, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.978 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:43.979 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.980 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x564fa1b0cb40": { 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:43.981 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:43.982 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.983 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:43.983 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-05T12:48:44.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:44 smithi123 ceph-mon[34321]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 178 KiB/s wr, 31 op/s 2024-09-05T12:48:44.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:44 smithi071 ceph-mon[29845]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 178 KiB/s wr, 31 op/s 2024-09-05T12:48:44.926 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-05T12:48:44.927 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-05T12:48:44.927 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-09-05T12:48:44.927 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:44.927 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:44.927 DEBUG:tasks.check_counter:Getting stats from h 2024-09-05T12:48:44.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-05T12:48:45.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:44 smithi174 ceph-mon[34313]: pgmap v665: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 178 KiB/s wr, 31 op/s 2024-09-05T12:48:45.176 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:48:45.829 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 120168, 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 156928, 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 38824205, 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 185491068, 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 20, 2024-09-05T12:48:45.830 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 9, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 7.264835967, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 3.209422399, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 4.289632384, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.754009464, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 156923, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 9.555542428, 2024-09-05T12:48:45.831 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000060893 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 142110, 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.042132754, 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000296 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2656, 2024-09-05T12:48:45.832 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 4000 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 78405, 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 68522, 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 25114394, 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 65506639, 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2024-09-05T12:48:45.833 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 5.207682142, 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 2.315651659, 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 3.000547309, 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.470892257, 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 68523, 2024-09-05T12:48:45.834 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2.845684445, 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000041528 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 126884, 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.020989063, 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000165 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.835 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 676048 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 25922, 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 26927, 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 10290830, 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 89383176, 2024-09-05T12:48:45.836 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 7, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 2.100539212, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.728632355, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.838198202, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.574092010, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 26926, 2024-09-05T12:48:45.837 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.631825631, 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000060604 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 26647, 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.005298477, 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000198 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2016304, 2024-09-05T12:48:45.838 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 264704 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:45.839 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25559, 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 26.817594093, 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001049242 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:45.840 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 15383, 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.514552912, 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000098456 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 57906, 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 57163, 2024-09-05T12:48:45.841 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 57163, 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 345.100313390, 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006037127 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 743, 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 2, 2024-09-05T12:48:45.842 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 36, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 3, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 1, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 1640, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 404, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 1224, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 12, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 1045, 2024-09-05T12:48:45.843 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 318, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 152, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 152, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 31, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 88225, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 55257, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 733, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 99, 2024-09-05T12:48:45.844 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 1, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 738, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 23601, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 4, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 195, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 9, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 762, 2024-09-05T12:48:45.845 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 703, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 216374592, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 6, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:45.846 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 6, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 5441, 2024-09-05T12:48:45.847 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 34184, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 1, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 27087, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 7305, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 1306, 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 43031 2024-09-05T12:48:45.848 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 14, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 77, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 49, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 169, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 56, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-05T12:48:45.849 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 7579, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 7579, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:45.850 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 16, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 1284, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 4, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 4, 2024-09-05T12:48:45.851 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 48061, 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-05T12:48:45.852 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 48061, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 68, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 68, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 5, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-05T12:48:45.853 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 192953944, 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 14186, 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 141.514757787, 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.009975663 2024-09-05T12:48:45.854 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 1436, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 9256, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 7820, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 162, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 197, 2024-09-05T12:48:45.855 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 35, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 1640, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 17023, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 15383, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 152, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 9249, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 9097, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 73468, 2024-09-05T12:48:45.856 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223520 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 82811, 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 1825, 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 57906, 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 954, 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 6025, 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-05T12:48:45.857 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7577, 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 84.564173725, 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.011160640 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2931, 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 38.589979559, 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.013166147 2024-09-05T12:48:45.858 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.859 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 12425, 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 19.659407151, 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001582246 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:45.860 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.861 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.862 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.863 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.864 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.865 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.866 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 24971, 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 70.113301160, 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002807789 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1425, 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 13.366746524, 2024-09-05T12:48:45.867 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.009380172 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:45.868 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.869 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.870 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 31, 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 32.148151092, 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 1.037037132 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.871 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 224, 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.172807580, 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000771462 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.872 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.873 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7579, 2024-09-05T12:48:45.874 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 86.485746599, 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.011411234 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.875 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 2, 2024-09-05T12:48:45.876 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 11638, 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 5819, 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1243, 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:45.877 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:45.878 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:45.879 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:45.880 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:45.881 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 307577, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 819, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 88, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 1, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-05T12:48:45.882 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 187, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:45.883 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 5049722, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 39135, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.884 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 30901, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 190672332, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 31, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 30901, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 30901, 2024-09-05T12:48:45.885 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 341.227587405, 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.011042606 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 30901, 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 31552 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 30901, 2024-09-05T12:48:45.886 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 387, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 30514, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 382, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 14568, 2024-09-05T12:48:45.887 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 262, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 3, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 15217, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:45.888 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-05T12:48:45.889 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-05T12:48:45.890 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-05T12:48:45.891 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 128, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-09-05T12:48:45.892 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 817, 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 7, 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 265 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-09-05T12:48:45.893 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 1186 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-05T12:48:45.894 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 5, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 15158, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 7579, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:45.895 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 224491, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 55418182, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 224491, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:45.896 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 224491, 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 55418182, 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.897 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:45.898 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 30901, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 191500124, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 30901, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 191500124, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.899 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:45.900 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 30901, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 30901, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 30901, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 30901, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.901 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 7579, 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 827911, 2024-09-05T12:48:45.902 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 7579, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 382, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 827911, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.903 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55a5e37f8960": { 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 574, 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 48061, 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 188759640, 2024-09-05T12:48:45.904 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 48061, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 14186, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 188759066, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:45.905 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.906 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:45.906 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-05T12:48:46.680 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:46 smithi123 ceph-mon[34321]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 24 op/s 2024-09-05T12:48:46.791 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-09-05T12:48:46.791 INFO:tasks.check_counter:Daemon mds.h mds.imported=9 2024-09-05T12:48:46.792 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-05T12:48:46.792 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:46.792 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:46.792 DEBUG:tasks.check_counter:Getting stats from k 2024-09-05T12:48:46.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-05T12:48:46.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:46 smithi071 ceph-mon[29845]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 24 op/s 2024-09-05T12:48:47.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:46 smithi174 ceph-mon[34313]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 154 KiB/s wr, 24 op/s 2024-09-05T12:48:47.044 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.b/config 2024-09-05T12:48:47.550 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-05T12:48:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1097, 2024-09-05T12:48:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1038, 2024-09-05T12:48:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 81885640, 2024-09-05T12:48:47.550 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 442842, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.158496646, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.038955143, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.114701115, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.015908981, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1037, 2024-09-05T12:48:47.551 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.054000540, 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000052073 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.552 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2002832, 2024-09-05T12:48:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 250128 2024-09-05T12:48:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1547, 2024-09-05T12:48:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1846, 2024-09-05T12:48:47.553 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 101074371, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1167619, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.197167789, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.044084265, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.096026060, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.042117652, 2024-09-05T12:48:47.554 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1843, 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.082435461, 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000044728 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.555 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 676304 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2713, 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2713, 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 70441174, 2024-09-05T12:48:47.556 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 861968, 2024-09-05T12:48:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:47.557 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.277611388, 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.076097774, 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.103507145, 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.077105096, 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2712, 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.104023595, 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000038356 2024-09-05T12:48:47.558 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T12:48:47.559 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8140, 2024-09-05T12:48:47.560 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.194096345, 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000023844 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.561 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.562 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-09-05T12:48:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-09-05T12:48:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:47.563 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-09-05T12:48:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-09-05T12:48:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-09-05T12:48:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 13297, 2024-09-05T12:48:47.564 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 34, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 13263, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 13249, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 4, 2024-09-05T12:48:47.565 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-09-05T12:48:47.566 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:47.567 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-09-05T12:48:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-09-05T12:48:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-09-05T12:48:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:47.568 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-09-05T12:48:47.569 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-09-05T12:48:47.570 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-09-05T12:48:47.571 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-09-05T12:48:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-09-05T12:48:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-09-05T12:48:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:47.572 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-09-05T12:48:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-09-05T12:48:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-09-05T12:48:47.573 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-09-05T12:48:47.574 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 67442, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-09-05T12:48:47.575 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 95, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 7, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-09-05T12:48:47.576 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 254177003, 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 254170511, 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.577 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 67442 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 917, 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 13403, 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 12486, 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 55, 2024-09-05T12:48:47.578 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 84, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 29, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 13297, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 13684, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 387, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-09-05T12:48:47.579 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 66632, 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215328 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-09-05T12:48:47.580 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.581 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.582 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-09-05T12:48:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-09-05T12:48:47.583 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.584 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-09-05T12:48:47.585 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.586 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.587 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-09-05T12:48:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.588 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.589 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.590 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-09-05T12:48:47.591 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.592 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.593 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.594 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.595 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.596 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-09-05T12:48:47.597 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.598 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.599 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 2, 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 3, 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 3, 2024-09-05T12:48:47.600 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 3318, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:47.601 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:47.602 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:47.603 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:47.604 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:47.605 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 273431, 2024-09-05T12:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 1155, 2024-09-05T12:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-05T12:48:47.606 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-05T12:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:47.607 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 187, 2024-09-05T12:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-05T12:48:47.608 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 11848222, 2024-09-05T12:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 78050, 2024-09-05T12:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-05T12:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:47.609 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-05T12:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-09-05T12:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-09-05T12:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-09-05T12:48:47.610 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 4860, 2024-09-05T12:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-09-05T12:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-09-05T12:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 4860, 2024-09-05T12:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-09-05T12:48:47.611 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4860, 2024-09-05T12:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 10.547739246, 2024-09-05T12:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002170316 2024-09-05T12:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-09-05T12:48:47.612 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-09-05T12:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4860, 2024-09-05T12:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4861 2024-09-05T12:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 4860, 2024-09-05T12:48:47.613 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 4860, 2024-09-05T12:48:47.614 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-09-05T12:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-09-05T12:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-09-05T12:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 2825, 2024-09-05T12:48:47.616 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-09-05T12:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 2033, 2024-09-05T12:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-09-05T12:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-09-05T12:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-09-05T12:48:47.617 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-09-05T12:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-09-05T12:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-09-05T12:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-09-05T12:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-09-05T12:48:47.618 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-09-05T12:48:47.619 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-09-05T12:48:47.620 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-09-05T12:48:47.621 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 128, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 34, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 10, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 11, 2024-09-05T12:48:47.622 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 1, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:47.623 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:47.624 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:47.625 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 5353, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 252891494, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 5353, 2024-09-05T12:48:47.626 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 5353, 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 252891494, 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.627 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:47.628 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 4860, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 250045537, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 4860, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 250045537, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.629 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:47.630 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 4860, 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 4860, 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 4860, 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 4860, 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.631 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:47.632 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-05T12:48:47.633 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55ae6bc06b40": { 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-09-05T12:48:47.634 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-05T12:48:47.635 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-05T12:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-05T12:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-05T12:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-05T12:48:47.636 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-05T12:48:48.416 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-05T12:48:48.416 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-05T12:48:48.416 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-05T12:48:48.416 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:48.417 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:48.417 DEBUG:tasks.check_counter:Getting stats from c 2024-09-05T12:48:48.417 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-05T12:48:48.679 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:48 smithi123 ceph-mon[34321]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 29 op/s 2024-09-05T12:48:48.800 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:48:48.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:48 smithi071 ceph-mon[29845]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 29 op/s 2024-09-05T12:48:48.921 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:48 smithi174 ceph-mon[34313]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 163 KiB/s wr, 29 op/s 2024-09-05T12:48:49.599 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-05T12:48:49.599 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 115130, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 98413, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 39118489, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 82644780, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 6.014254079, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 2.367321905, 2024-09-05T12:48:49.600 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 4.174830151, 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.307793022, 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 98412, 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.248891890, 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000043174 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:49.601 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 189845, 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.029699369, 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000156 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 134909, 2024-09-05T12:48:49.602 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 185201, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 45688557, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 210719102, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 16, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 7.907153754, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 3.730495051, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 4.726345536, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.423180668, 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:49.603 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 185200, 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 9.784795804, 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000052833 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 154778, 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.041826371, 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000270 2024-09-05T12:48:49.604 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2023264, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 257216 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 15236, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 15570, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 3861928, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 69052613, 2024-09-05T12:48:49.605 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 16, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 1.327653573, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.518725289, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.275558737, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.453341415, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 15567, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.936386137, 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000060151 2024-09-05T12:48:49.606 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 77, 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000014128, 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000183 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3936, 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 689424 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.607 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 31019, 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 32.804608314, 2024-09-05T12:48:49.608 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001057564 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 20562, 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.811750247, 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000088111 2024-09-05T12:48:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 69794, 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 66539, 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 66539, 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 510.936344449, 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007678749 2024-09-05T12:48:49.610 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 3255, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 1, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 35, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 3, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 1447, 2024-09-05T12:48:49.611 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 230, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 1079, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 138, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 1104, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 309, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 157, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 157, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 22, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 122655, 2024-09-05T12:48:49.612 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 70263, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 3252, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 89, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 1694, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 4192, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 6, 2024-09-05T12:48:49.613 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 143, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 8, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 541, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 703, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 216374592, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-05T12:48:49.614 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 36, 2024-09-05T12:48:49.615 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 6869, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 45782, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 29892, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 8297, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 2221, 2024-09-05T12:48:49.616 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 52024 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 12, 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 98, 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 65, 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 153, 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 146, 2024-09-05T12:48:49.617 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 10101, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 10101, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:49.618 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 16, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 1298, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 9, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 3, 2024-09-05T12:48:49.619 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 59739, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 59739, 2024-09-05T12:48:49.620 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 81, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 81, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 6, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-05T12:48:49.621 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 207709657, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 16597, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 158.352462498, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.009541029 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.622 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 1251, 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 11563, 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 10312, 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 147, 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 221, 2024-09-05T12:48:49.623 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 74, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 1455, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 20145, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 18690, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 127, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 11608, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 11481, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 73960, 2024-09-05T12:48:49.624 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 107169, 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 1763, 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 69794, 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 1278, 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 10510, 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-05T12:48:49.625 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10118, 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 230.124650931, 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.022744084 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2684, 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 35.793292843, 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.013335802 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.626 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.627 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 16818, 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 124.106340866, 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007379375 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.628 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.629 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.630 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-05T12:48:49.632 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.633 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5, 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002449757, 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000489951 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 23497, 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 74.699774955, 2024-09-05T12:48:49.634 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003179119 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2837, 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 9.288179931, 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003273944 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.635 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.636 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.637 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-05T12:48:49.638 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 255, 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.165607263, 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004571008 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.639 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 224, 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.220742369, 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000985457 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.640 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.641 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10101, 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 35.535305534, 2024-09-05T12:48:49.642 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003517998 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.643 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 5, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 3, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 10903, 2024-09-05T12:48:49.644 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 5451, 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1212, 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 120, 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 120, 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:49.645 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:49.646 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:49.647 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:49.648 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:49.649 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 345524, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 685, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:49.650 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 187, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 4449457, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 34776, 2024-09-05T12:48:49.651 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-05T12:48:49.652 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 39107, 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 205928746, 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 39107, 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 39107, 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 415.838709555, 2024-09-05T12:48:49.653 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.010633357 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 39107, 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 40035 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 39107, 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 607, 2024-09-05T12:48:49.654 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 38500, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 604, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 17201, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 387, 2024-09-05T12:48:49.655 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 3, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 20262, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:49.656 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-05T12:48:49.657 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-05T12:48:49.658 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-05T12:48:49.659 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 128, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 34, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-05T12:48:49.660 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 1147, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 4, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 397 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:49.661 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 1187 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:49.662 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 5, 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 20202, 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 10101, 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:49.663 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 265271, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 66456056, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 265271, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:49.664 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 265271, 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 66456056, 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.665 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:49.666 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 39107, 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 207031970, 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 39107, 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 207031970, 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.667 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 39107, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 39107, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 39107, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 39107, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.669 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 10101, 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1103297, 2024-09-05T12:48:49.670 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 10101, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 604, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1103297, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.671 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x556f8f9dc960": { 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 33196, 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 59739, 2024-09-05T12:48:49.672 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 203515353, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 59739, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 16597, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 203482157, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:49.673 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:49.674 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:49.674 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.674 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:49.674 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-05T12:48:50.597 INFO:tasks.check_counter:Daemon mds.c mds.exported=6 2024-09-05T12:48:50.597 INFO:tasks.check_counter:Daemon mds.c mds.imported=8 2024-09-05T12:48:50.597 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-09-05T12:48:50.597 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:50.597 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:50.597 DEBUG:tasks.check_counter:Getting stats from f 2024-09-05T12:48:50.598 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-05T12:48:50.859 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:48:50.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:50 smithi071 ceph-mon[29845]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-05T12:48:50.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:50 smithi123 ceph-mon[34321]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-05T12:48:50.984 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:50 smithi174 ceph-mon[34313]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-09-05T12:48:51.470 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-05T12:48:51.470 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 523, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 525, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 66058210, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 166320, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.086447385, 2024-09-05T12:48:51.471 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.011316218, 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.056087398, 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.014498030, 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 523, 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018764113, 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000035877 2024-09-05T12:48:51.472 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T12:48:51.473 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1689, 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1655, 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 80245306, 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 577487, 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-09-05T12:48:51.474 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.191357499, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.050476739, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.092689459, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.041272736, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1654, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.064459385, 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000038971 2024-09-05T12:48:51.475 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 454624, 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 119824 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.476 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 730, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 869, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 84076339, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 568384, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.114151874, 2024-09-05T12:48:51.477 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.023715281, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.066351846, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.017821551, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 868, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.038896894, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000044812 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.478 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 337824 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-05T12:48:51.479 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4455, 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.219329188, 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000049232 2024-09-05T12:48:51.480 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.481 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.482 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-05T12:48:51.483 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 10764, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 7394, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 3370, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 47, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-05T12:48:51.484 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 20, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:51.485 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:51.486 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:51.487 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:51.487 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 703, 2024-09-05T12:48:51.487 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 221376762, 2024-09-05T12:48:51.487 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-05T12:48:51.487 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:51.487 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-05T12:48:51.488 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-05T12:48:51.489 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-05T12:48:51.490 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:51.491 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:51.492 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-05T12:48:51.493 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.493 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-05T12:48:51.493 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-05T12:48:51.493 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-05T12:48:51.493 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-05T12:48:51.493 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 55757, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 51, 2024-09-05T12:48:51.494 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 354717870, 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 355327356, 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 355291874, 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.495 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 55757 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 687, 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 11832, 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 11145, 2024-09-05T12:48:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 63, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 178, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 115, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 10764, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 12462, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 1698, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-05T12:48:51.497 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 63480, 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-05T12:48:51.498 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.499 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.500 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.501 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.501 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.501 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-05T12:48:51.501 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.501 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.501 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:51.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.503 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.503 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.503 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.503 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-05T12:48:51.503 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.503 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.504 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.504 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.504 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-05T12:48:51.504 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.505 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.505 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.505 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.505 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:51.506 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.506 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.506 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.506 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.507 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:51.507 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.507 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.507 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.508 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.508 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-05T12:48:51.508 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.508 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.508 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.508 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-05T12:48:51.510 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.511 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:51.512 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.513 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.513 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.513 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.513 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:51.514 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.514 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.514 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.514 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.515 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-05T12:48:51.515 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.515 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.515 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.515 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:51.516 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.517 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.518 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.519 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.520 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.521 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-05T12:48:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 0, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-05T12:48:51.523 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2643, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:51.525 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:51.526 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:51.526 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:51.526 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:51.526 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:51.526 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:51.526 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:51.527 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:51.528 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 263916, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 947, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:51.529 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 187, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-05T12:48:51.530 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 9476870, 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 63366, 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-05T12:48:51.531 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 2687, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 26, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 4, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 2687, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2687, 2024-09-05T12:48:51.532 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.468943124, 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001663171 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2687, 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2688 2024-09-05T12:48:51.533 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 2687, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 2687, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 1462, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1222, 2024-09-05T12:48:51.534 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:51.535 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-05T12:48:51.536 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-05T12:48:51.537 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-05T12:48:51.538 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-05T12:48:51.538 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-05T12:48:51.538 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-05T12:48:51.538 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-05T12:48:51.538 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 128, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 24, 2024-09-05T12:48:51.539 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 10, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 10, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-05T12:48:51.540 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.541 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.542 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 2938, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 230099798, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 2938, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:51.543 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2938, 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 230099798, 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.544 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2687, 2024-09-05T12:48:51.545 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 229177327, 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2687, 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 229177327, 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.546 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:51.547 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 2687, 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2687, 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2687, 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2687, 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.548 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:51.549 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.550 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x563eba8e46e0": { 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:51.551 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:51.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:51.553 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.553 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:51.553 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-05T12:48:52.648 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-05T12:48:52.648 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-05T12:48:52.649 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-09-05T12:48:52.649 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:52.649 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:52.649 DEBUG:tasks.check_counter:Getting stats from i 2024-09-05T12:48:52.649 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-05T12:48:52.844 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:52 smithi174 ceph-mon[34313]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-09-05T12:48:52.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:52 smithi071 ceph-mon[29845]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-09-05T12:48:52.913 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:48:52.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:52 smithi123 ceph-mon[34321]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 17 op/s 2024-09-05T12:48:53.459 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-05T12:48:53.459 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:53.459 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 217399, 2024-09-05T12:48:53.459 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 267372, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 70528600, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 204328243, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 11.794004424, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 5.233130116, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 7.739336011, 2024-09-05T12:48:53.460 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.613741543, 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 267373, 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 11.790168609, 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000044096 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:53.461 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 309858, 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.069791312, 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000225 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2025968, 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 254400 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.462 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 35573, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 34846, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 11368600, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 138757727, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 19, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 10, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 3.071904775, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 1.109224532, 2024-09-05T12:48:53.463 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.951869225, 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.994902907, 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 34844, 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.060930979, 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000059147 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 26628, 2024-09-05T12:48:53.464 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.004938853, 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000185 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 5952, 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 683616 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 24110, 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 26262, 2024-09-05T12:48:53.465 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 7834048, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 88610640, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 11, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 2.157595578, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.703322005, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.655144728, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.763954336, 2024-09-05T12:48:53.466 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 26258, 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.759840399, 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000067021 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 16810, 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003626563, 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000215 2024-09-05T12:48:53.467 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2688, 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 12432 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-05T12:48:53.468 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 38423, 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 37.742288099, 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000982283 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:53.469 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 24337, 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.131469679, 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000087581 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 75916, 2024-09-05T12:48:53.470 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 72182, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 72182, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 540.964526122, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007494451 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 3734, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:53.471 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 27, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 2, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 1153, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 174, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 927, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 52, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 905, 2024-09-05T12:48:53.472 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 466, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 135, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 135, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 11, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 122252, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 69945, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 3730, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 77, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:53.473 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 1273, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 2721, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 7, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 398, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 9, 2024-09-05T12:48:53.474 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 500, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 1, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 703, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 221376762, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-09-05T12:48:53.475 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:53.476 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 4, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 6829, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 54521, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 17, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 33698, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 12269, 2024-09-05T12:48:53.477 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 2032, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 54665 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 13, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 78, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 47, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 135, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 75, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-05T12:48:53.478 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 12194, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 12194, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:53.479 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 16, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 1325, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 11, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 2, 2024-09-05T12:48:53.480 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 67445, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 67445, 2024-09-05T12:48:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 95, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 95, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 7, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-05T12:48:53.482 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 254193561, 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 20488, 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 133.626856700, 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.006522201 2024-09-05T12:48:53.483 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 952, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 13453, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 12501, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 139, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 197, 2024-09-05T12:48:53.484 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 58, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 1153, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 22833, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 21680, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 135, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 13662, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 13527, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 75528, 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223520 2024-09-05T12:48:53.485 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 111176, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 6, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 75916, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 2157, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 8367, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 12135, 2024-09-05T12:48:53.486 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 197.965359476, 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.016313585 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 992, 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 9.264755053, 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.009339470 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.487 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.488 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 17168, 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 25.134139741, 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001464010 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.489 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.490 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.491 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.492 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.492 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.492 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-05T12:48:53.492 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.492 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.493 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.494 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-05T12:48:53.495 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 26212, 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 122.854071316, 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004686940 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2757, 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 123.220303413, 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.044693617 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:53.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.497 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.498 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.499 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44, 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.374945059, 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.008521478 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 680, 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.399600175, 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000587647 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.501 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.502 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.503 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 12194, 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 61.751351889, 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005064076 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.504 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-05T12:48:53.505 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 2, 2024-09-05T12:48:53.506 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-05T12:48:53.506 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 10735, 2024-09-05T12:48:53.506 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 5367, 2024-09-05T12:48:53.506 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1250, 2024-09-05T12:48:53.507 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:53.507 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.507 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-05T12:48:53.507 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:53.507 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:53.508 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:53.508 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:53.508 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:53.508 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:53.508 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:53.510 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:53.511 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:53.512 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:53.512 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-05T12:48:53.512 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:53.512 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:53.512 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:53.513 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:53.513 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 362378, 2024-09-05T12:48:53.513 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 484, 2024-09-05T12:48:53.513 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:53.514 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:53.514 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-05T12:48:53.514 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-05T12:48:53.514 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:53.515 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:53.515 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:53.515 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:53.515 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:53.516 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 187, 2024-09-05T12:48:53.516 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:53.516 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:53.516 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:53.516 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 3501990, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 26592, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.517 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 47203, 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 252793039, 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 32, 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 47203, 2024-09-05T12:48:53.518 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 47203, 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 389.681038661, 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.008255429 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 47203, 2024-09-05T12:48:53.519 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 48168 2024-09-05T12:48:53.520 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.520 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 47203, 2024-09-05T12:48:53.520 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 604, 2024-09-05T12:48:53.520 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 46599, 2024-09-05T12:48:53.520 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-05T12:48:53.520 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-05T12:48:53.521 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-09-05T12:48:53.521 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-09-05T12:48:53.521 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 604, 2024-09-05T12:48:53.521 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 21092, 2024-09-05T12:48:53.521 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 393, 2024-09-05T12:48:53.521 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 2, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 24459, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:53.522 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:53.523 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-09-05T12:48:53.523 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-09-05T12:48:53.523 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:53.523 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:53.523 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:53.524 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-05T12:48:53.525 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-05T12:48:53.525 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-05T12:48:53.525 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-05T12:48:53.525 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-05T12:48:53.525 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-05T12:48:53.525 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-05T12:48:53.526 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-05T12:48:53.526 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-05T12:48:53.526 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-05T12:48:53.526 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-05T12:48:53.526 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 128, 2024-09-05T12:48:53.527 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-05T12:48:53.528 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 34, 2024-09-05T12:48:53.528 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-05T12:48:53.528 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-05T12:48:53.528 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-05T12:48:53.528 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-05T12:48:53.528 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 1160, 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 428 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-05T12:48:53.529 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 1211 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 10, 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 5, 2024-09-05T12:48:53.530 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 24388, 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 12194, 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:53.531 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 277078, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 66612356, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 277078, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 277078, 2024-09-05T12:48:53.532 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 66612356, 2024-09-05T12:48:53.533 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:53.533 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.533 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.533 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.533 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.533 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.536 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:53.536 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:53.536 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:53.536 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:53.536 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:53.536 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 47203, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 254124673, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 47203, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 254124673, 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:53.537 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-05T12:48:53.538 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 47203, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 47203, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 47203, 2024-09-05T12:48:53.539 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 47203, 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.540 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 12194, 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1331634, 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:53.541 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 12194, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 604, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1331634, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.542 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55aa6763a960": { 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 67445, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 249999257, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:53.543 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 67445, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 20488, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 249999257, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:53.544 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:53.545 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.545 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:53.545 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-05T12:48:54.348 INFO:tasks.check_counter:Daemon mds.i mds.exported=7 2024-09-05T12:48:54.348 INFO:tasks.check_counter:Daemon mds.i mds.imported=9 2024-09-05T12:48:54.348 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-05T12:48:54.348 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:54.349 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:54.349 DEBUG:tasks.check_counter:Getting stats from l 2024-09-05T12:48:54.349 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-05T12:48:54.597 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.c/config 2024-09-05T12:48:54.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:54 smithi174 ceph-mon[34313]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-09-05T12:48:54.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:54 smithi071 ceph-mon[29845]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-09-05T12:48:54.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:54 smithi123 ceph-mon[34321]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 KiB/s wr, 14 op/s 2024-09-05T12:48:55.143 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-05T12:48:55.143 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-05T12:48:55.144 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2323, 2024-09-05T12:48:55.144 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2325, 2024-09-05T12:48:55.144 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 46846692, 2024-09-05T12:48:55.144 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 738715, 2024-09-05T12:48:55.144 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-05T12:48:55.144 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.243100792, 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.069814609, 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.083746029, 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.069406469, 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2323, 2024-09-05T12:48:55.145 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.114381884, 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000049238 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.146 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2368, 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2307, 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 96964247, 2024-09-05T12:48:55.147 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 847210, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.255489935, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.075728019, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.142210181, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.044213876, 2024-09-05T12:48:55.148 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2307, 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.098644403, 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000042758 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.149 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.150 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.150 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2007008, 2024-09-05T12:48:55.150 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 252800 2024-09-05T12:48:55.150 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.150 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-05T12:48:55.150 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 644, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 945, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 63080348, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 882428, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 9, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.111704828, 2024-09-05T12:48:55.151 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.024665234, 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.062416131, 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.017112814, 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 942, 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.170054159, 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000180524 2024-09-05T12:48:55.152 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-05T12:48:55.153 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 678272 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:55.154 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8099, 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.191853531, 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000023688 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-05T12:48:55.155 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-05T12:48:55.156 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.157 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-05T12:48:55.158 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 11481, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 24, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 11457, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 11447, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-05T12:48:55.159 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 13, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-05T12:48:55.160 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-05T12:48:55.161 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-05T12:48:55.162 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-05T12:48:55.163 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-05T12:48:55.164 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-05T12:48:55.165 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-05T12:48:55.166 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-05T12:48:55.167 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.168 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 59739, 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-05T12:48:55.169 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 81, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 6, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-05T12:48:55.170 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 207709657, 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 207695958, 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.171 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 59739 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 1221, 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 11494, 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 10273, 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 65, 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 75, 2024-09-05T12:48:55.172 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 10, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 11481, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 11630, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 149, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 70440, 2024-09-05T12:48:55.173 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215328 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-05T12:48:55.174 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.175 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.176 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.177 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-05T12:48:55.177 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.177 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.177 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.180 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.180 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-05T12:48:55.180 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.180 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.181 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-05T12:48:55.182 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.183 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.184 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-05T12:48:55.185 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.185 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.185 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.185 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-05T12:48:55.186 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.187 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-05T12:48:55.188 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.189 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.190 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.191 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.192 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.193 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-05T12:48:55.193 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.193 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.193 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.193 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.193 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.194 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.195 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-05T12:48:55.196 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.197 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-05T12:48:55.198 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-09-05T12:48:55.199 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 5, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 3, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 4, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 4029, 2024-09-05T12:48:55.200 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-05T12:48:55.201 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-05T12:48:55.202 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-05T12:48:55.203 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-05T12:48:55.204 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-05T12:48:55.205 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 294223, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 1463, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-05T12:48:55.206 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 9296, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 187, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-05T12:48:55.207 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 11276739, 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 78035, 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-05T12:48:55.208 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-05T12:48:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-05T12:48:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-05T12:48:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 4834, 2024-09-05T12:48:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-09-05T12:48:55.209 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 4834, 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4834, 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 5.661927767, 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001171271 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.210 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4834, 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4835 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 4834, 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 4834, 2024-09-05T12:48:55.211 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-05T12:48:55.212 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-05T12:48:55.212 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-05T12:48:55.212 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2805, 2024-09-05T12:48:55.212 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-05T12:48:55.212 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 2027, 2024-09-05T12:48:55.212 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-05T12:48:55.213 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-05T12:48:55.214 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-05T12:48:55.215 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-05T12:48:55.216 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-05T12:48:55.217 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 128, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 34, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 11, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-05T12:48:55.218 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-05T12:48:55.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-05T12:48:55.220 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:55.221 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 5331, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 206383375, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 5331, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:55.222 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5331, 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 206383375, 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.223 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:55.224 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4834, 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 203541137, 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4834, 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 203541137, 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.225 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:55.226 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4834, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 4834, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4834, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4834, 2024-09-05T12:48:55.227 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:55.228 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:55.229 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-05T12:48:55.230 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55ad80960b40": { 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-05T12:48:55.231 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-05T12:48:55.232 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-05T12:48:55.233 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.233 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-05T12:48:55.233 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-05T12:48:55.912 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-05T12:48:55.912 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-05T12:48:55.912 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-09-05T12:48:55.913 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-09-05T12:48:55.913 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-09-05T12:48:55.913 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-05T12:48:55.923 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-05T12:48:55.923 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:48:55.923 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:48:55.965 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-09-05T12:48:55.965 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-05T12:48:56.043 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd blocklist ls 2024-09-05T12:48:56.456 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:48:56.821 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:56 smithi071 ceph-mon[29845]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-05T12:48:56.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:56 smithi123 ceph-mon[34321]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-05T12:48:57.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:56 smithi174 ceph-mon[34313]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-09-05T12:48:57.045 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 -- 172.21.15.71:0/1964039308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 msgr2=0x7f68701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 --2- 172.21.15.71:0/1964039308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f68701067e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f6858009900 tx=0x7f685802a410 comp rx=0 tx=0).stop 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 -- 172.21.15.71:0/1964039308 shutdown_connections 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 --2- 172.21.15.71:0/1964039308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870106f10 0x7f687010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 --2- 172.21.15.71:0/1964039308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f68701067e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 --2- 172.21.15.71:0/1964039308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6870101de0 0x7f68701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 -- 172.21.15.71:0/1964039308 >> 172.21.15.71:0/1964039308 conn(0x7f68700fd8c0 msgr2=0x7f68700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 -- 172.21.15.71:0/1964039308 shutdown_connections 2024-09-05T12:48:57.046 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.044+0000 7f6875c00640 1 -- 172.21.15.71:0/1964039308 wait complete. 2024-09-05T12:48:57.048 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.045+0000 7f6875c00640 1 Processor -- start 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.045+0000 7f6875c00640 1 -- start start 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.045+0000 7f6875c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 0x7f687019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6875c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f687019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6875c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6870106f10 0x7f687019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6875c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6870111f80 con 0x7f6870102790 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6875c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6870111e00 con 0x7f6870106f10 2024-09-05T12:48:57.049 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6875c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6870112100 con 0x7f6870101de0 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6866000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f687019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 0x7f687019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 0x7f687019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53840/0 (socket says 172.21.15.71:53840) 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 -- 172.21.15.71:0/3894228724 learned_addr learned my addr 172.21.15.71:0/3894228724 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686f400640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6870106f10 0x7f687019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 -- 172.21.15.71:0/3894228724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6870106f10 msgr2=0x7f687019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6870106f10 0x7f687019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.050 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 -- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 msgr2=0x7f687019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f687019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f686ea00640 1 -- 172.21.15.71:0/3894228724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687019f690 con 0x7f6870101de0 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.046+0000 7f6866000640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f687019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.047+0000 7f686ea00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 0x7f687019a6e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f686000ebc0 tx=0x7f686000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.047+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860010630 con 0x7f6870101de0 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.047+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6860010cb0 con 0x7f6870101de0 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.047+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687019f980 con 0x7f6870101de0 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.047+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6860018bd0 con 0x7f6870101de0 2024-09-05T12:48:57.051 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.047+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68701a7260 con 0x7f6870101de0 2024-09-05T12:48:57.052 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.048+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68701021c0 con 0x7f6870101de0 2024-09-05T12:48:57.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.050+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f68600107d0 con 0x7f6870101de0 2024-09-05T12:48:57.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.051+0000 7f6867e00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f68540a2cf0 0x7f68540a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:48:57.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.052+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f68600cfe40 con 0x7f6870101de0 2024-09-05T12:48:57.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.052+0000 7f6866000640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f68540a2cf0 0x7f68540a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:48:57.053 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.052+0000 7f6866000640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f68540a2cf0 0x7f68540a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6858000c00 tx=0x7f6858031510 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:48:57.056 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.056+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f686009c040 con 0x7f6870101de0 2024-09-05T12:48:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.244+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f68701a7430 con 0x7f6870101de0 2024-09-05T12:48:57.245 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.245+0000 7f6867e00640 1 -- 172.21.15.71:0/3894228724 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v128) ==== 81+0+945 (secure 0 0 0) 0x7f68600a0340 con 0x7f6870101de0 2024-09-05T12:48:57.246 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2607784636 2024-09-06T12:27:44.498834+0000 2024-09-05T12:48:57.246 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4208428214 2024-09-06T12:27:44.498834+0000 2024-09-05T12:48:57.246 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/2050574497 2024-09-06T12:27:44.498834+0000 2024-09-05T12:48:57.246 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4234071631 2024-09-06T12:17:04.816418+0000 2024-09-05T12:48:57.246 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/4092691810 2024-09-06T12:17:04.816418+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3671256491 2024-09-06T12:17:33.045867+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/1559694750 2024-09-06T12:17:04.816418+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/2578458480 2024-09-06T12:17:04.816418+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3836892613 2024-09-06T12:17:33.045867+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/1559694750 2024-09-06T12:17:04.816418+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/3648851306 2024-09-06T12:17:33.045867+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6801/3648851306 2024-09-06T12:17:33.045867+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:6800/2050574497 2024-09-06T12:27:44.498834+0000 2024-09-05T12:48:57.247 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/1492294353 2024-09-06T12:17:33.045867+0000 2024-09-05T12:48:57.248 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/3957197147 2024-09-06T12:27:44.498834+0000 2024-09-05T12:48:57.248 INFO:teuthology.orchestra.run.smithi071.stdout:172.21.15.71:0/372010080 2024-09-06T12:27:44.498834+0000 2024-09-05T12:48:57.248 INFO:teuthology.orchestra.run.smithi071.stderr:listed 16 entries 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f68540a2cf0 msgr2=0x7f68540a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f68540a2cf0 0x7f68540a51b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6858000c00 tx=0x7f6858031510 comp rx=0 tx=0).stop 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 msgr2=0x7f687019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 0x7f687019a6e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f686000ebc0 tx=0x7f686000c6a0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 shutdown_connections 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6870106f10 0x7f687019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f68540a2cf0 0x7f68540a51b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.249 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6870102790 0x7f687019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 --2- 172.21.15.71:0/3894228724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6870101de0 0x7f687019a6e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:48:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.247+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 >> 172.21.15.71:0/3894228724 conn(0x7f68700fd8c0 msgr2=0x7f68700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:48:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.248+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 shutdown_connections 2024-09-05T12:48:57.250 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:48:57.248+0000 7f6875c00640 1 -- 172.21.15.71:0/3894228724 wait complete. 2024-09-05T12:48:57.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:57 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3894228724' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:48:57.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:57 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3894228724' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:48:58.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:57 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3894228724' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-05T12:48:58.193 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-05T12:48:58.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:48:58.194 DEBUG:teuthology.orchestra.run.smithi071:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-05T12:48:58.466 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:48:58.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:48:58.466 DEBUG:teuthology.orchestra.run.smithi071:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-05T12:48:58.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:48:58.550 DEBUG:teuthology.orchestra.run.smithi071:> ip netns list 2024-09-05T12:48:58.620 INFO:teuthology.orchestra.run.smithi071.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-05T12:48:58.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:48:58.620 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-05T12:48:58.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-05T12:48:58.714 DEBUG:teuthology.orchestra.run.smithi071:> sudo ip link delete ceph-brx 2024-09-05T12:48:58.719 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:48:58 smithi071 ceph-mon[29845]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-09-05T12:48:58.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:48:58 smithi123 ceph-mon[34321]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-09-05T12:48:59.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:48:58 smithi174 ceph-mon[34313]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 32 KiB/s wr, 17 op/s 2024-09-05T12:48:59.056 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-05T12:48:59.067 INFO:tasks.cephadm:Teardown begin 2024-09-05T12:48:59.068 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-05T12:48:59.068 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:48:59.438 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:00.078 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 -- 172.21.15.71:0/2252352925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a74109120 msgr2=0x7f8a74110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2252352925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a74109120 0x7f8a74110ca0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009900 tx=0x7f8a6c02a3f0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 -- 172.21.15.71:0/2252352925 shutdown_connections 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2252352925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a74109120 0x7f8a74110ca0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2252352925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a741049a0 0x7f8a741089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2252352925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74103ff0 0x7f8a741043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 -- 172.21.15.71:0/2252352925 >> 172.21.15.71:0/2252352925 conn(0x7f8a740ffa90 msgr2=0x7f8a74101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 -- 172.21.15.71:0/2252352925 shutdown_connections 2024-09-05T12:49:00.079 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.069+0000 7f8a7b400640 1 -- 172.21.15.71:0/2252352925 wait complete. 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.070+0000 7f8a7b400640 1 Processor -- start 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.070+0000 7f8a7b400640 1 -- start start 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a7b400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74103ff0 0x7f8a7419a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a7b400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 0x7f8a7419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a7b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74109120 0x7f8a7419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a7b400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74114130 con 0x7f8a741049a0 2024-09-05T12:49:00.080 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a7b400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74113fb0 con 0x7f8a74109120 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a7b400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741142b0 con 0x7f8a74103ff0 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a73e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 0x7f8a7419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a78a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74103ff0 0x7f8a7419a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a73e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 0x7f8a7419aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50870/0 (socket says 172.21.15.71:50870) 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a73e00640 1 -- 172.21.15.71:0/2844845774 learned_addr learned my addr 172.21.15.71:0/2844845774 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a79400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74109120 0x7f8a7419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a78a00640 1 -- 172.21.15.71:0/2844845774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74109120 msgr2=0x7f8a7419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:00.081 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a78a00640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74109120 0x7f8a7419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a78a00640 1 -- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 msgr2=0x7f8a7419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a78a00640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 0x7f8a7419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a78a00640 1 -- 172.21.15.71:0/2844845774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7419f540 con 0x7f8a74103ff0 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.071+0000 7f8a73e00640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 0x7f8a7419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.072+0000 7f8a78a00640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74103ff0 0x7f8a7419a590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400dab0 tx=0x7f8a6400df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.072+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6400be30 con 0x7f8a74103ff0 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.072+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a64003e50 con 0x7f8a74103ff0 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.072+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7419f830 con 0x7f8a74103ff0 2024-09-05T12:49:00.082 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.072+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640106c0 con 0x7f8a74103ff0 2024-09-05T12:49:00.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.072+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a7110 con 0x7f8a74103ff0 2024-09-05T12:49:00.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.073+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a741043d0 con 0x7f8a74103ff0 2024-09-05T12:49:00.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.075+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f8a64004000 con 0x7f8a74103ff0 2024-09-05T12:49:00.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.077+0000 7f8a71600640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8a580a2cf0 0x7f8a580a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:00.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.077+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f8a640cff00 con 0x7f8a74103ff0 2024-09-05T12:49:00.083 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.077+0000 7f8a73e00640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8a580a2cf0 0x7f8a580a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:00.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.078+0000 7f8a73e00640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8a580a2cf0 0x7f8a580a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a740ffeb0 tx=0x7f8a6000b480 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:00.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.079+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a6409c750 con 0x7f8a74103ff0 2024-09-05T12:49:00.293 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.292+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a7419b8f0 con 0x7f8a74103ff0 2024-09-05T12:49:00.294 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.293+0000 7f8a71600640 1 -- 172.21.15.71:0/2844845774 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7f8a6409c050 con 0x7f8a74103ff0 2024-09-05T12:49:00.294 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:49:00.295 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:49:00.295 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8a580a2cf0 msgr2=0x7f8a580a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8a580a2cf0 0x7f8a580a51b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8a740ffeb0 tx=0x7f8a6000b480 comp rx=0 tx=0).stop 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74103ff0 msgr2=0x7f8a7419a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74103ff0 0x7f8a7419a590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400dab0 tx=0x7f8a6400df70 comp rx=0 tx=0).stop 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 shutdown_connections 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74109120 0x7f8a7419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.297 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f8a580a2cf0 0x7f8a580a51b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a741049a0 0x7f8a7419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 --2- 172.21.15.71:0/2844845774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a74103ff0 0x7f8a7419a590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.295+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 >> 172.21.15.71:0/2844845774 conn(0x7f8a740ffa90 msgr2=0x7f8a74101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.296+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 shutdown_connections 2024-09-05T12:49:00.298 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:00.296+0000 7f8a7b400640 1 -- 172.21.15.71:0/2844845774 wait complete. 2024-09-05T12:49:00.807 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:00 smithi071 ceph-mon[29845]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-05T12:49:00.808 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:00 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/2844845774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:49:00.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:00 smithi123 ceph-mon[34321]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-05T12:49:00.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:00 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/2844845774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:49:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:00 smithi174 ceph-mon[34313]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-09-05T12:49:01.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:00 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/2844845774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:49:01.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs dump --format=json 2024-09-05T12:49:01.496 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:02.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 -- 172.21.15.71:0/3013178752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 msgr2=0x7f29540749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:02.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 --2- 172.21.15.71:0/3013178752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f29540749a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2944009ca0 tx=0x7f294402a950 comp rx=0 tx=0).stop 2024-09-05T12:49:02.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 -- 172.21.15.71:0/3013178752 shutdown_connections 2024-09-05T12:49:02.127 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 --2- 172.21.15.71:0/3013178752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f295410e260 0x7f2954110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 --2- 172.21.15.71:0/3013178752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 --2- 172.21.15.71:0/3013178752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f29540749a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.125+0000 7f2958c00640 1 -- 172.21.15.71:0/3013178752 >> 172.21.15.71:0/3013178752 conn(0x7f29540fd6e0 msgr2=0x7f29540ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:02.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.126+0000 7f2958c00640 1 -- 172.21.15.71:0/3013178752 shutdown_connections 2024-09-05T12:49:02.128 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.126+0000 7f2958c00640 1 -- 172.21.15.71:0/3013178752 wait complete. 2024-09-05T12:49:02.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 Processor -- start 2024-09-05T12:49:02.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 -- start start 2024-09-05T12:49:02.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f295419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:02.130 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f295410e260 0x7f295419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2954114000 con 0x7f29540745c0 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2954113e80 con 0x7f295410e260 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.127+0000 7f2958c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2954114180 con 0x7f2954074ee0 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f295419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2952a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2952a00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295419a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:53878/0 (socket says 172.21.15.71:53878) 2024-09-05T12:49:02.131 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2952a00640 1 -- 172.21.15.71:0/1160885000 learned_addr learned my addr 172.21.15.71:0/1160885000 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953e00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f295410e260 0x7f295419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953400640 1 -- 172.21.15.71:0/1160885000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 msgr2=0x7f295419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953400640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953400640 1 -- 172.21.15.71:0/1160885000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f295410e260 msgr2=0x7f295419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953400640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f295410e260 0x7f295419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2953400640 1 -- 172.21.15.71:0/1160885000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295419f450 con 0x7f29540745c0 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.128+0000 7f2952a00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:49:02.132 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.129+0000 7f2953400640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f295419a4a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2944005e30 tx=0x7f2944003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:02.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.129+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944034040 con 0x7f29540745c0 2024-09-05T12:49:02.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.129+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295419f6e0 con 0x7f29540745c0 2024-09-05T12:49:02.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.129+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294402fe20 con 0x7f29540745c0 2024-09-05T12:49:02.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.129+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2944004830 con 0x7f29540745c0 2024-09-05T12:49:02.133 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.129+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29541a7020 con 0x7f29540745c0 2024-09-05T12:49:02.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.130+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29540749a0 con 0x7f29540745c0 2024-09-05T12:49:02.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.132+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f2944037020 con 0x7f29540745c0 2024-09-05T12:49:02.134 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.133+0000 7f294be00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29380a2d70 0x7f29380a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:02.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.134+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f29440f04b0 con 0x7f29540745c0 2024-09-05T12:49:02.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.134+0000 7f2952a00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29380a2d70 0x7f29380a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:02.135 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.134+0000 7f2952a00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29380a2d70 0x7f29380a5230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f295410f580 tx=0x7f294000b410 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:02.139 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.138+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29440bce10 con 0x7f29540745c0 2024-09-05T12:49:02.351 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.350+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29541093e0 con 0x7f29540745c0 2024-09-05T12:49:02.352 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.351+0000 7f294be00640 1 -- 172.21.15.71:0/1160885000 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+11971 (secure 0 0 0) 0x7f29440bc710 con 0x7f29540745c0 2024-09-05T12:49:02.352 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:49:02.353 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":76,"btime":"2024-09-05T12:45:24:749498+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14418,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/83855172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":83855172},{"type":"v1","addr":"172.21.15.71:6835","nonce":83855172}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14430,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/2997614259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":2997614259},{"type":"v1","addr":"172.21.15.71:6837","nonce":2997614259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":76,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-05T12:24:49.631865+0000","modified":"2024-09-05T12:45:24.134322+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24514,"mds_2":24502,"mds_3":24490,"mds_4":24485},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14469":{"gid":14469,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6841/991266062","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6840","nonce":991266062},{"type":"v1","addr":"172.21.15.71:6841","nonce":991266062}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/679053656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":679053656},{"type":"v1","addr":"172.21.15.174:6833","nonce":679053656}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1436525377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1436525377},{"type":"v1","addr":"172.21.15.123:6833","nonce":1436525377}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6835/2774200097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2774200097},{"type":"v1","addr":"172.21.15.123:6835","nonce":2774200097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":20,"addr":"172.21.15.123:6837/3417870839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":3417870839},{"type":"v1","addr":"172.21.15.123:6837","nonce":3417870839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24490":{"gid":24490,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":20,"addr":"172.21.15.174:6837/870087433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":870087433},{"type":"v1","addr":"172.21.15.174:6837","nonce":870087433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.174:6839/584492863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":584492863},{"type":"v1","addr":"172.21.15.174:6839","nonce":584492863}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"g","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.71:6839/2593934735","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":2593934735},{"type":"v1","addr":"172.21.15.71:6839","nonce":2593934735}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.123:6839/782979893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":782979893},{"type":"v1","addr":"172.21.15.123:6839","nonce":782979893}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26545":{"gid":26545,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2481371871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2481371871},{"type":"v1","addr":"172.21.15.174:6835","nonce":2481371871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24514,24502,24490,24524,24485]},"id":1}]} 2024-09-05T12:49:02.353 INFO:teuthology.orchestra.run.smithi071.stderr:dumped fsmap epoch 76 2024-09-05T12:49:02.354 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29380a2d70 msgr2=0x7f29380a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29380a2d70 0x7f29380a5230 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f295410f580 tx=0x7f294000b410 comp rx=0 tx=0).stop 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 msgr2=0x7f295419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f295419a4a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2944005e30 tx=0x7f2944003d60 comp rx=0 tx=0).stop 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 shutdown_connections 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f295410e260 0x7f295419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2954074ee0 0x7f295419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.355 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f29380a2d70 0x7f29380a5230 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 --2- 172.21.15.71:0/1160885000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f29540745c0 0x7f295419a4a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:02.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 >> 172.21.15.71:0/1160885000 conn(0x7f29540fd6e0 msgr2=0x7f29540fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:02.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 shutdown_connections 2024-09-05T12:49:02.356 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:02.353+0000 7f2958c00640 1 -- 172.21.15.71:0/1160885000 wait complete. 2024-09-05T12:49:02.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:02 smithi071 ceph-mon[29845]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 138 KiB/s wr, 19 op/s 2024-09-05T12:49:02.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:02 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1160885000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:49:02.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:02 smithi123 ceph-mon[34321]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 138 KiB/s wr, 19 op/s 2024-09-05T12:49:02.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:02 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1160885000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:49:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:02 smithi174 ceph-mon[34313]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 138 KiB/s wr, 19 op/s 2024-09-05T12:49:03.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:02 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1160885000' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-05T12:49:03.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd dump --format=json 2024-09-05T12:49:03.906 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:04.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 -- 172.21.15.71:0/2523231111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 msgr2=0x7f788c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:04.583 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 --2- 172.21.15.71:0/2523231111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c10ea90 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7884009ca0 tx=0x7f788402a8b0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 -- 172.21.15.71:0/2523231111 shutdown_connections 2024-09-05T12:49:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 --2- 172.21.15.71:0/2523231111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c10ea90 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 --2- 172.21.15.71:0/2523231111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788c102790 0x7f788c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 --2- 172.21.15.71:0/2523231111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c101de0 0x7f788c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 -- 172.21.15.71:0/2523231111 >> 172.21.15.71:0/2523231111 conn(0x7f788c0fd8c0 msgr2=0x7f788c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:04.584 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 -- 172.21.15.71:0/2523231111 shutdown_connections 2024-09-05T12:49:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.515+0000 7f7892000640 1 -- 172.21.15.71:0/2523231111 wait complete. 2024-09-05T12:49:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 Processor -- start 2024-09-05T12:49:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 -- start start 2024-09-05T12:49:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c101de0 0x7f788c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788c102790 0x7f788c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:04.585 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f788c111f70 con 0x7f788c106f10 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f788c111df0 con 0x7f788c101de0 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.517+0000 7f7892000640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f788c1120f0 con 0x7f788c102790 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788aa00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788c102790 0x7f788c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41802/0 (socket says 172.21.15.71:41802) 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 -- 172.21.15.71:0/45552728 learned_addr learned my addr 172.21.15.71:0/45552728 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:04.586 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788b400640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c101de0 0x7f788c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 -- 172.21.15.71:0/45552728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788c102790 msgr2=0x7f788c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788c102790 0x7f788c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 -- 172.21.15.71:0/45552728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c101de0 msgr2=0x7f788c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c101de0 0x7f788c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 -- 172.21.15.71:0/45552728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c1a68b0 con 0x7f788c106f10 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.518+0000 7f788be00640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c074f70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7884006010 tx=0x7f7884009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.519+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884004040 con 0x7f788c106f10 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.519+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c1a6b40 con 0x7f788c106f10 2024-09-05T12:49:04.587 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.519+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7884040070 con 0x7f788c106f10 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.519+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7884045bb0 con 0x7f788c106f10 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.519+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c1a7080 con 0x7f788c106f10 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.520+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c1021c0 con 0x7f788c106f10 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.522+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f78840041e0 con 0x7f788c106f10 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.523+0000 7f7883e00640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78700a2cf0 0x7f78700a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.523+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f78840f0b90 con 0x7f788c106f10 2024-09-05T12:49:04.588 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.523+0000 7f788b400640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78700a2cf0 0x7f78700a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:04.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.524+0000 7f788b400640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78700a2cf0 0x7f78700a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f787c00bbe0 tx=0x7f787c00b390 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:04.589 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.527+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78840bd580 con 0x7f788c106f10 2024-09-05T12:49:04.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:04 smithi071 ceph-mon[29845]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 183 KiB/s wr, 32 op/s 2024-09-05T12:49:04.714 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.713+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f788c10a0f0 con 0x7f788c106f10 2024-09-05T12:49:04.715 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.714+0000 7f7883e00640 1 -- 172.21.15.71:0/45552728 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+25237 (secure 0 0 0) 0x7f78840047e0 con 0x7f788c106f10 2024-09-05T12:49:04.716 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-09-05T12:49:04.716 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":128,"fsid":"8c9ae41c-6b80-11ef-bcd6-c7b262605968","created":"2024-09-05T12:16:33.931712+0000","modified":"2024-09-05T12:48:15.393851+0000","last_up_change":"2024-09-05T12:23:53.223167+0000","last_in_change":"2024-09-05T12:23:31.143776+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-05T12:19:53.639902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-05T12:24:44.687716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-05T12:24:47.492194+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-05T12:24:57.061844+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":37,"snap_epoch":126,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"43d56d11-d83e-4c6b-be5b-80824118d6a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6803","nonce":245899056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6805","nonce":245899056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6809","nonce":245899056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":245899056},{"type":"v1","addr":"172.21.15.71:6807","nonce":245899056}]},"public_addr":"172.21.15.71:6803/245899056","cluster_addr":"172.21.15.71:6805/245899056","heartbeat_back_addr":"172.21.15.71:6809/245899056","heartbeat_front_addr":"172.21.15.71:6807/245899056","state":["exists","up"]},{"osd":1,"uuid":"4234dbed-6c5c-4abf-892a-d13fc22b44b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6801","nonce":1018377338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6803","nonce":1018377338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6807","nonce":1018377338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1018377338},{"type":"v1","addr":"172.21.15.123:6805","nonce":1018377338}]},"public_addr":"172.21.15.123:6801/1018377338","cluster_addr":"172.21.15.123:6803/1018377338","heartbeat_back_addr":"172.21.15.123:6807/1018377338","heartbeat_front_addr":"172.21.15.123:6805/1018377338","state":["exists","up"]},{"osd":2,"uuid":"c1b0236e-25dc-47d7-b952-edec01683c2e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6801","nonce":1307000891}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6803","nonce":1307000891}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6807","nonce":1307000891}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":1307000891},{"type":"v1","addr":"172.21.15.174:6805","nonce":1307000891}]},"public_addr":"172.21.15.174:6801/1307000891","cluster_addr":"172.21.15.174:6803/1307000891","heartbeat_back_addr":"172.21.15.174:6807/1307000891","heartbeat_front_addr":"172.21.15.174:6805/1307000891","state":["exists","up"]},{"osd":3,"uuid":"9b33ca22-8dc7-41ca-8cd6-44d700c3d3da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6811","nonce":3336671100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6813","nonce":3336671100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6817","nonce":3336671100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3336671100},{"type":"v1","addr":"172.21.15.71:6815","nonce":3336671100}]},"public_addr":"172.21.15.71:6811/3336671100","cluster_addr":"172.21.15.71:6813/3336671100","heartbeat_back_addr":"172.21.15.71:6817/3336671100","heartbeat_front_addr":"172.21.15.71:6815/3336671100","state":["exists","up"]},{"osd":4,"uuid":"f983652a-0773-4d6d-9cac-5a0f4602a729","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6809","nonce":1661996031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6811","nonce":1661996031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6815","nonce":1661996031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1661996031},{"type":"v1","addr":"172.21.15.123:6813","nonce":1661996031}]},"public_addr":"172.21.15.123:6809/1661996031","cluster_addr":"172.21.15.123:6811/1661996031","heartbeat_back_addr":"172.21.15.123:6815/1661996031","heartbeat_front_addr":"172.21.15.123:6813/1661996031","state":["exists","up"]},{"osd":5,"uuid":"379b7d31-7108-49f2-a223-93c1214c9460","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6809","nonce":2379389937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6811","nonce":2379389937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6815","nonce":2379389937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":2379389937},{"type":"v1","addr":"172.21.15.174:6813","nonce":2379389937}]},"public_addr":"172.21.15.174:6809/2379389937","cluster_addr":"172.21.15.174:6811/2379389937","heartbeat_back_addr":"172.21.15.174:6815/2379389937","heartbeat_front_addr":"172.21.15.174:6813/2379389937","state":["exists","up"]},{"osd":6,"uuid":"6ac35b8a-e077-44a6-91ea-1e5efd56d8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6819","nonce":1490057676}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6821","nonce":1490057676}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6825","nonce":1490057676}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1490057676},{"type":"v1","addr":"172.21.15.71:6823","nonce":1490057676}]},"public_addr":"172.21.15.71:6819/1490057676","cluster_addr":"172.21.15.71:6821/1490057676","heartbeat_back_addr":"172.21.15.71:6825/1490057676","heartbeat_front_addr":"172.21.15.71:6823/1490057676","state":["exists","up"]},{"osd":7,"uuid":"43640d60-2b1a-44c8-8be5-e1540265c057","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6817","nonce":83143340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6819","nonce":83143340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6823","nonce":83143340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":83143340},{"type":"v1","addr":"172.21.15.123:6821","nonce":83143340}]},"public_addr":"172.21.15.123:6817/83143340","cluster_addr":"172.21.15.123:6819/83143340","heartbeat_back_addr":"172.21.15.123:6823/83143340","heartbeat_front_addr":"172.21.15.123:6821/83143340","state":["exists","up"]},{"osd":8,"uuid":"8a331c8c-f658-4b3c-b1a8-25199643e7d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6817","nonce":2584468270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6819","nonce":2584468270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6823","nonce":2584468270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":2584468270},{"type":"v1","addr":"172.21.15.174:6821","nonce":2584468270}]},"public_addr":"172.21.15.174:6817/2584468270","cluster_addr":"172.21.15.174:6819/2584468270","heartbeat_back_addr":"172.21.15.174:6823/2584468270","heartbeat_front_addr":"172.21.15.174:6821/2584468270","state":["exists","up"]},{"osd":9,"uuid":"9fcc4a9e-318d-482f-babb-6ad8e5ff5790","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6827","nonce":4063696611}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6829","nonce":4063696611}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6833","nonce":4063696611}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":4063696611},{"type":"v1","addr":"172.21.15.71:6831","nonce":4063696611}]},"public_addr":"172.21.15.71:6827/4063696611","cluster_addr":"172.21.15.71:6829/4063696611","heartbeat_back_addr":"172.21.15.71:6833/4063696611","heartbeat_front_addr":"172.21.15.71:6831/4063696611","state":["exists","up"]},{"osd":10,"uuid":"cbae3521-9c6b-45e7-9912-2afebeecdf3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6825","nonce":256837648}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6827","nonce":256837648}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6831","nonce":256837648}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":256837648},{"type":"v1","addr":"172.21.15.123:6829","nonce":256837648}]},"public_addr":"172.21.15.123:6825/256837648","cluster_addr":"172.21.15.123:6827/256837648","heartbeat_back_addr":"172.21.15.123:6831/256837648","heartbeat_front_addr":"172.21.15.123:6829/256837648","state":["exists","up"]},{"osd":11,"uuid":"3749498e-3656-4d1e-aa2c-40b565b46488","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6825","nonce":1335368098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6827","nonce":1335368098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6831","nonce":1335368098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1335368098},{"type":"v1","addr":"172.21.15.174:6829","nonce":1335368098}]},"public_addr":"172.21.15.174:6825/1335368098","cluster_addr":"172.21.15.174:6827/1335368098","heartbeat_back_addr":"172.21.15.174:6831/1335368098","heartbeat_front_addr":"172.21.15.174:6829/1335368098","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:01.873529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:26.941759+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:19:50.896690+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:20.396527+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:20:46.376016+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:10.593307+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:21:41.500882+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:07.748064+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:22:32.061862+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:00.979593+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:27.118380+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-05T12:23:51.683961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.23","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.7","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.10","mappings":[{"from":9,"to":8}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.71:0/2607784636":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/4208428214":"2024-09-06T12:27:44.498834+0000","172.21.15.71:6801/2050574497":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/4234071631":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/4092691810":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3671256491":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/2578458480":"2024-09-06T12:17:04.816418+0000","172.21.15.71:0/3836892613":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/1559694750":"2024-09-06T12:17:04.816418+0000","172.21.15.71:6800/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6801/3648851306":"2024-09-06T12:17:33.045867+0000","172.21.15.71:6800/2050574497":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/1492294353":"2024-09-06T12:17:33.045867+0000","172.21.15.71:0/3957197147":"2024-09-06T12:27:44.498834+0000","172.21.15.71:0/372010080":"2024-09-06T12:27:44.498834+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":24,"length":1},{"begin":37,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-05T12:49:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78700a2cf0 msgr2=0x7f78700a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78700a2cf0 0x7f78700a51b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f787c00bbe0 tx=0x7f787c00b390 comp rx=0 tx=0).stop 2024-09-05T12:49:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 msgr2=0x7f788c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:04.718 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c074f70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7884006010 tx=0x7f7884009360 comp rx=0 tx=0).stop 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 shutdown_connections 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f78700a2cf0 0x7f78700a51b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f788c106f10 0x7f788c074f70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f788c102790 0x7f788c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 --2- 172.21.15.71:0/45552728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f788c101de0 0x7f788c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 >> 172.21.15.71:0/45552728 conn(0x7f788c0fd8c0 msgr2=0x7f788c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 shutdown_connections 2024-09-05T12:49:04.719 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:04.717+0000 7f7892000640 1 -- 172.21.15.71:0/45552728 wait complete. 2024-09-05T12:49:04.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:04 smithi123 ceph-mon[34321]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 183 KiB/s wr, 32 op/s 2024-09-05T12:49:05.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:04 smithi174 ceph-mon[34313]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 183 KiB/s wr, 32 op/s 2024-09-05T12:49:05.694 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs fail cephfs 2024-09-05T12:49:05.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:05 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/45552728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:49:05.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:05 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/45552728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:49:06.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:05 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/45552728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-05T12:49:06.065 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:06.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 -- 172.21.15.71:0/3055160258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4112b90 msgr2=0x7f90e4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:06.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3055160258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4112b90 0x7f90e4114f80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f90dc009ca0 tx=0x7f90dc02a950 comp rx=0 tx=0).stop 2024-09-05T12:49:06.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 -- 172.21.15.71:0/3055160258 shutdown_connections 2024-09-05T12:49:06.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3055160258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4112b90 0x7f90e4114f80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3055160258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e4074eb0 0x7f90e4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.745 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3055160258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e4074590 0x7f90e4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 -- 172.21.15.71:0/3055160258 >> 172.21.15.71:0/3055160258 conn(0x7f90e40fd8c0 msgr2=0x7f90e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.690+0000 7f90e8c00640 1 -- 172.21.15.71:0/3055160258 shutdown_connections 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.691+0000 7f90e8c00640 1 -- 172.21.15.71:0/3055160258 wait complete. 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.691+0000 7f90e8c00640 1 Processor -- start 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 -- start start 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e4074590 0x7f90e41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 0x7f90e41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:06.746 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e4112b90 0x7f90e419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f90e4118560 con 0x7f90e4074eb0 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90e41183e0 con 0x7f90e4074590 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e8c00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90e41186e0 con 0x7f90e4112b90 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 0x7f90e41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e1400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 0x7f90e41a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41810/0 (socket says 172.21.15.71:41810) 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e1400640 1 -- 172.21.15.71:0/3896208762 learned_addr learned my addr 172.21.15.71:0/3896208762 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e1e00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e4074590 0x7f90e41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.692+0000 7f90e2800640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e4112b90 0x7f90e419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:06.747 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e1400640 1 -- 172.21.15.71:0/3896208762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e4112b90 msgr2=0x7f90e419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e1400640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e4112b90 0x7f90e419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e1400640 1 -- 172.21.15.71:0/3896208762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e4074590 msgr2=0x7f90e41a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e1400640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e4074590 0x7f90e41a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e1400640 1 -- 172.21.15.71:0/3896208762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e419f020 con 0x7f90e4074eb0 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e1400640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 0x7f90e41a4c70 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e9f0 tx=0x7f90d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d000cd80 con 0x7f90e4074eb0 2024-09-05T12:49:06.748 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90d0010910 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e419f310 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0018990 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.693+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e41ab600 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.694+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90e4074970 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.696+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f90d0010ac0 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.697+0000 7f90daa00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f90c80a2cf0 0x7f90c80a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.698+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9394+0+0 (secure 0 0 0) 0x7f90d00cfeb0 con 0x7f90e4074eb0 2024-09-05T12:49:06.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.698+0000 7f90e1e00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f90c80a2cf0 0x7f90c80a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:06.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.698+0000 7f90e1e00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f90c80a2cf0 0x7f90c80a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90e419fa30 tx=0x7f90d4005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:06.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.700+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f90d009c0b0 con 0x7f90e4074eb0 2024-09-05T12:49:06.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:06 smithi071 ceph-mon[29845]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 173 KiB/s wr, 29 op/s 2024-09-05T12:49:06.921 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.920+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f90e419fe20 con 0x7f90e4074eb0 2024-09-05T12:49:06.924 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.923+0000 7f90daa00640 1 -- 172.21.15.71:0/3896208762 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7f90e419fe20 con 0x7f90e4074eb0 2024-09-05T12:49:06.924 INFO:teuthology.orchestra.run.smithi071.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-05T12:49:06.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f90c80a2cf0 msgr2=0x7f90c80a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:06.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f90c80a2cf0 0x7f90c80a51b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f90e419fa30 tx=0x7f90d4005f90 comp rx=0 tx=0).stop 2024-09-05T12:49:06.930 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 msgr2=0x7f90e41a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 0x7f90e41a4c70 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f90d000e9f0 tx=0x7f90d000eeb0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 shutdown_connections 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e4112b90 0x7f90e419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f90c80a2cf0 0x7f90c80a51b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90e4074eb0 0x7f90e41a4c70 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 --2- 172.21.15.71:0/3896208762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e4074590 0x7f90e41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 >> 172.21.15.71:0/3896208762 conn(0x7f90e40fd8c0 msgr2=0x7f90e40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:06.931 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 shutdown_connections 2024-09-05T12:49:06.932 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:06.928+0000 7f90e8c00640 1 -- 172.21.15.71:0/3896208762 wait complete. 2024-09-05T12:49:06.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-mon[34321]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 173 KiB/s wr, 29 op/s 2024-09-05T12:49:06.967 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-mon[34313]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 173 KiB/s wr, 29 op/s 2024-09-05T12:49:07.260 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l[63481]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.261 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l[63481]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.261 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l[63481]: starting mds.l at 2024-09-05T12:49:07.261 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c[64095]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.261 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c[64095]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.261 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-mds[64099]: -- 172.21.15.174:0/3930637895 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594276df860 con 0x5594276bd400 2024-09-05T12:49:07.262 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c[64095]: starting mds.c at 2024-09-05T12:49:07.262 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i[64397]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.262 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i[64397]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.262 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i[64397]: starting mds.i at 2024-09-05T12:49:07.262 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.263 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.263 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-mds[63797]: -- 172.21.15.174:0/387782054 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c7dec9d860 con 0x55c7dec7a800 2024-09-05T12:49:07.263 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:49:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: starting mds.f at 2024-09-05T12:49:07.292 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:06 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a[29841]: 2024-09-05T12:49:06.921+0000 7f7321a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:49:07.292 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a[70877]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.292 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a[70877]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.292 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mds[70884]: -- 172.21.15.71:0/4080279484 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e868fd1860 con 0x55e868faf000 2024-09-05T12:49:07.293 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a[70877]: starting mds.a at 2024-09-05T12:49:07.293 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g[70532]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.293 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g[70532]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.293 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mds[70537]: -- 172.21.15.71:0/3288198318 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565558e11c20 con 0x565558dee800 2024-09-05T12:49:07.293 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g[70532]: starting mds.g at 2024-09-05T12:49:07.430 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e[64210]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.430 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e[64210]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.430 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-mds[64214]: -- 172.21.15.123:0/3765614952 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56387190f860 con 0x5638718ec800 2024-09-05T12:49:07.430 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e[64210]: starting mds.e at 2024-09-05T12:49:07.431 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k[63898]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.431 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k[63898]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.431 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-mds[63902]: -- 172.21.15.123:0/2407569874 <== mon.0 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b8864bc20 con 0x557b88629400 2024-09-05T12:49:07.431 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k[63898]: starting mds.k at 2024-09-05T12:49:07.431 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-mds[64822]: -- 172.21.15.123:0/1441184589 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55995fe2b860 con 0x55995fe09000 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: starting mds.b at 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h[64519]: ignoring --setuser ceph since I am not root 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h[64519]: ignoring --setgroup ceph since I am not root 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-mds[64523]: -- 172.21.15.123:0/2345661294 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5637b8029c20 con 0x5637b8007000 2024-09-05T12:49:07.432 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:49:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h[64519]: starting mds.h at 2024-09-05T12:49:07.747 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-05T12:49:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3896208762' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:49:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mon[29845]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T12:49:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mon[29845]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:49:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3896208762' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T12:49:07.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mon[29845]: osdmap e129: 12 total, 12 up, 12 in 2024-09-05T12:49:07.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:07 smithi071 ceph-mon[29845]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T12:49:07.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3896208762' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:49:07.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:07 smithi123 ceph-mon[34321]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T12:49:07.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:07 smithi123 ceph-mon[34321]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:49:07.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:07 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3896208762' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T12:49:07.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:07 smithi123 ceph-mon[34321]: osdmap e129: 12 total, 12 up, 12 in 2024-09-05T12:49:07.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:07 smithi123 ceph-mon[34321]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T12:49:08.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3896208762' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-05T12:49:08.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:07 smithi174 ceph-mon[34313]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-05T12:49:08.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:07 smithi174 ceph-mon[34313]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-05T12:49:08.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:07 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3896208762' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-05T12:49:08.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:07 smithi174 ceph-mon[34313]: osdmap e129: 12 total, 12 up, 12 in 2024-09-05T12:49:08.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:07 smithi174 ceph-mon[34313]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-05T12:49:08.126 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:08.749 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.747+0000 7fe3b9400640 1 -- 172.21.15.71:0/1235864717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 msgr2=0x7fe3b4114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:08.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.747+0000 7fe3b9400640 1 --2- 172.21.15.71:0/1235864717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b4114f80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009ca0 tx=0x7fe3ac02a950 comp rx=0 tx=0).stop 2024-09-05T12:49:08.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 -- 172.21.15.71:0/1235864717 shutdown_connections 2024-09-05T12:49:08.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 --2- 172.21.15.71:0/1235864717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b4114f80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:08.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 --2- 172.21.15.71:0/1235864717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074eb0 0x7fe3b4112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:08.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 --2- 172.21.15.71:0/1235864717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b4074590 0x7fe3b4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:08.750 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 -- 172.21.15.71:0/1235864717 >> 172.21.15.71:0/1235864717 conn(0x7fe3b40fd8c0 msgr2=0x7fe3b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:08.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 -- 172.21.15.71:0/1235864717 shutdown_connections 2024-09-05T12:49:08.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.748+0000 7fe3b9400640 1 -- 172.21.15.71:0/1235864717 wait complete. 2024-09-05T12:49:08.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 Processor -- start 2024-09-05T12:49:08.751 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 -- start start 2024-09-05T12:49:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 0x7fe3b419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b4074eb0 0x7fe3b419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b41a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b41185a0 con 0x7fe3b4112b90 2024-09-05T12:49:08.753 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4118420 con 0x7fe3b4074590 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b9400640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3b4118720 con 0x7fe3b4074eb0 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 0x7fe3b419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 0x7fe3b419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42588/0 (socket says 172.21.15.71:42588) 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b2800640 1 -- 172.21.15.71:0/310583322 learned_addr learned my addr 172.21.15.71:0/310583322 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.749+0000 7fe3b3200640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b41a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3abe00640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b4074eb0 0x7fe3b419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b2800640 1 -- 172.21.15.71:0/310583322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b4074eb0 msgr2=0x7fe3b419f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:08.754 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b2800640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b4074eb0 0x7fe3b419f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b2800640 1 -- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 msgr2=0x7fe3b41a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b2800640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b41a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b2800640 1 -- 172.21.15.71:0/310583322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3b41a3b20 con 0x7fe3b4074590 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b3200640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b41a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.750+0000 7fe3b2800640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 0x7fe3b419eaf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400e9f0 tx=0x7fe3a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.751+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a400cd80 con 0x7fe3b4074590 2024-09-05T12:49:08.755 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.751+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3b41a3e10 con 0x7fe3b4074590 2024-09-05T12:49:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.751+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3a4010910 con 0x7fe3b4074590 2024-09-05T12:49:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.751+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3b41ab6a0 con 0x7fe3b4074590 2024-09-05T12:49:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.751+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4018980 con 0x7fe3b4074590 2024-09-05T12:49:08.756 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.752+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3b4074970 con 0x7fe3b4074590 2024-09-05T12:49:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.754+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe3a4010ac0 con 0x7fe3b4074590 2024-09-05T12:49:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.755+0000 7fe3ab400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3980a2d70 0x7fe3980a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.755+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9780+0+0 (secure 0 0 0) 0x7fe3a40cfcf0 con 0x7fe3b4074590 2024-09-05T12:49:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.755+0000 7fe3abe00640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3980a2d70 0x7fe3980a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:08.757 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.756+0000 7fe3abe00640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3980a2d70 0x7fe3980a5230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40fecf0 tx=0x7fe39c005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:08.760 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.759+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3a409bd70 con 0x7fe3b4074590 2024-09-05T12:49:08.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: pgmap v678: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 37 op/s 2024-09-05T12:49:08.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6832/980989484,v1:172.21.15.174:6834/980989484] up:boot 2024-09-05T12:49:08.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6833/2359046129,v1:172.21.15.174:6835/2359046129] up:boot 2024-09-05T12:49:08.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6836/2689455324,v1:172.21.15.174:6837/2689455324] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6833/1409255606,v1:172.21.15.123:6835/1409255606] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6832/3011208004,v1:172.21.15.123:6834/3011208004] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.174:6838/82977924,v1:172.21.15.174:6839/82977924] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6836/2796191033,v1:172.21.15.123:6838/2796191033] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.123:6837/2015187449,v1:172.21.15.123:6839/2015187449] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6839/3154038760,v1:172.21.15.71:6841/3154038760] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: mds.? [v2:172.21.15.71:6838/1808063940,v1:172.21.15.71:6840/1808063940] up:boot 2024-09-05T12:49:08.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:49:08.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:49:08.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:49:08.914 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:08 smithi071 ceph-mon[29845]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: pgmap v678: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 37 op/s 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6832/980989484,v1:172.21.15.174:6834/980989484] up:boot 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6833/2359046129,v1:172.21.15.174:6835/2359046129] up:boot 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6836/2689455324,v1:172.21.15.174:6837/2689455324] up:boot 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6833/1409255606,v1:172.21.15.123:6835/1409255606] up:boot 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6832/3011208004,v1:172.21.15.123:6834/3011208004] up:boot 2024-09-05T12:49:08.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.174:6838/82977924,v1:172.21.15.174:6839/82977924] up:boot 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6836/2796191033,v1:172.21.15.123:6838/2796191033] up:boot 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.123:6837/2015187449,v1:172.21.15.123:6839/2015187449] up:boot 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6839/3154038760,v1:172.21.15.71:6841/3154038760] up:boot 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: mds.? [v2:172.21.15.71:6838/1808063940,v1:172.21.15.71:6840/1808063940] up:boot 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:49:08.931 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:49:08.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:49:08.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:49:08.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:49:08.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:49:08.932 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:08 smithi123 ceph-mon[34321]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:49:08.970 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:08.969+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe3b4105030 con 0x7fe3b4074590 2024-09-05T12:49:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: pgmap v678: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 37 op/s 2024-09-05T12:49:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6832/980989484,v1:172.21.15.174:6834/980989484] up:boot 2024-09-05T12:49:09.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6833/2359046129,v1:172.21.15.174:6835/2359046129] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6836/2689455324,v1:172.21.15.174:6837/2689455324] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6833/1409255606,v1:172.21.15.123:6835/1409255606] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6832/3011208004,v1:172.21.15.123:6834/3011208004] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.174:6838/82977924,v1:172.21.15.174:6839/82977924] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6836/2796191033,v1:172.21.15.123:6838/2796191033] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.123:6837/2015187449,v1:172.21.15.123:6839/2015187449] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6839/3154038760,v1:172.21.15.71:6841/3154038760] up:boot 2024-09-05T12:49:09.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: mds.? [v2:172.21.15.71:6838/1808063940,v1:172.21.15.71:6840/1808063940] up:boot 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-05T12:49:09.012 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-05T12:49:09.013 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-05T12:49:09.013 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:08 smithi174 ceph-mon[34313]: from='mgr.15165 172.21.15.71:0/4143423332' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-05T12:49:09.590 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.589+0000 7fe3ab400640 1 -- 172.21.15.71:0/310583322 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) ==== 104+0+0 (secure 0 0 0) 0x7fe3a40a0070 con 0x7fe3b4074590 2024-09-05T12:49:09.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3980a2d70 msgr2=0x7fe3980a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:09.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3980a2d70 0x7fe3980a5230 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40fecf0 tx=0x7fe39c005f90 comp rx=0 tx=0).stop 2024-09-05T12:49:09.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 msgr2=0x7fe3b419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:09.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 0x7fe3b419eaf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe3a400e9f0 tx=0x7fe3a400eeb0 comp rx=0 tx=0).stop 2024-09-05T12:49:09.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 shutdown_connections 2024-09-05T12:49:09.593 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe3980a2d70 0x7fe3980a5230 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:09.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3b4112b90 0x7fe3b41a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:09.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe3b4074eb0 0x7fe3b419f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:09.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 --2- 172.21.15.71:0/310583322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3b4074590 0x7fe3b419eaf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:09.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 >> 172.21.15.71:0/310583322 conn(0x7fe3b40fd8c0 msgr2=0x7fe3b40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:09.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 shutdown_connections 2024-09-05T12:49:09.594 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:09.592+0000 7fe3b9400640 1 -- 172.21.15.71:0/310583322 wait complete. 2024-09-05T12:49:09.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:09 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/310583322' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:49:09.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:09 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:49:09.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:09 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/310583322' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:49:09.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:09 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:49:10.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:09 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/310583322' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:49:10.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:09 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-05T12:49:10.500 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-05T12:49:10.871 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:10.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:10 smithi071 ceph-mon[29845]: pgmap v679: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 37 op/s 2024-09-05T12:49:10.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:10 smithi071 ceph-mon[29845]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T12:49:10.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:10 smithi071 ceph-mon[29845]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T12:49:10.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:10 smithi071 ceph-mon[29845]: Cluster is now healthy 2024-09-05T12:49:10.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:10 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:49:10.912 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:10 smithi071 ceph-mon[29845]: fsmap 12 up:standby 2024-09-05T12:49:10.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:10 smithi123 ceph-mon[34321]: pgmap v679: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 37 op/s 2024-09-05T12:49:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:10 smithi123 ceph-mon[34321]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T12:49:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:10 smithi123 ceph-mon[34321]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T12:49:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:10 smithi123 ceph-mon[34321]: Cluster is now healthy 2024-09-05T12:49:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:10 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:49:10.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:10 smithi123 ceph-mon[34321]: fsmap 12 up:standby 2024-09-05T12:49:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:10 smithi174 ceph-mon[34313]: pgmap v679: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 226 KiB/s wr, 37 op/s 2024-09-05T12:49:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:10 smithi174 ceph-mon[34313]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-05T12:49:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:10 smithi174 ceph-mon[34313]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-05T12:49:11.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:10 smithi174 ceph-mon[34313]: Cluster is now healthy 2024-09-05T12:49:11.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:10 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-05T12:49:11.011 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:10 smithi174 ceph-mon[34313]: fsmap 12 up:standby 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 -- 172.21.15.71:0/4119470896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c103e30 msgr2=0x7fe30c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 --2- 172.21.15.71:0/4119470896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c103e30 0x7fe30c104210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe300009900 tx=0x7fe30002a410 comp rx=0 tx=0).stop 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 -- 172.21.15.71:0/4119470896 shutdown_connections 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 --2- 172.21.15.71:0/4119470896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c108f60 0x7fe30c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 --2- 172.21.15.71:0/4119470896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 --2- 172.21.15.71:0/4119470896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c103e30 0x7fe30c104210 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:11.567 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 -- 172.21.15.71:0/4119470896 >> 172.21.15.71:0/4119470896 conn(0x7fe30c0ff910 msgr2=0x7fe30c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 -- 172.21.15.71:0/4119470896 shutdown_connections 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.512+0000 7fe311e00640 1 -- 172.21.15.71:0/4119470896 wait complete. 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.513+0000 7fe311e00640 1 Processor -- start 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.513+0000 7fe311e00640 1 -- start start 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c103e30 0x7fe30c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c108f60 0x7fe30c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:11.568 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c113fd0 con 0x7fe30c1047e0 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c113e50 con 0x7fe30c103e30 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311e00640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c114150 con 0x7fe30c108f60 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe30be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe310a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c103e30 0x7fe30c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe30be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:41862/0 (socket says 172.21.15.71:41862) 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe310a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c103e30 0x7fe30c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.71:42594/0 (socket says 172.21.15.71:42594) 2024-09-05T12:49:11.569 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe30be00640 1 -- 172.21.15.71:0/1337283016 learned_addr learned my addr 172.21.15.71:0/1337283016 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.514+0000 7fe311400640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c108f60 0x7fe30c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe30be00640 1 -- 172.21.15.71:0/1337283016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c108f60 msgr2=0x7fe30c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe30be00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c108f60 0x7fe30c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe30be00640 1 -- 172.21.15.71:0/1337283016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c103e30 msgr2=0x7fe30c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe30be00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c103e30 0x7fe30c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe30be00640 1 -- 172.21.15.71:0/1337283016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c19f480 con 0x7fe30c1047e0 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe30be00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c19aa10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800e9f0 tx=0x7fe2f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f800cd80 con 0x7fe30c1047e0 2024-09-05T12:49:11.570 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c19f770 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2f8010910 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c1a7050 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.515+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8018910 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.517+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c105310 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.518+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7fe2f8010ac0 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.520+0000 7fe309600640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe2f40a2cf0 0x7fe2f40a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.520+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9780+0+0 (secure 0 0 0) 0x7fe2f80cfd20 con 0x7fe30c1047e0 2024-09-05T12:49:11.571 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.520+0000 7fe310a00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe2f40a2cf0 0x7fe2f40a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:11.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.521+0000 7fe310a00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe2f40a2cf0 0x7fe2f40a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe300005b50 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:11.572 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.524+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe2f809bda0 con 0x7fe30c1047e0 2024-09-05T12:49:11.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:11.699+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe30c104210 con 0x7fe30c1047e0 2024-09-05T12:49:12.599 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.597+0000 7fe309600640 1 -- 172.21.15.71:0/1337283016 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) ==== 181+0+0 (secure 0 0 0) 0x7fe30c104210 con 0x7fe30c1047e0 2024-09-05T12:49:12.599 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_metadata' removed 2024-09-05T12:49:12.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.602+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe2f40a2cf0 msgr2=0x7fe2f40a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:12.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.602+0000 7fe311e00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe2f40a2cf0 0x7fe2f40a51b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe300005b50 comp rx=0 tx=0).stop 2024-09-05T12:49:12.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.602+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 msgr2=0x7fe30c19aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:12.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.602+0000 7fe311e00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c19aa10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800e9f0 tx=0x7fe2f800eeb0 comp rx=0 tx=0).stop 2024-09-05T12:49:12.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 shutdown_connections 2024-09-05T12:49:12.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe30c108f60 0x7fe30c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:12.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7fe2f40a2cf0 0x7fe2f40a51b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:12.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe30c1047e0 0x7fe30c19aa10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:12.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 --2- 172.21.15.71:0/1337283016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c103e30 0x7fe30c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:12.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 >> 172.21.15.71:0/1337283016 conn(0x7fe30c0ff910 msgr2=0x7fe30c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:12.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.604+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 shutdown_connections 2024-09-05T12:49:12.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:12.605+0000 7fe311e00640 1 -- 172.21.15.71:0/1337283016 wait complete. 2024-09-05T12:49:12.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:12 smithi071 ceph-mon[29845]: pgmap v680: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 100 KiB/s wr, 30 op/s 2024-09-05T12:49:12.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:12 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1337283016' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:12.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:12 smithi123 ceph-mon[34321]: pgmap v680: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 100 KiB/s wr, 30 op/s 2024-09-05T12:49:12.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:12 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1337283016' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:13.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:12 smithi174 ceph-mon[34313]: pgmap v680: 129 pgs: 129 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 100 KiB/s wr, 30 op/s 2024-09-05T12:49:13.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:12 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1337283016' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:13.561 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-05T12:49:13.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:13 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1337283016' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:13.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:13 smithi071 ceph-mon[29845]: osdmap e130: 12 total, 12 up, 12 in 2024-09-05T12:49:13.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:13 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1337283016' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:13.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:13 smithi123 ceph-mon[34321]: osdmap e130: 12 total, 12 up, 12 in 2024-09-05T12:49:13.941 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:14.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:13 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1337283016' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:14.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:13 smithi174 ceph-mon[34313]: osdmap e130: 12 total, 12 up, 12 in 2024-09-05T12:49:14.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 -- 172.21.15.71:0/1700849541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a4105d90 msgr2=0x7f04a4106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:14.528 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 --2- 172.21.15.71:0/1700849541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a4105d90 0x7f04a4106220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f049c009ca0 tx=0x7f049c02a940 comp rx=0 tx=0).stop 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 -- 172.21.15.71:0/1700849541 shutdown_connections 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 --2- 172.21.15.71:0/1700849541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a4105d90 0x7f04a4106220 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 --2- 172.21.15.71:0/1700849541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a41012a0 0x7f04a4105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 --2- 172.21.15.71:0/1700849541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4100980 0x7f04a4100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.524+0000 7f04a3200640 1 -- 172.21.15.71:0/1700849541 >> 172.21.15.71:0/1700849541 conn(0x7f04a40fc610 msgr2=0x7f04a40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.525+0000 7f04a3200640 1 -- 172.21.15.71:0/1700849541 shutdown_connections 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.525+0000 7f04a3200640 1 -- 172.21.15.71:0/1700849541 wait complete. 2024-09-05T12:49:14.529 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 Processor -- start 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 -- start start 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a4100980 0x7f04a41a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a41012a0 0x7f04a41a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4105d90 0x7f04a419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f04a4118420 con 0x7f04a41012a0 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04a41182a0 con 0x7f04a4105d90 2024-09-05T12:49:14.530 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.526+0000 7f04a3200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04a41185a0 con 0x7f04a4100980 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f049be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a41012a0 0x7f04a41a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a4100980 0x7f04a41a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f049be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a41012a0 0x7f04a41a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49370/0 (socket says 172.21.15.71:49370) 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a2800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4105d90 0x7f04a419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f049be00640 1 -- 172.21.15.71:0/3553125020 learned_addr learned my addr 172.21.15.71:0/3553125020 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a1e00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a4100980 0x7f04a41a45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.71:44396/0 (socket says 172.21.15.71:44396) 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a2800640 1 -- 172.21.15.71:0/3553125020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a4100980 msgr2=0x7f04a41a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:14.531 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a2800640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a4100980 0x7f04a41a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a2800640 1 -- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a41012a0 msgr2=0x7f04a41a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a2800640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a41012a0 0x7f04a41a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:14.532 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.527+0000 7f04a2800640 1 -- 172.21.15.71:0/3553125020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a419ef20 con 0x7f04a4105d90 2024-09-05T12:49:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.528+0000 7f04a2800640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4105d90 0x7f04a419e6b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f049c005e30 tx=0x7f049c02f630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.528+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c034040 con 0x7f04a4105d90 2024-09-05T12:49:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.528+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a419f1b0 con 0x7f04a4105d90 2024-09-05T12:49:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.528+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f049c02fe10 con 0x7f04a4105d90 2024-09-05T12:49:14.533 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.528+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c0045e0 con 0x7f04a4105d90 2024-09-05T12:49:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.528+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a41ab390 con 0x7f04a4105d90 2024-09-05T12:49:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.529+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04a406a850 con 0x7f04a4105d90 2024-09-05T12:49:14.534 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.531+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f049c037020 con 0x7f04a4105d90 2024-09-05T12:49:14.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.532+0000 7f049aa00640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04880a2d70 0x7f04880a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:14.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.533+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 9284+0+0 (secure 0 0 0) 0x7f049c0f0600 con 0x7f04a4105d90 2024-09-05T12:49:14.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.533+0000 7f04a1e00640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04880a2d70 0x7f04880a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:14.535 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.533+0000 7f04a1e00640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04880a2d70 0x7f04880a5230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0494009bd0 tx=0x7f0494005f90 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:14.536 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.535+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f049c0bcfd0 con 0x7f04a4105d90 2024-09-05T12:49:14.726 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:14.725+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f04a4105770 con 0x7f04a4105d90 2024-09-05T12:49:14.726 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:14 smithi071 ceph-mon[29845]: pgmap v682: 97 pgs: 97 active+clean; 333 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:14.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:14 smithi123 ceph-mon[34321]: pgmap v682: 97 pgs: 97 active+clean; 333 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:15.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:14 smithi174 ceph-mon[34313]: pgmap v682: 97 pgs: 97 active+clean; 333 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:15.615 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.614+0000 7f049aa00640 1 -- 172.21.15.71:0/3553125020 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) ==== 169+0+0 (secure 0 0 0) 0x7f04a4105770 con 0x7f04a4105d90 2024-09-05T12:49:15.616 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data' removed 2024-09-05T12:49:15.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.618+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04880a2d70 msgr2=0x7f04880a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.618+0000 7f04a3200640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04880a2d70 0x7f04880a5230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0494009bd0 tx=0x7f0494005f90 comp rx=0 tx=0).stop 2024-09-05T12:49:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.618+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4105d90 msgr2=0x7f04a419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:15.620 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.618+0000 7f04a3200640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4105d90 0x7f04a419e6b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f049c005e30 tx=0x7f049c02f630 comp rx=0 tx=0).stop 2024-09-05T12:49:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 shutdown_connections 2024-09-05T12:49:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04a4105d90 0x7f04a419e6b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f04880a2d70 0x7f04880a5230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04a41012a0 0x7f04a41a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 --2- 172.21.15.71:0/3553125020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04a4100980 0x7f04a41a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:15.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 >> 172.21.15.71:0/3553125020 conn(0x7f04a40fc610 msgr2=0x7f04a4103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:15.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 shutdown_connections 2024-09-05T12:49:15.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:15.619+0000 7f04a3200640 1 -- 172.21.15.71:0/3553125020 wait complete. 2024-09-05T12:49:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:15 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/3553125020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:15.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:15 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:15.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:15 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/3553125020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:15.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:15 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:15 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/3553125020' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:16.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:15 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:16.471 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac shell --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-05T12:49:16.850 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/mon.a/config 2024-09-05T12:49:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:16 smithi071 ceph-mon[29845]: pgmap v683: 97 pgs: 97 active+clean; 333 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:16 smithi071 ceph-mon[29845]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:16.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:16 smithi071 ceph-mon[29845]: osdmap e131: 12 total, 12 up, 12 in 2024-09-05T12:49:16.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:16 smithi123 ceph-mon[34321]: pgmap v683: 97 pgs: 97 active+clean; 333 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:16.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:16 smithi123 ceph-mon[34321]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:16.930 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:16 smithi123 ceph-mon[34321]: osdmap e131: 12 total, 12 up, 12 in 2024-09-05T12:49:17.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:16 smithi174 ceph-mon[34313]: pgmap v683: 97 pgs: 97 active+clean; 333 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:17.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:16 smithi174 ceph-mon[34313]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:17.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:16 smithi174 ceph-mon[34313]: osdmap e131: 12 total, 12 up, 12 in 2024-09-05T12:49:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 -- 172.21.15.71:0/3718334660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118104800 msgr2=0x7f7118108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 --2- 172.21.15.71:0/3718334660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118104800 0x7f7118108960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7104009900 tx=0x7f710402a3f0 comp rx=0 tx=0).stop 2024-09-05T12:49:17.646 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 -- 172.21.15.71:0/3718334660 shutdown_connections 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 --2- 172.21.15.71:0/3718334660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7118109090 0x7f7118110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 --2- 172.21.15.71:0/3718334660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118104800 0x7f7118108960 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 --2- 172.21.15.71:0/3718334660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118103e50 0x7f7118104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 -- 172.21.15.71:0/3718334660 >> 172.21.15.71:0/3718334660 conn(0x7f71180ff8b0 msgr2=0x7f7118101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 -- 172.21.15.71:0/3718334660 shutdown_connections 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.601+0000 7f711e200640 1 -- 172.21.15.71:0/3718334660 wait complete. 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 Processor -- start 2024-09-05T12:49:17.647 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 -- start start 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7118103e50 0x7f711819a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118104800 0x7f711819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 0x7f711819ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114090 con 0x7f7118109090 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7118113f10 con 0x7f7118103e50 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.602+0000 7f711e200640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114210 con 0x7f7118104800 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 0x7f711819ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:17.648 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7118103e50 0x7f711819a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 0x7f711819ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:49378/0 (socket says 172.21.15.71:49378) 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 -- 172.21.15.71:0/1521651464 learned_addr learned my addr 172.21.15.71:0/1521651464 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f7117e00640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118104800 0x7f711819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 -- 172.21.15.71:0/1521651464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118104800 msgr2=0x7f711819aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118104800 0x7f711819aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 -- 172.21.15.71:0/1521651464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7118103e50 msgr2=0x7f711819a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7118103e50 0x7f711819a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:17.649 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 -- 172.21.15.71:0/1521651464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711819f520 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.603+0000 7f711d800640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 0x7f711819ee40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f711000eb10 tx=0x7f711000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.604+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004b20 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.604+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7110004cc0 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.604+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711819f810 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.604+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010760 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.604+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a70f0 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.605+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7118105330 con 0x7f7118109090 2024-09-05T12:49:17.650 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.607+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 137964+0+0 (secure 0 0 0) 0x7f7110005020 con 0x7f7118109090 2024-09-05T12:49:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.608+0000 7f7115600640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71000a2cf0 0x7f71000a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-05T12:49:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.608+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 8754+0+0 (secure 0 0 0) 0x7f71100cfb40 con 0x7f7118109090 2024-09-05T12:49:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.608+0000 7f711ce00640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71000a2cf0 0x7f71000a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-05T12:49:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.609+0000 7f711ce00640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71000a2cf0 0x7f71000a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f710c00bbe0 tx=0x7f710c00b390 comp rx=0 tx=0).ready entity=mgr.15165 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-05T12:49:17.651 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.612+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f711009c610 con 0x7f7118109090 2024-09-05T12:49:17.791 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:17.789+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f711810c2d0 con 0x7f7118109090 2024-09-05T12:49:18.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.625+0000 7f7115600640 1 -- 172.21.15.71:0/1521651464 <== mon.0 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v132) ==== 178+0+0 (secure 0 0 0) 0x7f711009bf10 con 0x7f7118109090 2024-09-05T12:49:18.627 INFO:teuthology.orchestra.run.smithi071.stderr:pool 'cephfs_data_ec' removed 2024-09-05T12:49:18.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.628+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71000a2cf0 msgr2=0x7f71000a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:18.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.628+0000 7f711e200640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71000a2cf0 0x7f71000a51b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f710c00bbe0 tx=0x7f710c00b390 comp rx=0 tx=0).stop 2024-09-05T12:49:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.628+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 msgr2=0x7f711819ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-05T12:49:18.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.628+0000 7f711e200640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 0x7f711819ee40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f711000eb10 tx=0x7f711000c2d0 comp rx=0 tx=0).stop 2024-09-05T12:49:18.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.630+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 shutdown_connections 2024-09-05T12:49:18.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.631+0000 7f711e200640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:6800/1162352135,v1:172.21.15.71:6801/1162352135] conn(0x7f71000a2cf0 0x7f71000a51b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:18.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.631+0000 7f711e200640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7118109090 0x7f711819ee40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:18.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.631+0000 7f711e200640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7118104800 0x7f711819aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:18.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.631+0000 7f711e200640 1 --2- 172.21.15.71:0/1521651464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7118103e50 0x7f711819a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-05T12:49:18.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.631+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 >> 172.21.15.71:0/1521651464 conn(0x7f71180ff8b0 msgr2=0x7f7118100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-05T12:49:18.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.632+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 shutdown_connections 2024-09-05T12:49:18.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-09-05T12:49:18.632+0000 7f711e200640 1 -- 172.21.15.71:0/1521651464 wait complete. 2024-09-05T12:49:18.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:18 smithi071 ceph-mon[29845]: pgmap v685: 33 pgs: 33 active+clean; 333 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:18.913 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:18 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1521651464' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:18 smithi123 ceph-mon[34321]: pgmap v685: 33 pgs: 33 active+clean; 333 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:18.929 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:18 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1521651464' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:18 smithi174 ceph-mon[34313]: pgmap v685: 33 pgs: 33 active+clean; 333 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-05T12:49:19.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:18 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1521651464' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-05T12:49:19.597 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:49:19.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:49:19.711 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:49:19.739 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-05T12:49:19.740 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:49:19.775 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:49:19.804 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-05T12:49:19.863 INFO:tasks.cephadm:Stopping all daemons... 2024-09-05T12:49:19.863 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-05T12:49:19.863 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a 2024-09-05T12:49:19.910 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:19 smithi071 ceph-mon[29845]: from='client.? 172.21.15.71:0/1521651464' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:19.911 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:19 smithi071 ceph-mon[29845]: osdmap e132: 12 total, 12 up, 12 in 2024-09-05T12:49:19.933 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:19 smithi123 ceph-mon[34321]: from='client.? 172.21.15.71:0/1521651464' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:19.933 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:19 smithi123 ceph-mon[34321]: osdmap e132: 12 total, 12 up, 12 in 2024-09-05T12:49:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:19 smithi174 ceph-mon[34313]: from='client.? 172.21.15.71:0/1521651464' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-05T12:49:20.010 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:19 smithi174 ceph-mon[34313]: osdmap e132: 12 total, 12 up, 12 in 2024-09-05T12:49:20.286 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:19 smithi071 systemd[1]: Stopping Ceph mon.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a[29841]: 2024-09-05T12:49:20.284+0000 7f7328800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a[29841]: 2024-09-05T12:49:20.284+0000 7f7328800640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-05T12:49:20.661 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:20 smithi071 podman[106880]: 2024-09-05 12:49:20.336146919 +0000 UTC m=+0.193251396 container died e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:49:21.093 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:20 smithi071 podman[106880]: 2024-09-05 12:49:20.840821193 +0000 UTC m=+0.697925668 container cleanup e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T12:49:21.093 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:20 smithi071 bash[106880]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a 2024-09-05T12:49:21.354 INFO:journalctl@ceph.mon.a.smithi071.stdout:Sep 05 12:49:21 smithi071 podman[106892]: 2024-09-05 12:49:21.092723284 +0000 UTC m=+0.744248115 container remove e2795b5003206bafb378167fe4482568e0df2e0c3c9880076a94d0d344d98b33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-05T12:49:21.375 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.a.service' 2024-09-05T12:49:21.460 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:21.461 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-05T12:49:21.461 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-05T12:49:21.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.b 2024-09-05T12:49:21.791 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:21 smithi123 systemd[1]: Stopping Ceph mon.b for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:22.135 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-b[34317]: 2024-09-05T12:49:21.790+0000 7fc9901a5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:22.135 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-b[34317]: 2024-09-05T12:49:21.790+0000 7fc9901a5640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-05T12:49:22.135 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:21 smithi123 podman[73344]: 2024-09-05 12:49:21.811211479 +0000 UTC m=+0.133780783 container died 1dcd4e6c34fc11d7c08497c125fa1f9e52a23c8f0f89841be25347ebdfd27f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2) 2024-09-05T12:49:22.429 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:22 smithi123 podman[73344]: 2024-09-05 12:49:22.232938009 +0000 UTC m=+0.555507357 container cleanup 1dcd4e6c34fc11d7c08497c125fa1f9e52a23c8f0f89841be25347ebdfd27f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-05T12:49:22.430 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:22 smithi123 bash[73344]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-b 2024-09-05T12:49:22.808 INFO:journalctl@ceph.mon.b.smithi123.stdout:Sep 05 12:49:22 smithi123 podman[73357]: 2024-09-05 12:49:22.551371069 +0000 UTC m=+0.733022069 container remove 1dcd4e6c34fc11d7c08497c125fa1f9e52a23c8f0f89841be25347ebdfd27f7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:49:22.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.b.service' 2024-09-05T12:49:22.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:22.867 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-05T12:49:22.867 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-05T12:49:22.868 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.c 2024-09-05T12:49:23.260 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:22 smithi174 systemd[1]: Stopping Ceph mon.c for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:23.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c[34309]: 2024-09-05T12:49:23.281+0000 7f8e8def2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:23.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c[34309]: 2024-09-05T12:49:23.281+0000 7f8e8def2640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-05T12:49:23.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:23 smithi174 podman[73355]: 2024-09-05 12:49:23.495789716 +0000 UTC m=+0.320761598 container died 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T12:49:24.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:23 smithi174 podman[73355]: 2024-09-05 12:49:23.907996397 +0000 UTC m=+0.732968288 container cleanup 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T12:49:24.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:23 smithi174 bash[73355]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c 2024-09-05T12:49:24.438 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.c.service' 2024-09-05T12:49:24.460 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:24 smithi174 podman[73368]: 2024-09-05 12:49:24.192964449 +0000 UTC m=+0.701383255 container remove 1b2c606f8355b28ebc308898c39daee25b67791dcfc779dbb335d957f197d397 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:49:24.460 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:24 smithi174 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-05T12:49:24.460 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:24 smithi174 systemd[1]: Stopped Ceph mon.c for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:49:24.460 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 05 12:49:24 smithi174 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mon.c.service: Consumed 18.616s CPU time. 2024-09-05T12:49:24.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:24.540 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-05T12:49:24.540 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-05T12:49:24.540 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x 2024-09-05T12:49:24.911 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:24 smithi071 systemd[1]: Stopping Ceph mgr.x for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:25.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:25 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:49:25.056+0000 7f0b32800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:25.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:25 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x[30082]: 2024-09-05T12:49:25.056+0000 7f0b32800640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-05T12:49:25.411 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:25 smithi071 podman[106999]: 2024-09-05 12:49:25.140027198 +0000 UTC m=+0.278597969 container died 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True) 2024-09-05T12:49:26.077 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:25 smithi071 podman[106999]: 2024-09-05 12:49:25.823523638 +0000 UTC m=+0.962094390 container cleanup 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T12:49:26.077 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:25 smithi071 bash[106999]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x 2024-09-05T12:49:26.344 INFO:journalctl@ceph.mgr.x.smithi071.stdout:Sep 05 12:49:26 smithi071 podman[107012]: 2024-09-05 12:49:26.076678092 +0000 UTC m=+0.924321594 container remove 549122bed565ad8d28be5213ff351db3a09f32087d41f5aac7177f256fe47868 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e856d6b2462509a377ef39149a05475ed006ebac, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-09-05T12:49:26.365 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.x.service' 2024-09-05T12:49:26.439 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:26.440 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-05T12:49:26.440 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-05T12:49:26.441 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.y 2024-09-05T12:49:26.929 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:49:26 smithi123 systemd[1]: Stopping Ceph mgr.y for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:27.322 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:49:27 smithi123 podman[73467]: 2024-09-05 12:49:27.01670467 +0000 UTC m=+0.383430814 container died c9167a856f555ca8fd87540539f411373a4bb9fd44308f22671aabd6507b2720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True) 2024-09-05T12:49:27.680 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:49:27 smithi123 podman[73467]: 2024-09-05 12:49:27.421355694 +0000 UTC m=+0.788081880 container cleanup c9167a856f555ca8fd87540539f411373a4bb9fd44308f22671aabd6507b2720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-05T12:49:27.680 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Sep 05 12:49:27 smithi123 bash[73467]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-y 2024-09-05T12:49:27.990 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.y.service' 2024-09-05T12:49:28.024 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:28.024 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-05T12:49:28.024 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-05T12:49:28.024 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.z 2024-09-05T12:49:28.361 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:49:28 smithi174 systemd[1]: Stopping Ceph mgr.z for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:28.699 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:49:28 smithi174 podman[73492]: 2024-09-05 12:49:28.360816428 +0000 UTC m=+0.140281029 container died 1841ff4db577a98d8633a19428c422f726a4a943e147f98f03bb7cf5220df738 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:49:29.010 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:49:28 smithi174 podman[73492]: 2024-09-05 12:49:28.780334902 +0000 UTC m=+0.559799500 container cleanup 1841ff4db577a98d8633a19428c422f726a4a943e147f98f03bb7cf5220df738 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-05T12:49:29.010 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 05 12:49:28 smithi174 bash[73492]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mgr-z 2024-09-05T12:49:29.169 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mgr.z.service' 2024-09-05T12:49:29.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:29.205 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-05T12:49:29.205 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-05T12:49:29.205 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.0 2024-09-05T12:49:29.633 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:29 smithi071 systemd[1]: Stopping Ceph osd.0 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:29.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0[44160]: 2024-09-05T12:49:29.631+0000 7f9af1600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:29.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0[44160]: 2024-09-05T12:49:29.631+0000 7f9af1600640 -1 osd.0 132 *** Got signal Terminated *** 2024-09-05T12:49:29.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0[44160]: 2024-09-05T12:49:29.631+0000 7f9af1600640 -1 osd.0 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:49:35.582 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:35 smithi071 podman[107120]: 2024-09-05 12:49:35.209393612 +0000 UTC m=+5.714018061 container died b22ff260ac7bfe31e8c1815147317f893d3bf76e23472edf6724cc046179ed8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:49:35.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:35 smithi071 podman[107120]: 2024-09-05 12:49:35.721138371 +0000 UTC m=+6.225762811 container cleanup b22ff260ac7bfe31e8c1815147317f893d3bf76e23472edf6724cc046179ed8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-05T12:49:35.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:35 smithi071 bash[107120]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0 2024-09-05T12:49:36.411 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:36 smithi071 podman[107138]: 2024-09-05 12:49:36.031685036 +0000 UTC m=+0.811684451 container remove b22ff260ac7bfe31e8c1815147317f893d3bf76e23472edf6724cc046179ed8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T12:49:36.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:36 smithi071 podman[107225]: 2024-09-05 12:49:36.534503458 +0000 UTC m=+0.026999807 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:49:36.911 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:36 smithi071 podman[107225]: 2024-09-05 12:49:36.699458292 +0000 UTC m=+0.191954639 container create 82726cca8810caa642a92977ecc18a00b4d48f5a6cc60b09f1435a909a0dc031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-05T12:49:37.279 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:37 smithi071 podman[107225]: 2024-09-05 12:49:37.012340391 +0000 UTC m=+0.504836740 container init 82726cca8810caa642a92977ecc18a00b4d48f5a6cc60b09f1435a909a0dc031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T12:49:37.280 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:37 smithi071 podman[107225]: 2024-09-05 12:49:37.019098042 +0000 UTC m=+0.511594391 container start 82726cca8810caa642a92977ecc18a00b4d48f5a6cc60b09f1435a909a0dc031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:49:37.280 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:37 smithi071 podman[107225]: 2024-09-05 12:49:37.127458849 +0000 UTC m=+0.619955209 container attach 82726cca8810caa642a92977ecc18a00b4d48f5a6cc60b09f1435a909a0dc031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-05T12:49:37.661 INFO:journalctl@ceph.osd.0.smithi071.stdout:Sep 05 12:49:37 smithi071 podman[107225]: 2024-09-05 12:49:37.279075057 +0000 UTC m=+0.771571411 container died 82726cca8810caa642a92977ecc18a00b4d48f5a6cc60b09f1435a909a0dc031 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux ) 2024-09-05T12:49:38.233 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.0.service' 2024-09-05T12:49:38.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:38.309 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-05T12:49:38.310 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-05T12:49:38.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.1 2024-09-05T12:49:38.669 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:38 smithi123 systemd[1]: Stopping Ceph osd.1 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:38.929 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1[38751]: 2024-09-05T12:49:38.667+0000 7f4233f09640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:38.930 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1[38751]: 2024-09-05T12:49:38.667+0000 7f4233f09640 -1 osd.1 132 *** Got signal Terminated *** 2024-09-05T12:49:38.930 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1[38751]: 2024-09-05T12:49:38.667+0000 7f4233f09640 -1 osd.1 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:49:44.461 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:44 smithi123 podman[73590]: 2024-09-05 12:49:44.193923407 +0000 UTC m=+5.683869657 container died 0e1ba30d5f2f812c705e3a2dcd73dcc1e99b3888f71dc1782da98bb37ced0a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-05T12:49:44.861 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:44 smithi123 podman[73590]: 2024-09-05 12:49:44.543058607 +0000 UTC m=+6.033004856 container cleanup 0e1ba30d5f2f812c705e3a2dcd73dcc1e99b3888f71dc1782da98bb37ced0a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:49:44.862 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:44 smithi123 bash[73590]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1 2024-09-05T12:49:45.168 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:44 smithi123 podman[73608]: 2024-09-05 12:49:44.861054571 +0000 UTC m=+0.661941410 container remove 0e1ba30d5f2f812c705e3a2dcd73dcc1e99b3888f71dc1782da98bb37ced0a98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:49:45.430 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:45 smithi123 podman[73695]: 2024-09-05 12:49:45.070055175 +0000 UTC m=+0.015837230 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:49:45.430 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:45 smithi123 podman[73695]: 2024-09-05 12:49:45.203891369 +0000 UTC m=+0.149673427 container create 0964f01377b28058f80c592aae378392069a6afac112296595062cec5907ac2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T12:49:45.697 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:45 smithi123 podman[73695]: 2024-09-05 12:49:45.465629918 +0000 UTC m=+0.411411972 container init 0964f01377b28058f80c592aae378392069a6afac112296595062cec5907ac2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux ) 2024-09-05T12:49:45.697 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:45 smithi123 podman[73695]: 2024-09-05 12:49:45.468870914 +0000 UTC m=+0.414652974 container start 0964f01377b28058f80c592aae378392069a6afac112296595062cec5907ac2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-05T12:49:45.697 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:45 smithi123 podman[73695]: 2024-09-05 12:49:45.556646145 +0000 UTC m=+0.502428196 container attach 0964f01377b28058f80c592aae378392069a6afac112296595062cec5907ac2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-05T12:49:45.993 INFO:journalctl@ceph.osd.1.smithi123.stdout:Sep 05 12:49:45 smithi123 podman[73695]: 2024-09-05 12:49:45.696678715 +0000 UTC m=+0.642460770 container died 0964f01377b28058f80c592aae378392069a6afac112296595062cec5907ac2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True) 2024-09-05T12:49:46.331 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.1.service' 2024-09-05T12:49:46.372 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:46.372 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-05T12:49:46.372 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-05T12:49:46.372 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.2 2024-09-05T12:49:46.685 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:46 smithi174 systemd[1]: Stopping Ceph osd.2 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:47.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2[39237]: 2024-09-05T12:49:46.684+0000 7fab1af49640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:47.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2[39237]: 2024-09-05T12:49:46.684+0000 7fab1af49640 -1 osd.2 132 *** Got signal Terminated *** 2024-09-05T12:49:47.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2[39237]: 2024-09-05T12:49:46.684+0000 7fab1af49640 -1 osd.2 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:49:52.477 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:52 smithi174 podman[73620]: 2024-09-05 12:49:52.196535691 +0000 UTC m=+5.636743496 container died 7908211a270c40ff80136ed236e6bf53f753e2fda3af7f684e2741856018437f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:49:52.760 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:52 smithi174 podman[73620]: 2024-09-05 12:49:52.575663629 +0000 UTC m=+6.015871419 container cleanup 7908211a270c40ff80136ed236e6bf53f753e2fda3af7f684e2741856018437f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-05T12:49:52.760 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:52 smithi174 bash[73620]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2 2024-09-05T12:49:53.192 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:52 smithi174 podman[73637]: 2024-09-05 12:49:52.885697809 +0000 UTC m=+0.691512795 container remove 7908211a270c40ff80136ed236e6bf53f753e2fda3af7f684e2741856018437f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T12:49:53.465 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:53 smithi174 podman[73724]: 2024-09-05 12:49:53.094619804 +0000 UTC m=+0.015741373 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:49:53.465 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:53 smithi174 podman[73724]: 2024-09-05 12:49:53.220095648 +0000 UTC m=+0.141217213 container create 8bd28505709354073485a3172302b84893e69b4d9e93454a87f88c934755c5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:49:53.716 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:53 smithi174 podman[73724]: 2024-09-05 12:49:53.464392985 +0000 UTC m=+0.385514552 container init 8bd28505709354073485a3172302b84893e69b4d9e93454a87f88c934755c5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:49:53.716 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:53 smithi174 podman[73724]: 2024-09-05 12:49:53.467991307 +0000 UTC m=+0.389112881 container start 8bd28505709354073485a3172302b84893e69b4d9e93454a87f88c934755c5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-05T12:49:53.716 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:53 smithi174 podman[73724]: 2024-09-05 12:49:53.572897093 +0000 UTC m=+0.494018676 container attach 8bd28505709354073485a3172302b84893e69b4d9e93454a87f88c934755c5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux ) 2024-09-05T12:49:54.010 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 05 12:49:53 smithi174 podman[73724]: 2024-09-05 12:49:53.715924167 +0000 UTC m=+0.637045737 container died 8bd28505709354073485a3172302b84893e69b4d9e93454a87f88c934755c5bc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-2-deactivate, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:49:54.352 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.2.service' 2024-09-05T12:49:54.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:49:54.393 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-05T12:49:54.394 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-05T12:49:54.394 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.3 2024-09-05T12:49:54.863 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:49:54 smithi071 systemd[1]: Stopping Ceph osd.3 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:49:55.161 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:49:54 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:49:54.861+0000 7fdd41e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:49:55.161 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:49:54 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:49:54.861+0000 7fdd41e00640 -1 osd.3 132 *** Got signal Terminated *** 2024-09-05T12:49:55.161 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:49:54 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:49:54.861+0000 7fdd41e00640 -1 osd.3 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:49:57.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:49:57 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:49:57.127+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:49:57.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:49:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:49:57.496+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:49:58.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:49:58 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:49:58.100+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:49:58.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:49:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:49:58.383+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:49:58.911 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:49:58 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:49:58.489+0000 7fdd3d800640 -1 osd.3 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.186531+0000 front 2024-09-05T12:49:33.186520+0000 (oldest deadline 2024-09-05T12:49:57.886668+0000) 2024-09-05T12:49:58.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:49:58 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:49:58.584+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:49:58.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:49:58 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:49:58.512+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:49:59.410 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:49:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:49:59.051+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:49:59.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:49:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:49:59.380+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:49:59.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:49:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:49:59.261+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:49:59.911 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:49:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3[50615]: 2024-09-05T12:49:59.453+0000 7fdd3d800640 -1 osd.3 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.186531+0000 front 2024-09-05T12:49:33.186520+0000 (oldest deadline 2024-09-05T12:49:57.886668+0000) 2024-09-05T12:49:59.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:49:59 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:49:59.541+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:49:59.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:49:59 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:49:59.508+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:00.406 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:00.015+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:00.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:00.347+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:00.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:00.215+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:00.661 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:00 smithi071 podman[107327]: 2024-09-05 12:50:00.405687343 +0000 UTC m=+5.722300349 container died d33033dee4448490011d6e1729e32c2817508c130ba1d6e456541de78a22f758 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731) 2024-09-05T12:50:00.661 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:00 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:00.495+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:00.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:00 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:00.491+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:01.045 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:00 smithi071 podman[107327]: 2024-09-05 12:50:00.858953106 +0000 UTC m=+6.175566111 container cleanup d33033dee4448490011d6e1729e32c2817508c130ba1d6e456541de78a22f758 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-05T12:50:01.046 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:00 smithi071 bash[107327]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3 2024-09-05T12:50:01.411 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:01 smithi071 podman[107357]: 2024-09-05 12:50:01.127930733 +0000 UTC m=+0.714248289 container remove d33033dee4448490011d6e1729e32c2817508c130ba1d6e456541de78a22f758 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-05T12:50:01.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:01 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:01.044+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:01.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:01.332+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:01.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:01.167+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:01.908 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:01 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:01.544+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:01.909 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:01 smithi071 podman[107442]: 2024-09-05 12:50:01.574301579 +0000 UTC m=+0.028092768 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:01.909 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:01 smithi071 podman[107442]: 2024-09-05 12:50:01.746247561 +0000 UTC m=+0.200038738 container create ab4ba32752835e0c50e30febe4feeb8a4a569952d2aa71719398c29c195993e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T12:50:01.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:01 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:01.489+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:02.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:02.020+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:02.161 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:02 smithi071 podman[107442]: 2024-09-05 12:50:02.023523589 +0000 UTC m=+0.477314788 container init ab4ba32752835e0c50e30febe4feeb8a4a569952d2aa71719398c29c195993e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True) 2024-09-05T12:50:02.161 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:02 smithi071 podman[107442]: 2024-09-05 12:50:02.030366683 +0000 UTC m=+0.484157867 container start ab4ba32752835e0c50e30febe4feeb8a4a569952d2aa71719398c29c195993e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:50:02.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:02.334+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:02.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:02.210+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:02.587 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:02 smithi071 podman[107442]: 2024-09-05 12:50:02.18213542 +0000 UTC m=+0.635926609 container attach ab4ba32752835e0c50e30febe4feeb8a4a569952d2aa71719398c29c195993e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-05T12:50:02.587 INFO:journalctl@ceph.osd.3.smithi071.stdout:Sep 05 12:50:02 smithi071 podman[107442]: 2024-09-05 12:50:02.290107132 +0000 UTC m=+0.743898314 container died ab4ba32752835e0c50e30febe4feeb8a4a569952d2aa71719398c29c195993e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_CLEAN=True) 2024-09-05T12:50:02.910 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:02 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:02.585+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:02.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:02 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:02.516+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:03.072 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.3.service' 2024-09-05T12:50:03.148 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:03.149 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-05T12:50:03.149 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-05T12:50:03.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.4 2024-09-05T12:50:03.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:03.009+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:03.430 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:03 smithi123 systemd[1]: Stopping Ceph osd.4 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:03.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:03.347+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:03.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:03.162+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:03.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:03.604+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:03.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:03 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:03.489+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:03.930 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:03 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4[45623]: 2024-09-05T12:50:03.471+0000 7f900f95d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:03.930 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:03 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4[45623]: 2024-09-05T12:50:03.471+0000 7f900f95d640 -1 osd.4 132 *** Got signal Terminated *** 2024-09-05T12:50:03.930 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:03 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4[45623]: 2024-09-05T12:50:03.471+0000 7f900f95d640 -1 osd.4 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:04.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:03 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:03.982+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:04.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:04.161+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:04.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:04.336+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:04.511 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:04.336+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:04.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:04 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:04.561+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:04.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:04 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:04.467+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:05.339 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:05.158+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:05.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:04 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:04.962+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:05.622 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:05.338+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:05.622 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:05.338+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:05.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:05 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:05.562+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:05.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:05 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:05.431+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:06.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:06 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:06.012+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:06.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:06 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:06.012+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:06.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:06.125+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:06.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:06.356+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:06.511 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:06 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:06.356+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:06.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:06.417+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:06.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:06 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:06.608+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:07.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:06 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:06.819+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:07.389 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:07 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:07.095+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:07.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:06 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:06.963+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:07.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:06 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:06.963+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:07.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:07 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:07.389+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:07.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:07 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:07.389+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:07.871 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:07 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:07.442+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:07.871 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:07 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:07.442+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:07.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:07.619+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:07.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:07.619+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:08.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:07 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:07.870+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:08.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:07.918+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:08.411 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:07 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:07.918+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:08.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:08 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:08.065+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:08.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:08 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:08.420+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:08.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:08 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:08.420+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:08.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:08 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:08.424+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:08.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:08 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:08.424+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:08.894 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:08 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:08.627+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:08.894 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:08 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:08.627+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:09.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:08 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:08.893+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:09.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:08 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:08.893+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:09.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:08 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:08.877+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:09.180 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:09 smithi123 podman[73812]: 2024-09-05 12:50:09.039064467 +0000 UTC m=+5.691376924 container died b15007424ca12e39b77213d95786ef22adea63510cc1ad8516aa14d4720e3ca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-05T12:50:09.382 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:09 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:09.073+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:09.649 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:09 smithi123 podman[73812]: 2024-09-05 12:50:09.438605423 +0000 UTC m=+6.090917869 container cleanup b15007424ca12e39b77213d95786ef22adea63510cc1ad8516aa14d4720e3ca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux ) 2024-09-05T12:50:09.649 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:09 smithi123 bash[73812]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4 2024-09-05T12:50:09.649 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:09 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:09.409+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:09.649 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:09 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:09.409+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:09.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:09 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:09.381+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:09.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:09 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:09.381+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:09.849 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:09.586+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:09.849 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:09.586+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:09.908 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:09 smithi123 podman[73827]: 2024-09-05 12:50:09.648305444 +0000 UTC m=+0.604120212 container remove b15007424ca12e39b77213d95786ef22adea63510cc1ad8516aa14d4720e3ca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T12:50:10.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:09.848+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:10.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:09.848+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:10.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:09 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:09.906+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:10.180 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:09 smithi123 podman[73913]: 2024-09-05 12:50:09.876809483 +0000 UTC m=+0.015537511 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:10.180 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:09 smithi123 podman[73913]: 2024-09-05 12:50:09.999064562 +0000 UTC m=+0.137792590 container create eee9ac482e6193d4f80b77fbc234d756ce0cc1376131d518c4c8a9cf90ab9001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:50:10.382 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:10 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:10.087+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:10.383 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:10 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:10.060+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:10.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:10 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:10.433+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:10.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:10 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:10.433+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:10.680 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:10 smithi123 podman[73913]: 2024-09-05 12:50:10.227065892 +0000 UTC m=+0.365793912 container init eee9ac482e6193d4f80b77fbc234d756ce0cc1376131d518c4c8a9cf90ab9001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-05T12:50:10.680 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:10 smithi123 podman[73913]: 2024-09-05 12:50:10.230378159 +0000 UTC m=+0.369106181 container start eee9ac482e6193d4f80b77fbc234d756ce0cc1376131d518c4c8a9cf90ab9001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:50:10.681 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:10 smithi123 podman[73913]: 2024-09-05 12:50:10.310341469 +0000 UTC m=+0.449069506 container attach eee9ac482e6193d4f80b77fbc234d756ce0cc1376131d518c4c8a9cf90ab9001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-05T12:50:10.681 INFO:journalctl@ceph.osd.4.smithi123.stdout:Sep 05 12:50:10 smithi123 podman[73913]: 2024-09-05 12:50:10.415702417 +0000 UTC m=+0.554430437 container died eee9ac482e6193d4f80b77fbc234d756ce0cc1376131d518c4c8a9cf90ab9001 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:10.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:10 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:10.381+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:10.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:10 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:10.381+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:10.847 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:10.542+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:10.847 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:10.542+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:11.073 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:10 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:10.908+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:11.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.4.service' 2024-09-05T12:50:11.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:11.131 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-05T12:50:11.131 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-05T12:50:11.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.5 2024-09-05T12:50:11.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:10.846+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:11.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:10 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:10.846+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:11.398 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:11 smithi174 systemd[1]: Stopping Ceph osd.5 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:11.399 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:11.130+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:11.399 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:11.039+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:11.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:11.397+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:11.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:11.397+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:11.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:11.462+0000 7f56400dc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:11.761 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:11.462+0000 7f56400dc640 -1 osd.5 132 *** Got signal Terminated *** 2024-09-05T12:50:11.761 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:11 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:11.462+0000 7f56400dc640 -1 osd.5 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:11.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:11 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:11.588+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:11.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:11 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:11.588+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:11.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:11 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:11.801+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:11.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:11 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:11.801+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:11.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:11 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:11.429+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:11.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:11 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:11.429+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:12.352 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:12 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:12.103+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:12.352 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:12 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:12.021+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:12.429 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:11 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:11.952+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:12.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:12 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:12.350+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:12.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:12 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:12.350+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:12.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:12 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:12.635+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:12.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:12 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:12.635+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:12.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:12 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:12.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:12.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:12 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:12.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:12.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:12 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:12.478+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:12.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:12 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:12.478+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:12.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:12 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:12.478+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:13.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:13 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:13.095+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:13.260 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:12 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:12.993+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:13.429 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:12 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:12.951+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:13.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:13 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:13.343+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:13.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:13 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:13.343+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:13.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:13 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:13.623+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:13.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:13 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:13.623+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:13.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:13 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:13.796+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:13.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:13 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:13.796+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:13.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:13 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:13.526+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:13.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:13 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:13.526+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:13.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:13 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:13.526+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:14.260 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:13 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:13.986+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:14.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:14 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:14.096+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:14.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:14 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:14.096+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:14.430 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:13 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:13.936+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:14.430 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:13 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:13.936+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:14.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:14 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:14.326+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:14.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:14 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:14.326+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:14.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:14 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:14.326+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:50.699532+0000 front 2024-09-05T12:49:50.699558+0000 (oldest deadline 2024-09-05T12:50:14.199764+0000) 2024-09-05T12:50:14.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:14 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:14.593+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:14.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:14 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:14.593+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:14.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:14 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:14.752+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:14.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:14 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:14.752+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:14.923 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:14 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:14.481+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:14.923 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:14 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:14.481+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:14.923 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:14 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:14.481+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:15.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:14 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:14.922+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:15.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:14 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:14.922+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:15.369 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:15 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:15.023+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:15.369 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:15 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:15.130+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:15.369 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:15 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:15.130+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:15.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:15 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:15.367+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:15.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:15 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:15.367+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:15.761 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:15 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:15.367+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:50.699532+0000 front 2024-09-05T12:49:50.699558+0000 (oldest deadline 2024-09-05T12:50:14.199764+0000) 2024-09-05T12:50:15.876 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:15 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:15.461+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:15.877 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:15 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:15.461+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:15.877 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:15 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:15.461+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:15.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:15 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:15.639+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:15.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:15 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:15.639+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:15.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:15 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:15.704+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:15.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:15 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:15.704+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:16.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:15 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:15.875+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:16.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:15 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:15.875+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:16.412 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:16.053+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:16.412 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:16.053+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:16.413 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:16.121+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:16.413 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:16.121+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:16.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:16 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:16.419+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:16.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:16 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:16.419+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:16.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:16 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:16.419+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:16.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:16.410+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.296127+0000 front 2024-09-05T12:49:30.296236+0000 (oldest deadline 2024-09-05T12:49:57.897053+0000) 2024-09-05T12:50:16.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:16.410+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:39.598131+0000 front 2024-09-05T12:49:39.598292+0000 (oldest deadline 2024-09-05T12:50:04.298256+0000) 2024-09-05T12:50:16.760 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:16 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5[45649]: 2024-09-05T12:50:16.410+0000 7f563c6f2640 -1 osd.5 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:50.699532+0000 front 2024-09-05T12:49:50.699558+0000 (oldest deadline 2024-09-05T12:50:14.199764+0000) 2024-09-05T12:50:16.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:16 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:16.616+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:16.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:16 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:16.616+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:16.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:16 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:16.616+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:16.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:16 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:16.702+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:16.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:16 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:16.702+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:17.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:16 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:16.910+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:17.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:16 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:16.910+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:17.260 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:16 smithi174 podman[73841]: 2024-09-05 12:50:16.97784467 +0000 UTC m=+5.646663562 container died 6316a7839e5a5dbf327881b3b512ee32d83e53fa43110ec1f54e43097a55820c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:17.261 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:17.083+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:17.261 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:17.083+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:17.261 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:17.040+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:17.261 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:17 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:17.040+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:17.648 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:17 smithi174 podman[73841]: 2024-09-05 12:50:17.388155444 +0000 UTC m=+6.056974336 container cleanup 6316a7839e5a5dbf327881b3b512ee32d83e53fa43110ec1f54e43097a55820c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T12:50:17.648 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:17 smithi174 bash[73841]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5 2024-09-05T12:50:17.869 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:17 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:17.436+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:17.870 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:17 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:17.436+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:17.870 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:17 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:17.436+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:17.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:17.656+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:17.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:17.656+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:17.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:17.656+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:17.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:17.687+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:17.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:17.687+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:17.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:17 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:17.687+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:17.973 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:17 smithi174 podman[73860]: 2024-09-05 12:50:17.647688899 +0000 UTC m=+0.672451930 container remove 6316a7839e5a5dbf327881b3b512ee32d83e53fa43110ec1f54e43097a55820c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:50:18.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:17 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:17.868+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:18.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:17 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:17.868+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:18.235 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:18 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:18.102+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:18.235 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:18 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:18.102+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:18.236 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:17 smithi174 podman[73949]: 2024-09-05 12:50:17.97330446 +0000 UTC m=+0.101077899 container create 33e57ece0a94182d45f6d37dcf3bbe0d704027f0821d061445b860a1a71224b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T12:50:18.236 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:17 smithi174 podman[73949]: 2024-09-05 12:50:17.887610782 +0000 UTC m=+0.015384230 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:18.236 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:18 smithi174 podman[73949]: 2024-09-05 12:50:18.234391921 +0000 UTC m=+0.362165364 container init 33e57ece0a94182d45f6d37dcf3bbe0d704027f0821d061445b860a1a71224b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-05T12:50:18.236 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:18 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:18.007+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:18.237 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:18 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:18.007+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:18.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:18 smithi174 podman[73949]: 2024-09-05 12:50:18.238744361 +0000 UTC m=+0.366517811 container start 33e57ece0a94182d45f6d37dcf3bbe0d704027f0821d061445b860a1a71224b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-05T12:50:18.510 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:18 smithi174 podman[73949]: 2024-09-05 12:50:18.342803208 +0000 UTC m=+0.470576647 container attach 33e57ece0a94182d45f6d37dcf3bbe0d704027f0821d061445b860a1a71224b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-09-05T12:50:18.511 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 05 12:50:18 smithi174 podman[73949]: 2024-09-05 12:50:18.424132695 +0000 UTC m=+0.551906132 container died 33e57ece0a94182d45f6d37dcf3bbe0d704027f0821d061445b860a1a71224b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-05T12:50:18.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:18 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:18.411+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:18.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:18 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:18.411+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:18.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:18 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:18.411+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:19.045 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.5.service' 2024-09-05T12:50:19.083 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:19.083 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-05T12:50:19.084 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-05T12:50:19.084 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.6 2024-09-05T12:50:19.138 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:18 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:18.672+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:19.138 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:18 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:18.672+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:19.138 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:18 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:18.672+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:19.139 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:18 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:18.726+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:19.139 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:18 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:18.726+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:19.139 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:18 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:18.726+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:19.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:18 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:18.874+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:19.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:18 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:18.874+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:19.411 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 systemd[1]: Stopping Ceph osd.6 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:19.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:19 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:19.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:19.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:19 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:19.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:19.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:19 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:19.051+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:19.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:19 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:19.051+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:19.679 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:19 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:19.361+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:19.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:19 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:19.361+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:19.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:19 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:19.361+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:19.772 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:19.517+0000 7f2b4d600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:19.772 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:19.517+0000 7f2b4d600640 -1 osd.6 132 *** Got signal Terminated *** 2024-09-05T12:50:19.772 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:19.517+0000 7f2b4d600640 -1 osd.6 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:19.772 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:19.679+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:19.773 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:19.679+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:19.773 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:19.679+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:20.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:19.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:20.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:19.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:20.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:19 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:19.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:20.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:19 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:19.875+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:20.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:19 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:19.875+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:20.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:20 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:20.065+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:20.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:20 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:20.065+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:20.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:20 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:20.149+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:20.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:20 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:20.149+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:20.679 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:20 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:20.410+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:20.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:20 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:20.410+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:20.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:20 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:20.410+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:21.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:20.737+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:21.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:20.737+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:21.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:20.737+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:21.162 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:20.679+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:21.162 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:20.679+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:21.162 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:20.679+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:21.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:20 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:20.834+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:21.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:20 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:20.834+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:21.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:21 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:21.090+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:21.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:21 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:21.090+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:21.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:21 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:21.123+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:21.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:21 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:21.123+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:21.679 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:21.402+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:21.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:21.402+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:21.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:21.402+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:21.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:21 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:21.636+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:21.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:21 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:21.636+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:21.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:21 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:21.636+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:21.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:21 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:21.692+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:21.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:21 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:21.692+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:21.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:21 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:21.692+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:22.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:21.790+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:22.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:21 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:21.790+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:22.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:22 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:22.093+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:22.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:22 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:22.093+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:22.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:22 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:22.147+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:22.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:22 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:22.147+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:22.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:22 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:22.375+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:22.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:22 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:22.375+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:22.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:22 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:22.375+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:22.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:22 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:22.375+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:22.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:22 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:22.664+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:22.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:22 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:22.664+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:22.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:22 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:22.664+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:22.912 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:22 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:22.623+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:22.912 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:22 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:22.623+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:22.912 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:22 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:22.623+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:23.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:22 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:22.783+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:23.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:22 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:22.783+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:23.411 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:23.103+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:23.412 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:23.103+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:23.412 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:23.194+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:23.412 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:23.194+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:23.412 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:23 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:23.194+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:23.679 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:23.382+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:23.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:23.382+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:23.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:23.382+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:23.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:23.382+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:23.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:23 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:23.703+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:23.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:23 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:23.703+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:23.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:23 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:23.703+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:23.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:23 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:23.622+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:32.913164+0000 front 2024-09-05T12:49:32.913204+0000 (oldest deadline 2024-09-05T12:49:58.213286+0000) 2024-09-05T12:50:23.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:23 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:23.622+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.714688+0000 front 2024-09-05T12:49:41.714784+0000 (oldest deadline 2024-09-05T12:50:07.014878+0000) 2024-09-05T12:50:23.912 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:23 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6[57207]: 2024-09-05T12:50:23.622+0000 7f2b49000640 -1 osd.6 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.015449+0000 front 2024-09-05T12:49:47.015421+0000 (oldest deadline 2024-09-05T12:50:16.416597+0000) 2024-09-05T12:50:24.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:23.771+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:24.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:23.771+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:24.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:24 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:24.198+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:24.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:24 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:24.198+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:24.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:24 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:24.198+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:24.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:24 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:24.119+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:24.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:24 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:24.119+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:24.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:24.415+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:24.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:24.415+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:24.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:24.415+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:24.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:24.415+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:25.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:24 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:24.734+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:25.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:24 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:24.734+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:25.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:24 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:24.734+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:25.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:24.768+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:25.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:24.768+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:25.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:24 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:24.768+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:25.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:25 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:25.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:25.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:25 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:25.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:25.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:25 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:25.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:25.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:25 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:25.164+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:25.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:25 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:25.164+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:25.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:25 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:25.164+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:25.516 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:25 smithi071 podman[107542]: 2024-09-05 12:50:25.261335004 +0000 UTC m=+5.892817520 container died b1d9750bb3ede6a05fb5cdf1f1aa6dac224aafa6a4364a93362e2fa2911736c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-05T12:50:25.756 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:25.448+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:25.756 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:25.448+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:25.756 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:25.448+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:25.756 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:25.448+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:25.773 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:25 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:25.749+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:25.773 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:25 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:25.749+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:25.773 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:25 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:25.749+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:25.773 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:25 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:25.749+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:25.774 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:25 smithi071 podman[107542]: 2024-09-05 12:50:25.771928902 +0000 UTC m=+6.403411417 container cleanup b1d9750bb3ede6a05fb5cdf1f1aa6dac224aafa6a4364a93362e2fa2911736c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:26.161 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:25 smithi071 bash[107542]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6 2024-09-05T12:50:26.161 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:26 smithi071 podman[107562]: 2024-09-05 12:50:26.006853599 +0000 UTC m=+0.732945310 container remove b1d9750bb3ede6a05fb5cdf1f1aa6dac224aafa6a4364a93362e2fa2911736c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T12:50:26.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:25.755+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:26.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:25.755+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:26.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:25 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:25.755+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:26.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:26.163+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:26.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:26.163+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:26.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:26.163+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:26.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:26.115+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:26.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:26.115+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:26.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:26 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:26.115+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:26.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:26 smithi071 podman[107659]: 2024-09-05 12:50:26.52320687 +0000 UTC m=+0.026940044 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:26.911 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:26 smithi071 podman[107659]: 2024-09-05 12:50:26.65790634 +0000 UTC m=+0.161639514 container create 32ccecc27b21a3ca60595f00a64dc96d4195986bdfce8024271aaca5150f29e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:26.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:26 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:26.721+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:26.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:26 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:26.721+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:26.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:26 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:26.721+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:26.912 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:26 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:26.721+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:26.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:26.475+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:26.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:26.475+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:26.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:26.475+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:26.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:26.475+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:26.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:26.713+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:26.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:26.713+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:26.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:26 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:26.713+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:27.193 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:26 smithi071 podman[107659]: 2024-09-05 12:50:26.91929573 +0000 UTC m=+0.423028904 container init 32ccecc27b21a3ca60595f00a64dc96d4195986bdfce8024271aaca5150f29e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-05T12:50:27.193 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:26 smithi071 podman[107659]: 2024-09-05 12:50:26.926017732 +0000 UTC m=+0.429750909 container start 32ccecc27b21a3ca60595f00a64dc96d4195986bdfce8024271aaca5150f29e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:27.194 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:27 smithi071 podman[107659]: 2024-09-05 12:50:27.027506575 +0000 UTC m=+0.531239754 container attach 32ccecc27b21a3ca60595f00a64dc96d4195986bdfce8024271aaca5150f29e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-05T12:50:27.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:27 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:27.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:27.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:27 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:27.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:27.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:27 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:27.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:27.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:27 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:27.147+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:27.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:27 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:27.147+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:27.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:27 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:27.147+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:27.639 INFO:journalctl@ceph.osd.6.smithi071.stdout:Sep 05 12:50:27 smithi071 podman[107659]: 2024-09-05 12:50:27.192940064 +0000 UTC m=+0.696673243 container died 32ccecc27b21a3ca60595f00a64dc96d4195986bdfce8024271aaca5150f29e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-05T12:50:27.756 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:27.442+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:27.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:27.442+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:27.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:27.442+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:27.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:27.442+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:27.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:27.696+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:27.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:27.696+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:27.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:27.696+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:27.911 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:27 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:27.696+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:28.022 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:27.754+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:28.022 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:27.755+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:28.023 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:27 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:27.755+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:28.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.6.service' 2024-09-05T12:50:28.115 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:28.115 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-05T12:50:28.115 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-05T12:50:28.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.7 2024-09-05T12:50:28.429 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 systemd[1]: Stopping Ceph osd.7 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:28.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:28 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:28.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:28.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:28 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:28.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:28.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:28 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:28.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:28.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:28 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:28.155+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:28.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:28 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:28.155+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:28.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:28 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:28.155+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:28.745 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:28.439+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:28.745 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:28.439+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:28.745 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:28.439+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:28.439+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:28.458+0000 7fe2aa5b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:28.458+0000 7fe2aa5b7640 -1 osd.7 132 *** Got signal Terminated *** 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:28.458+0000 7fe2aa5b7640 -1 osd.7 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:28.743+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:28.743+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:28.746 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:28.743+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:29.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:28.690+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:29.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:28.690+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:29.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:28.690+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:29.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:28 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:28.690+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:29.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:29 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:29.140+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:29.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:29 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:29.140+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:29.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:29 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:29.140+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:29.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:29 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:29.174+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:29.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:29 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:29.174+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:29.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:29 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:29.174+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:29.779 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:29.487+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:29.779 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:29.487+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:29.779 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:29.487+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:29.780 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:29.487+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:30.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:29.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:30.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:29.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:30.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:29.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:30.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:29 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:29.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:30.179 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:29.777+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:30.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:29.777+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:30.180 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:29 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:29.777+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:30.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:30.152+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:30.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:30.152+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:30.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:30.152+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:30.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:30.148+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:30.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:30.148+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:30.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:30.148+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:30.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:30 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:30.148+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:30.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:30.764+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:30.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:30.764+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:30.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:30.764+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:30.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:30.516+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:30.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:30.516+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:30.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:30.516+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:30.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:30 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:30.516+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:31.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:30.756+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:31.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:30.756+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:31.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:30.756+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:31.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:30.756+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:31.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:30 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:30.756+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:31.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:31.156+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:31.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:31.156+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:31.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:31.156+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:31.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:31.152+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:31.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:31.152+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:31.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:31.152+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:31.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:31 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:31.152+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:31.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:31.774+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:31.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:31.774+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:31.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:31.774+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:31.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:31.557+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:31.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:31.557+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:31.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:31.557+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:31.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:31.557+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:31.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:31 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:31.557+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:32.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:31.768+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:32.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:31.768+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:32.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:31.768+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:32.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:31.768+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:32.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:31 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:31.768+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:32.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:32.108+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:32.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:32.108+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:32.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:32.108+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:32.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:32.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:32.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:32.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:32.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:32.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:32.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:32 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:32.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:32.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:32.759+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.181098+0000 front 2024-09-05T12:49:43.181074+0000 (oldest deadline 2024-09-05T12:50:06.081203+0000) 2024-09-05T12:50:32.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:32.759+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.782232+0000 front 2024-09-05T12:49:47.782128+0000 (oldest deadline 2024-09-05T12:50:13.682170+0000) 2024-09-05T12:50:32.930 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7[52381]: 2024-09-05T12:50:32.759+0000 7fe2a2bc5640 -1 osd.7 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:58.383716+0000 front 2024-09-05T12:49:58.383752+0000 (oldest deadline 2024-09-05T12:50:24.283763+0000) 2024-09-05T12:50:32.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:32.538+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:32.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:32.538+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:32.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:32.538+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:32.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:32.538+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:32.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:32.538+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:33.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:32.762+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:33.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:32.762+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:33.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:32.762+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:33.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:32.762+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:33.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:32 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:32.762+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:33.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:33.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:33.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:33.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:33.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:33.126+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:33.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:33.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:33.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:33.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:33.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:33.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:33.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:33 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:33.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:33.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:33 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:33.502+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:33.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:33 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:33.502+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:33.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:33 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:33.502+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:33.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:33 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:33.502+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:33.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:33 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:33.502+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:34.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:33 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:33.773+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:34.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:33 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:33.773+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:34.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:33 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:33.773+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:34.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:33 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:33.773+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:34.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:33 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:33.773+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:34.369 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:34 smithi123 podman[74025]: 2024-09-05 12:50:34.030478246 +0000 UTC m=+5.700990486 container died 14e217548d5c5ae3d4bbe36a104561d5edb44c27a8056e26e687b4834a9a5420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:50:34.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:34.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:34.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:34.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:34.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:34.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:34.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:34.181+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:34.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:34.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:34.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:34.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:34.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:34.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:34.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:34 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:34.129+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:34.679 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:34 smithi123 podman[74025]: 2024-09-05 12:50:34.46668599 +0000 UTC m=+6.137198231 container cleanup 14e217548d5c5ae3d4bbe36a104561d5edb44c27a8056e26e687b4834a9a5420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:50:34.680 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:34 smithi123 bash[74025]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7 2024-09-05T12:50:34.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:34 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:34.536+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:34.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:34 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:34.536+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:34.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:34 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:34.536+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:34.681 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:34 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:34.536+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:34.681 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:34 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:34.536+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:35.135 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:34 smithi123 podman[74043]: 2024-09-05 12:50:34.701247023 +0000 UTC m=+0.665076945 container remove 14e217548d5c5ae3d4bbe36a104561d5edb44c27a8056e26e687b4834a9a5420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:50:35.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:34 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:34.760+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:35.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:34 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:34.760+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:35.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:34 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:34.760+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:35.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:34 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:34.760+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:35.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:34 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:34.760+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:35.388 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:35 smithi123 podman[74132]: 2024-09-05 12:50:35.03730455 +0000 UTC m=+0.015600230 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:35.388 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:35 smithi123 podman[74132]: 2024-09-05 12:50:35.14366734 +0000 UTC m=+0.121963047 container create 03e6849e2eee28e53c6988c38c43edbc98b36254e11586001dac1387093a5557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-05T12:50:35.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:35.123+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:35.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:35.123+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:35.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:35.123+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:35.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:35.123+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:35.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:35.166+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:35.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:35.166+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:35.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:35.166+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:35.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:35 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:35.166+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:35.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:35 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:35.572+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:35.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:35 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:35.572+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:35.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:35 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:35.572+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:35.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:35 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:35.572+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:35.680 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:35 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:35.572+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:35.681 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:35 smithi123 podman[74132]: 2024-09-05 12:50:35.387710784 +0000 UTC m=+0.366006450 container init 03e6849e2eee28e53c6988c38c43edbc98b36254e11586001dac1387093a5557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-05T12:50:35.681 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:35 smithi123 podman[74132]: 2024-09-05 12:50:35.391109216 +0000 UTC m=+0.369404881 container start 03e6849e2eee28e53c6988c38c43edbc98b36254e11586001dac1387093a5557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:50:35.681 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:35 smithi123 podman[74132]: 2024-09-05 12:50:35.521646779 +0000 UTC m=+0.499942458 container attach 03e6849e2eee28e53c6988c38c43edbc98b36254e11586001dac1387093a5557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:50:35.681 INFO:journalctl@ceph.osd.7.smithi123.stdout:Sep 05 12:50:35 smithi123 podman[74132]: 2024-09-05 12:50:35.573329149 +0000 UTC m=+0.551624816 container died 03e6849e2eee28e53c6988c38c43edbc98b36254e11586001dac1387093a5557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-05T12:50:36.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:35.805+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:36.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:35.805+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:36.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:35.805+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:36.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:35.805+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:36.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:35 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:35.805+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:36.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.7.service' 2024-09-05T12:50:36.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:36.312 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-05T12:50:36.312 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-05T12:50:36.312 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.8 2024-09-05T12:50:36.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:36.169+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:36.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:36.169+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:36.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:36.169+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:36.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:36.169+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:36.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:36.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:36.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:36.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.139+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:36.512 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 systemd[1]: Stopping Ceph osd.8 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:36.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:36.545+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:36.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:36.545+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:36.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:36.545+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:36.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:36.545+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:36.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:36.545+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:37.010 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.682+0000 7f2522bae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:37.010 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.682+0000 7f2522bae640 -1 osd.8 132 *** Got signal Terminated *** 2024-09-05T12:50:37.010 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:36 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:36.682+0000 7f2522bae640 -1 osd.8 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:37.131 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:36 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:36.843+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:37.132 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:36 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:36.843+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:37.132 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:36 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:36.843+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:37.132 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:36 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:36.843+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:37.132 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:36 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:36.843+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:37.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:37.173+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:37.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:37.173+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:37.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:37.173+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:37.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:37.173+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:37.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:37.185+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:37.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:37.185+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:37.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:37.185+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:37.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:37 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:37.185+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:37.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:37.596+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:37.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:37.596+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:37.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:37.596+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:37.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:37.596+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:37.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:37.596+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:37.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:37 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:37.596+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:38.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:37 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:37.856+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:38.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:37 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:37.856+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:38.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:37 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:37.856+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:38.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:37 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:37.856+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:38.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:37 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:37.856+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:38.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:38.221+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:38.221+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:38.221+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:38.221+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:38.221+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:11.874265+0000 front 2024-09-05T12:50:11.874283+0000 (oldest deadline 2024-09-05T12:50:37.774407+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:38.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:38.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:38.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:38.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:38.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:38.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:38.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:38 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:38.180+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:38.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:38.605+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:38.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:38.605+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:38.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:38.605+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:38.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:38.605+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:38.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:38.605+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:38.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:38 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:38.605+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:39.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:38.828+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:39.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:38.828+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:39.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:38.828+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:39.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:38.828+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:39.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:38.828+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:39.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:38 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:38.828+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:39.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:39.249+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:39.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:39.249+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:39.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:39.249+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:39.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:39.249+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:39.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:39.249+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:11.874265+0000 front 2024-09-05T12:50:11.874283+0000 (oldest deadline 2024-09-05T12:50:37.774407+0000) 2024-09-05T12:50:39.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:39.181+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:39.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:39.181+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:39.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:39.181+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:39.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:39.181+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:39.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:39 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:39.181+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:39.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:39.606+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:39.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:39.606+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:39.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:39.606+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:39.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:39.606+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:39.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:39.606+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:39.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:39 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:39.606+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:40.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:39.795+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:40.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:39.795+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:40.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:39.795+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:40.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:39.795+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:40.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:39.795+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:40.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:39 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:39.795+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:40.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:40.227+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:40.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:40.227+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:40.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:40.227+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:40.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:40.227+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:40.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:40.227+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:11.874265+0000 front 2024-09-05T12:50:11.874283+0000 (oldest deadline 2024-09-05T12:50:37.774407+0000) 2024-09-05T12:50:40.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:40.142+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:40.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:40.142+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:40.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:40.142+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:40.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:40.142+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:40.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:40.142+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:40.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:40.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:40.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:40.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:40.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:40.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:40.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:40.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:40.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:40.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:40.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:40 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:40.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:41.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:40.792+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:41.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:40.792+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:41.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:40.792+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:41.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:40.792+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:41.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:40.792+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:41.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:40 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:40.792+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:41.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:41.233+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:43.370221+0000 front 2024-09-05T12:49:43.370017+0000 (oldest deadline 2024-09-05T12:50:09.269886+0000) 2024-09-05T12:50:41.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:41.233+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:49.270216+0000 front 2024-09-05T12:49:49.270522+0000 (oldest deadline 2024-09-05T12:50:15.170341+0000) 2024-09-05T12:50:41.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:41.233+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.771720+0000 front 2024-09-05T12:49:59.771820+0000 (oldest deadline 2024-09-05T12:50:24.471702+0000) 2024-09-05T12:50:41.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:41.233+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:06.772777+0000 front 2024-09-05T12:50:06.772802+0000 (oldest deadline 2024-09-05T12:50:29.573431+0000) 2024-09-05T12:50:41.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8[52115]: 2024-09-05T12:50:41.233+0000 7f251b1bc640 -1 osd.8 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:11.874265+0000 front 2024-09-05T12:50:11.874283+0000 (oldest deadline 2024-09-05T12:50:37.774407+0000) 2024-09-05T12:50:41.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:41.191+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:41.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:41.191+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:41.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:41.191+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:41.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:41.191+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:41.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:41 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:41.191+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:41.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:41.604+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:41.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:41.604+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:41.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:41.604+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:41.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:41.604+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:41.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:41.604+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:41.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:41 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:41.604+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:42.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:41.775+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:42.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:41.775+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:42.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:41.775+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:42.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:41.775+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:42.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:41.775+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:42.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:41 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:41.775+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:42.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:42 smithi174 podman[74064]: 2024-09-05 12:50:42.243324751 +0000 UTC m=+5.731053566 container died 9ba614411c8df50a6fb7873839ecad9eb0eaf9f75491c20acd9268c8565c5c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:50:42.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:42.200+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:42.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:42.200+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:42.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:42.200+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:42.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:42.200+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:42.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:42 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:42.200+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:42.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:42.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:42.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:42.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:42.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:42.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:42.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:42.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:42.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:42.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:42.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:42 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:42.627+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:42.942 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:42 smithi174 podman[74064]: 2024-09-05 12:50:42.598877027 +0000 UTC m=+6.086605832 container cleanup 9ba614411c8df50a6fb7873839ecad9eb0eaf9f75491c20acd9268c8565c5c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, CEPH_POINT_RELEASE=) 2024-09-05T12:50:42.942 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:42 smithi174 bash[74064]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8 2024-09-05T12:50:43.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:42.820+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:43.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:42.820+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:43.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:42.820+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:43.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:42.820+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:43.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:42.820+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:43.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:42 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:42.820+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:43.232 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:42 smithi174 podman[74079]: 2024-09-05 12:50:42.941964979 +0000 UTC m=+0.694904263 container remove 9ba614411c8df50a6fb7873839ecad9eb0eaf9f75491c20acd9268c8565c5c4e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T12:50:43.510 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:43 smithi174 podman[74167]: 2024-09-05 12:50:43.150500994 +0000 UTC m=+0.015618617 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:43.511 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:43 smithi174 podman[74167]: 2024-09-05 12:50:43.267706886 +0000 UTC m=+0.132824500 container create 11e394af4a5f0a8d5389f5db83ebe0fe05cb063e27c5a1a78bdaf066f7d2ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731) 2024-09-05T12:50:43.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:43 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:43.231+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:43.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:43 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:43.231+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:43.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:43 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:43.231+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:43.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:43 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:43.231+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:43.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:43 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:43.231+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:43.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:43.610+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:43.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:43.610+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:43.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:43.610+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:43.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:43.610+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:43.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:43.610+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:43.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:43 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:43.610+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:44.010 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:43 smithi174 podman[74167]: 2024-09-05 12:50:43.536925916 +0000 UTC m=+0.402043530 container init 11e394af4a5f0a8d5389f5db83ebe0fe05cb063e27c5a1a78bdaf066f7d2ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:44.011 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:43 smithi174 podman[74167]: 2024-09-05 12:50:43.540369177 +0000 UTC m=+0.405486794 container start 11e394af4a5f0a8d5389f5db83ebe0fe05cb063e27c5a1a78bdaf066f7d2ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:50:44.011 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:43 smithi174 podman[74167]: 2024-09-05 12:50:43.620155198 +0000 UTC m=+0.485272820 container attach 11e394af4a5f0a8d5389f5db83ebe0fe05cb063e27c5a1a78bdaf066f7d2ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T12:50:44.011 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 05 12:50:43 smithi174 podman[74167]: 2024-09-05 12:50:43.726891838 +0000 UTC m=+0.592009453 container died 11e394af4a5f0a8d5389f5db83ebe0fe05cb063e27c5a1a78bdaf066f7d2ce46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-05T12:50:44.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:43.841+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:44.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:43.841+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:44.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:43.841+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:44.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:43.841+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:44.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:43.841+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:44.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:43 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:43.841+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:44.316 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:44.202+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:44.316 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:44.202+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:44.316 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:44.202+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:44.316 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:44.202+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:44.316 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:44.202+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:44.339 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.8.service' 2024-09-05T12:50:44.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:44.382 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-05T12:50:44.382 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-05T12:50:44.382 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.9 2024-09-05T12:50:44.838 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 systemd[1]: Stopping Ceph osd.9 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:44.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:44 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:44.582+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:44.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:44 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:44.582+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:44.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:44 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:44.582+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:44.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:44 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:44.582+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:44.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:44 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:44.582+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:44.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:44 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:44.582+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:45.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.836+0000 7f3468000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:45.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.836+0000 7f3468000640 -1 osd.9 132 *** Got signal Terminated *** 2024-09-05T12:50:45.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.836+0000 7f3468000640 -1 osd.9 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:45.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.885+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:45.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.885+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:45.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.885+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:45.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.885+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:45.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.885+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:45.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:44 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:44.885+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:45.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:45 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:45.211+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:45.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:45 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:45.211+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:45.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:45 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:45.211+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:45.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:45 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:45.211+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:45.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:45 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:45.211+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:45.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:45.609+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:45.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:45.609+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:45.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:45.609+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:45.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:45.609+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:45.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:45.609+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:45.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:45 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:45.609+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:46.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:45.850+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:46.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:45.850+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:46.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:45.850+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:46.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:45.850+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:46.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:45.850+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:46.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:45 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:45.850+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:46.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:46.238+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:46.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:46.238+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:46.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:46.238+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:46.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:46.238+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:46.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:46.238+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:46.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:46 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:46.238+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:46.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:46 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:46.568+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:46.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:46 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:46.568+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:46.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:46 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:46.568+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:46.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:46 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:46.568+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:46.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:46 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:46.568+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:46.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:46 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:46.568+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:47.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:46 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:46.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:47.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:46 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:46.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:47.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:46 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:46.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:47.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:46 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:46.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:47.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:46 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:46.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:47.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:46 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:46.823+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:47.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:47 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:47.284+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:47.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:47 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:47.284+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:47.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:47 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:47.284+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:47.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:47 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:47.284+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:47.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:47 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:47.284+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:47.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:47 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:47.284+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:47.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:47.531+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:47.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:47.531+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:47.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:47.531+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:47.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:47.531+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:47.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:47.531+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:47.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:47 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:47.531+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:48.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:47 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:47.785+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:48.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:47 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:47.785+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:48.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:47 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:47.785+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:48.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:47 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:47.785+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:48.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:47 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:47.785+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:48.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:47 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:47.785+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:48.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:48.333+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:48.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:48.333+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:48.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:48.333+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:48.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:48.333+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:48.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:48.333+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:48.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:48.333+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:48.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:48 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:48.569+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:48.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:48 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:48.569+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:48.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:48 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:48.569+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:48.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:48 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:48.569+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:48.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:48 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:48.569+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:48.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:48 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:48.569+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:49.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:48.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:49.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:48.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:49.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:48.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:49.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:48.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:49.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:48.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:49.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:48 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:48.771+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:49.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:49.313+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:49.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:49.313+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:49.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:49.313+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:49.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:49.313+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:49.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:49.313+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:49.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:49 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:49.313+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:49.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:49.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:49.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:49.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:49.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:49.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:49.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:49 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:49.611+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:50.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:49.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.666896+0000 front 2024-09-05T12:49:30.667076+0000 (oldest deadline 2024-09-05T12:49:56.566980+0000) 2024-09-05T12:50:50.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:49.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:38.868270+0000 front 2024-09-05T12:49:38.868480+0000 (oldest deadline 2024-09-05T12:50:05.868984+0000) 2024-09-05T12:50:50.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:49.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:45.869755+0000 front 2024-09-05T12:49:45.869856+0000 (oldest deadline 2024-09-05T12:50:17.070427+0000) 2024-09-05T12:50:50.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:49.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.070867+0000 front 2024-09-05T12:49:57.070802+0000 (oldest deadline 2024-09-05T12:50:25.271231+0000) 2024-09-05T12:50:50.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:49.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:05.271817+0000 front 2024-09-05T12:50:05.271758+0000 (oldest deadline 2024-09-05T12:50:29.971683+0000) 2024-09-05T12:50:50.162 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:49 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9[64164]: 2024-09-05T12:50:49.731+0000 7f345ea00640 -1 osd.9 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:14.573520+0000 front 2024-09-05T12:50:14.573616+0000 (oldest deadline 2024-09-05T12:50:38.673383+0000) 2024-09-05T12:50:50.661 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:50 smithi071 podman[107762]: 2024-09-05 12:50:50.39016568 +0000 UTC m=+5.719843493 container died 3bbc3561934aaa280052fc6289f1900d884a657c54918ead4fea2e87098993e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-05T12:50:50.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:50.289+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:50.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:50.289+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:50.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:50.289+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:50.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:50.289+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:50.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:50.289+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:50.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:50 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:50.289+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:50.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:50 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:50.573+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:51.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:50 smithi071 podman[107762]: 2024-09-05 12:50:50.899840639 +0000 UTC m=+6.229518445 container cleanup 3bbc3561934aaa280052fc6289f1900d884a657c54918ead4fea2e87098993e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:50:51.161 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:50 smithi071 bash[107762]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9 2024-09-05T12:50:51.661 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:51 smithi071 podman[107778]: 2024-09-05 12:50:51.209945056 +0000 UTC m=+0.807988865 container remove 3bbc3561934aaa280052fc6289f1900d884a657c54918ead4fea2e87098993e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T12:50:51.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:51.324+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:51.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:51.324+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:51.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:51.324+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:51.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:51.324+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:51.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:51.324+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:51.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:51 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:51.324+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:51.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:51 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:51.565+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:52.106 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:51 smithi071 podman[107865]: 2024-09-05 12:50:51.711101981 +0000 UTC m=+0.026498282 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:50:52.106 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:51 smithi071 podman[107865]: 2024-09-05 12:50:51.84422542 +0000 UTC m=+0.159621720 container create dd2b01b0d5ecc59c0d2ca2aa46d2dbe43091b7374131a2bb3ec3fc4b6b5a3754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-05T12:50:52.107 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:52 smithi071 podman[107865]: 2024-09-05 12:50:52.105410937 +0000 UTC m=+0.420807239 container init dd2b01b0d5ecc59c0d2ca2aa46d2dbe43091b7374131a2bb3ec3fc4b6b5a3754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T12:50:52.369 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:52 smithi071 podman[107865]: 2024-09-05 12:50:52.112348425 +0000 UTC m=+0.427744732 container start dd2b01b0d5ecc59c0d2ca2aa46d2dbe43091b7374131a2bb3ec3fc4b6b5a3754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-05T12:50:52.370 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:52 smithi071 podman[107865]: 2024-09-05 12:50:52.238589586 +0000 UTC m=+0.553985903 container attach dd2b01b0d5ecc59c0d2ca2aa46d2dbe43091b7374131a2bb3ec3fc4b6b5a3754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD) 2024-09-05T12:50:52.661 INFO:journalctl@ceph.osd.9.smithi071.stdout:Sep 05 12:50:52 smithi071 podman[107865]: 2024-09-05 12:50:52.369062006 +0000 UTC m=+0.684458312 container died dd2b01b0d5ecc59c0d2ca2aa46d2dbe43091b7374131a2bb3ec3fc4b6b5a3754 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-05T12:50:52.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:52.352+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:52.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:52.352+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:52.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:52.352+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:52.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:52.352+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:52.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:52.352+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:52.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:52.352+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:52.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:52.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:52.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:52.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:52.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:52.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:52.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:52 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:52.549+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:53.146 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.9.service' 2024-09-05T12:50:53.223 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:50:53.223 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-05T12:50:53.223 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-05T12:50:53.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.10 2024-09-05T12:50:53.545 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 systemd[1]: Stopping Ceph osd.10 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:50:53.661 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:53 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:53.398+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:53.662 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:53 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:53.398+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:53.662 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:53 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:53.398+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:53.662 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:53 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:53.398+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:53.662 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:53 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:53.398+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:53.662 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:53 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:53.398+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:53.929 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.543+0000 7f9d8e354640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:50:53.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.543+0000 7f9d8e354640 -1 osd.10 132 *** Got signal Terminated *** 2024-09-05T12:50:53.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.543+0000 7f9d8e354640 -1 osd.10 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:50:53.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:53.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:53.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:53.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:53.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:53.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:53.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:53 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:53.597+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:54.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:54 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:54.443+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:54.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:54 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:54.443+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:54.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:54 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:54.443+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:54.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:54 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:54.443+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:54.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:54 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:54.443+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:54.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:54 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:54.443+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:54.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:54.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:54.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:54.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:54.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:54.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:54.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:54 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:54.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:55.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:55.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:55.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:55.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:55.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:55.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:55.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:55 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:55.447+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:50:55.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:55.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:55.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:55.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:55.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:55.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:55.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:55 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:55.663+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:56.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:56.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:56.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:56.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:56.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:56.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:56.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:56 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:56.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:50:56.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:56.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:56.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:56.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:56.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:56.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:56.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:56 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:56.639+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:57.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:57.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:57.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:57.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:57.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:57.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:57.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:57 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:57.463+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:50:57.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:33.824746+0000 front 2024-09-05T12:49:33.824685+0000 (oldest deadline 2024-09-05T12:49:57.324587+0000) 2024-09-05T12:50:57.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-09-05T12:49:41.326303+0000 front 2024-09-05T12:49:41.326241+0000 (oldest deadline 2024-09-05T12:50:07.226381+0000) 2024-09-05T12:50:57.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:47.226919+0000 front 2024-09-05T12:49:47.226802+0000 (oldest deadline 2024-09-05T12:50:12.427489+0000) 2024-09-05T12:50:57.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:59.429236+0000 front 2024-09-05T12:49:59.429274+0000 (oldest deadline 2024-09-05T12:50:22.329160+0000) 2024-09-05T12:50:57.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.630209+0000 front 2024-09-05T12:50:07.630228+0000 (oldest deadline 2024-09-05T12:50:30.530405+0000) 2024-09-05T12:50:57.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:16.431728+0000 front 2024-09-05T12:50:16.431789+0000 (oldest deadline 2024-09-05T12:50:37.531627+0000) 2024-09-05T12:50:57.931 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:57 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10[59356]: 2024-09-05T12:50:57.638+0000 7f9d8a96a640 -1 osd.10 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:23.933214+0000 front 2024-09-05T12:50:23.933334+0000 (oldest deadline 2024-09-05T12:50:48.633108+0000) 2024-09-05T12:50:58.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:58.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:58.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:58.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:58.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:58.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:58.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:58 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:58.497+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:50:59.245 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:58 smithi123 podman[74236]: 2024-09-05 12:50:58.978535085 +0000 UTC m=+5.566362719 container died b4324beaa073a726dfd57cad648b0ddea29f44720f14adef742c0023d57065bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T12:50:59.604 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:59 smithi123 podman[74236]: 2024-09-05 12:50:59.343915763 +0000 UTC m=+5.931743434 container cleanup b4324beaa073a726dfd57cad648b0ddea29f44720f14adef742c0023d57065bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:50:59.604 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:59 smithi123 bash[74236]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10 2024-09-05T12:50:59.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:50:59.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:50:59.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:50:59.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:50:59.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:50:59.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:50:59.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:50:59 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:50:59.488+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:50:59.930 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:59 smithi123 podman[74263]: 2024-09-05 12:50:59.603386608 +0000 UTC m=+0.625505570 container remove b4324beaa073a726dfd57cad648b0ddea29f44720f14adef742c0023d57065bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T12:51:00.207 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:59 smithi123 podman[74353]: 2024-09-05 12:50:59.856397544 +0000 UTC m=+0.015630831 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:51:00.207 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:50:59 smithi123 podman[74353]: 2024-09-05 12:50:59.970731605 +0000 UTC m=+0.129964878 container create 0f9edc7f7c50aff33eaeaa65dab3c72019fec5944fe5a973abf537d73bba0eed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-05T12:51:00.627 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:51:00 smithi123 podman[74353]: 2024-09-05 12:51:00.206294455 +0000 UTC m=+0.365527739 container init 0f9edc7f7c50aff33eaeaa65dab3c72019fec5944fe5a973abf537d73bba0eed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-05T12:51:00.627 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:51:00 smithi123 podman[74353]: 2024-09-05 12:51:00.209586785 +0000 UTC m=+0.368820059 container start 0f9edc7f7c50aff33eaeaa65dab3c72019fec5944fe5a973abf537d73bba0eed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-05T12:51:00.627 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:51:00 smithi123 podman[74353]: 2024-09-05 12:51:00.29794684 +0000 UTC m=+0.457180126 container attach 0f9edc7f7c50aff33eaeaa65dab3c72019fec5944fe5a973abf537d73bba0eed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-05T12:51:00.627 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:51:00 smithi123 podman[74376]: 2024-09-05 12:51:00.418768463 +0000 UTC m=+0.016638556 container died 0f9edc7f7c50aff33eaeaa65dab3c72019fec5944fe5a973abf537d73bba0eed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:00.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:51:00.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:51:00.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:51:00.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:51:00.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:51:00.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:51:00.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:00.455+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:51:00.878 INFO:journalctl@ceph.osd.10.smithi123.stdout:Sep 05 12:51:00 smithi123 podman[74376]: 2024-09-05 12:51:00.861387789 +0000 UTC m=+0.459257885 container remove 0f9edc7f7c50aff33eaeaa65dab3c72019fec5944fe5a973abf537d73bba0eed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0) 2024-09-05T12:51:00.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.10.service' 2024-09-05T12:51:00.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:00.920 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-05T12:51:00.920 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-05T12:51:00.920 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.11 2024-09-05T12:51:01.218 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:00 smithi174 systemd[1]: Stopping Ceph osd.11 for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:01.510 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.217+0000 7fafecadc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:01.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.217+0000 7fafecadc640 -1 osd.11 132 *** Got signal Terminated *** 2024-09-05T12:51:01.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.217+0000 7fafecadc640 -1 osd.11 132 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-05T12:51:01.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.448+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:51:01.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.448+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:51:01.511 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.448+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:51:01.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.449+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:51:01.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.449+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:51:01.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.449+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:51:01.512 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:01 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:01.449+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:51:02.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:51:02.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:51:02.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:51:02.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:51:02.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:51:02.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:51:02.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:02 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:02.490+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:51:03.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:51:03.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:51:03.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:51:03.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:51:03.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:51:03.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:51:03.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:03 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:03.456+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:51:04.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:51:04.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:51:04.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:51:04.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:51:04.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:51:04.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:51:04.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:04 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:04.482+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:51:05.760 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6806 osd.0 since back 2024-09-05T12:49:30.207186+0000 front 2024-09-05T12:49:30.207339+0000 (oldest deadline 2024-09-05T12:49:58.407791+0000) 2024-09-05T12:51:05.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-05T12:49:48.409547+0000 front 2024-09-05T12:49:48.409574+0000 (oldest deadline 2024-09-05T12:50:13.709705+0000) 2024-09-05T12:51:05.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6814 osd.3 since back 2024-09-05T12:49:57.711159+0000 front 2024-09-05T12:49:57.711263+0000 (oldest deadline 2024-09-05T12:50:23.011131+0000) 2024-09-05T12:51:05.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-09-05T12:50:07.612744+0000 front 2024-09-05T12:50:07.612797+0000 (oldest deadline 2024-09-05T12:50:33.512602+0000) 2024-09-05T12:51:05.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-05T12:50:13.512916+0000 front 2024-09-05T12:50:13.512953+0000 (oldest deadline 2024-09-05T12:50:37.612937+0000) 2024-09-05T12:51:05.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.71:6822 osd.6 since back 2024-09-05T12:50:22.314226+0000 front 2024-09-05T12:50:22.314236+0000 (oldest deadline 2024-09-05T12:50:45.814000+0000) 2024-09-05T12:51:05.761 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:05 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11[59108]: 2024-09-05T12:51:05.458+0000 7fafe90f2640 -1 osd.11 132 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-09-05T12:50:33.415380+0000 front 2024-09-05T12:50:33.415420+0000 (oldest deadline 2024-09-05T12:50:55.115306+0000) 2024-09-05T12:51:06.941 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:06 smithi174 podman[74282]: 2024-09-05 12:51:06.643067085 +0000 UTC m=+5.531952639 container died ff837d0ac31a2d093acb938880ce0054f1a8598d73ebffe5a808fd93e1fb0767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-05T12:51:07.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74282]: 2024-09-05 12:51:07.025196413 +0000 UTC m=+5.914081957 container cleanup ff837d0ac31a2d093acb938880ce0054f1a8598d73ebffe5a808fd93e1fb0767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-05T12:51:07.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 bash[74282]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11 2024-09-05T12:51:07.594 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74299]: 2024-09-05 12:51:07.293169163 +0000 UTC m=+0.647534318 container remove ff837d0ac31a2d093acb938880ce0054f1a8598d73ebffe5a808fd93e1fb0767 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS) 2024-09-05T12:51:07.896 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74387]: 2024-09-05 12:51:07.593884661 +0000 UTC m=+0.107713402 container create 1423a8f0b571a64f7290d0f33bb8c9578f43f117045f7a1c77c73c984b68bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS) 2024-09-05T12:51:07.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74387]: 2024-09-05 12:51:07.501890219 +0000 UTC m=+0.015718932 image pull 1d96423d1332efb9e5f8063bc531a59b7531a45a24fb75f81483029528a85232 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945 2024-09-05T12:51:07.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74387]: 2024-09-05 12:51:07.804551256 +0000 UTC m=+0.318379967 container init 1423a8f0b571a64f7290d0f33bb8c9578f43f117045f7a1c77c73c984b68bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:51:07.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74387]: 2024-09-05 12:51:07.807816116 +0000 UTC m=+0.321644823 container start 1423a8f0b571a64f7290d0f33bb8c9578f43f117045f7a1c77c73c984b68bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:07.897 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74387]: 2024-09-05 12:51:07.895944391 +0000 UTC m=+0.409773089 container attach 1423a8f0b571a64f7290d0f33bb8c9578f43f117045f7a1c77c73c984b68bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11-deactivate, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:51:08.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 05 12:51:07 smithi174 podman[74387]: 2024-09-05 12:51:07.994014564 +0000 UTC m=+0.507843263 container died 1423a8f0b571a64f7290d0f33bb8c9578f43f117045f7a1c77c73c984b68bffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-05T12:51:08.591 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@osd.11.service' 2024-09-05T12:51:08.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:08.629 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-05T12:51:08.629 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-05T12:51:08.629 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.a 2024-09-05T12:51:09.039 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:08 smithi071 systemd[1]: Stopping Ceph mds.a for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:09.411 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a[70877]: 2024-09-05T12:51:09.037+0000 7fb679e00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:09.411 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:09 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a[70877]: 2024-09-05T12:51:09.037+0000 7fb679e00640 -1 mds.a *** got signal Terminated *** 2024-09-05T12:51:11.338 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:11 smithi071 podman[107980]: 2024-09-05 12:51:11.040407055 +0000 UTC m=+2.116966978 container died 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-05T12:51:11.855 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:11 smithi071 podman[107980]: 2024-09-05 12:51:11.603251336 +0000 UTC m=+2.679811259 container cleanup 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2) 2024-09-05T12:51:11.856 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:11 smithi071 bash[107980]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a 2024-09-05T12:51:12.116 INFO:journalctl@ceph.mds.a.smithi071.stdout:Sep 05 12:51:11 smithi071 podman[107992]: 2024-09-05 12:51:11.854858492 +0000 UTC m=+0.800355480 container remove 351b4bf41e1ac690f55a4f00b3c3e489a0226e0d51bb37a6190d7bd3f77fce90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:51:12.141 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.a.service' 2024-09-05T12:51:12.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:12.212 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-05T12:51:12.213 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-05T12:51:12.213 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.d 2024-09-05T12:51:12.411 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:12 smithi071 systemd[1]: Stopping Ceph mds.d for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:12.911 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:12 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d[69773]: 2024-09-05T12:51:12.643+0000 7f04c9600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:12.911 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:12 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d[69773]: 2024-09-05T12:51:12.643+0000 7f04c9600640 -1 mds.d *** got signal Terminated *** 2024-09-05T12:51:13.911 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:13 smithi071 podman[108101]: 2024-09-05 12:51:13.650345833 +0000 UTC m=+1.171629020 container died 1b24c4f7298180850da42e69a3bc2d4e657d52cd9006d91786897e66294e6316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:14.196 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:14 smithi071 podman[108101]: 2024-09-05 12:51:14.191141084 +0000 UTC m=+1.712424264 container cleanup 1b24c4f7298180850da42e69a3bc2d4e657d52cd9006d91786897e66294e6316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-05T12:51:14.197 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:14 smithi071 bash[108101]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d 2024-09-05T12:51:14.720 INFO:journalctl@ceph.mds.d.smithi071.stdout:Sep 05 12:51:14 smithi071 podman[108113]: 2024-09-05 12:51:14.46793162 +0000 UTC m=+0.808103146 container remove 1b24c4f7298180850da42e69a3bc2d4e657d52cd9006d91786897e66294e6316 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-d, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T12:51:14.728 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.d.service' 2024-09-05T12:51:14.798 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:14.798 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-05T12:51:14.799 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-05T12:51:14.799 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.g 2024-09-05T12:51:15.161 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:14 smithi071 systemd[1]: Stopping Ceph mds.g for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:15.661 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:15 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g[70532]: 2024-09-05T12:51:15.206+0000 7fcf66800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:15.661 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:15 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g[70532]: 2024-09-05T12:51:15.206+0000 7fcf66800640 -1 mds.g *** got signal Terminated *** 2024-09-05T12:51:19.411 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:19 smithi071 podman[108219]: 2024-09-05 12:51:19.041714282 +0000 UTC m=+3.971629995 container died f59cfe6fb680d8313b27ea7d1a84e7212de79c34aea29bb294e958dd1426c24f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-05T12:51:19.789 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:19 smithi071 podman[108219]: 2024-09-05 12:51:19.586830181 +0000 UTC m=+4.516745873 container cleanup f59cfe6fb680d8313b27ea7d1a84e7212de79c34aea29bb294e958dd1426c24f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-05T12:51:19.789 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:19 smithi071 bash[108219]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g 2024-09-05T12:51:20.044 INFO:journalctl@ceph.mds.g.smithi071.stdout:Sep 05 12:51:19 smithi071 podman[108231]: 2024-09-05 12:51:19.788293526 +0000 UTC m=+0.734855923 container remove f59cfe6fb680d8313b27ea7d1a84e7212de79c34aea29bb294e958dd1426c24f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:20.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.g.service' 2024-09-05T12:51:20.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:20.120 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-05T12:51:20.121 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-05T12:51:20.121 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.j 2024-09-05T12:51:20.411 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:20 smithi071 systemd[1]: Stopping Ceph mds.j for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:20.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j[69367]: 2024-09-05T12:51:20.501+0000 7ff741800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:20.911 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:20 smithi071 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j[69367]: 2024-09-05T12:51:20.501+0000 7ff741800640 -1 mds.j *** got signal Terminated *** 2024-09-05T12:51:22.109 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:21 smithi071 podman[108337]: 2024-09-05 12:51:21.767006332 +0000 UTC m=+1.367591804 container died ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:22.411 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:22 smithi071 podman[108337]: 2024-09-05 12:51:22.223828383 +0000 UTC m=+1.824413854 container cleanup ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:51:22.411 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:22 smithi071 bash[108337]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j 2024-09-05T12:51:22.712 INFO:journalctl@ceph.mds.j.smithi071.stdout:Sep 05 12:51:22 smithi071 podman[108348]: 2024-09-05 12:51:22.475349768 +0000 UTC m=+0.693651941 container remove ff9bc1b3844bcc168f94df6109b27fe50723b36df52d275aac83e14e835dc51a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-j, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-05T12:51:22.732 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.j.service' 2024-09-05T12:51:22.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:22.800 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-05T12:51:22.800 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-05T12:51:22.800 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.b 2024-09-05T12:51:23.080 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:22 smithi123 systemd[1]: Stopping Ceph mds.b for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:23.429 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: 2024-09-05T12:51:23.078+0000 7fd8fb975640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:23.430 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:23 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b[64818]: 2024-09-05T12:51:23.078+0000 7fd8fb975640 -1 mds.b *** got signal Terminated *** 2024-09-05T12:51:27.298 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 podman[74454]: 2024-09-05 12:51:27.002748874 +0000 UTC m=+4.016119061 container died 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-05T12:51:27.646 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 podman[74454]: 2024-09-05 12:51:27.436961344 +0000 UTC m=+4.450331533 container cleanup 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:27.647 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 bash[74454]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b 2024-09-05T12:51:27.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.b.service' 2024-09-05T12:51:27.908 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 podman[74466]: 2024-09-05 12:51:27.646307259 +0000 UTC m=+0.639257933 container remove 50cf64e4c8ba4c131304988ecde39aabd3ffec6b7d18140a31350d3999e347d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:51:27.908 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-05T12:51:27.908 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 systemd[1]: Stopped Ceph mds.b for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:51:27.909 INFO:journalctl@ceph.mds.b.smithi123.stdout:Sep 05 12:51:27 smithi123 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.b.service: Consumed 4min 53.589s CPU time. 2024-09-05T12:51:27.922 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:27.922 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-05T12:51:27.923 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-05T12:51:27.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.e 2024-09-05T12:51:28.180 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:51:28 smithi123 systemd[1]: Stopping Ceph mds.e for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:28.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:51:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e[64210]: 2024-09-05T12:51:28.273+0000 7f40cd757640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:28.680 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:51:28 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e[64210]: 2024-09-05T12:51:28.273+0000 7f40cd757640 -1 mds.e *** got signal Terminated *** 2024-09-05T12:51:31.292 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:51:31 smithi123 podman[74586]: 2024-09-05 12:51:31.006538953 +0000 UTC m=+2.833091026 container died d44c02bcb32f2463ea64f5bf648419895905426f75f01cc3aa6309a8e581c200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-05T12:51:31.625 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:51:31 smithi123 podman[74586]: 2024-09-05 12:51:31.399106459 +0000 UTC m=+3.225658515 container cleanup d44c02bcb32f2463ea64f5bf648419895905426f75f01cc3aa6309a8e581c200 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-05T12:51:31.626 INFO:journalctl@ceph.mds.e.smithi123.stdout:Sep 05 12:51:31 smithi123 bash[74586]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-e 2024-09-05T12:51:31.755 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.e.service' 2024-09-05T12:51:31.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:31.789 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-05T12:51:31.789 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-05T12:51:31.790 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.h 2024-09-05T12:51:31.930 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:31 smithi123 systemd[1]: Stopping Ceph mds.h for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:32.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h[64519]: 2024-09-05T12:51:32.152+0000 7f31760ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:32.430 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:32 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h[64519]: 2024-09-05T12:51:32.152+0000 7f31760ab640 -1 mds.h *** got signal Terminated *** 2024-09-05T12:51:35.363 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:35 smithi123 podman[74708]: 2024-09-05 12:51:35.00960737 +0000 UTC m=+3.004925425 container died 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-05T12:51:35.680 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:35 smithi123 podman[74708]: 2024-09-05 12:51:35.444342765 +0000 UTC m=+3.439660861 container cleanup 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-05T12:51:35.680 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:35 smithi123 bash[74708]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h 2024-09-05T12:51:35.680 INFO:journalctl@ceph.mds.h.smithi123.stdout:Sep 05 12:51:35 smithi123 podman[74721]: 2024-09-05 12:51:35.60362473 +0000 UTC m=+0.594224902 container remove 0e4f9bd171ddf70ae0ea92df592ef5b4ede0411389f343f78a8266e07dec5500 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-05T12:51:35.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.h.service' 2024-09-05T12:51:35.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:35.859 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-05T12:51:35.859 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-05T12:51:35.859 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.k 2024-09-05T12:51:36.179 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:51:35 smithi123 systemd[1]: Stopping Ceph mds.k for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:36.679 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:51:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k[63898]: 2024-09-05T12:51:36.231+0000 7fad8e11c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:36.680 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:51:36 smithi123 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k[63898]: 2024-09-05T12:51:36.231+0000 7fad8e11c640 -1 mds.k *** got signal Terminated *** 2024-09-05T12:51:39.284 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:51:38 smithi123 podman[74831]: 2024-09-05 12:51:38.996629767 +0000 UTC m=+2.911688261 container died 3da75bb26ba549f87cb37fb7c684b3a1d4b133fc34826313a001afa87c8919c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k, org.label-schema.build-date=20240731, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-05T12:51:39.649 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:51:39 smithi123 podman[74831]: 2024-09-05 12:51:39.36449424 +0000 UTC m=+3.279552734 container cleanup 3da75bb26ba549f87cb37fb7c684b3a1d4b133fc34826313a001afa87c8919c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-05T12:51:39.649 INFO:journalctl@ceph.mds.k.smithi123.stdout:Sep 05 12:51:39 smithi123 bash[74831]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-k 2024-09-05T12:51:39.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.k.service' 2024-09-05T12:51:39.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:39.802 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-05T12:51:39.802 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-05T12:51:39.802 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.c 2024-09-05T12:51:40.116 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:51:39 smithi174 systemd[1]: Stopping Ceph mds.c for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:40.510 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:51:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c[64095]: 2024-09-05T12:51:40.113+0000 7f708ef51640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:40.510 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:51:40 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c[64095]: 2024-09-05T12:51:40.113+0000 7f708ef51640 -1 mds.c *** got signal Terminated *** 2024-09-05T12:51:43.260 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:51:43 smithi174 podman[74489]: 2024-09-05 12:51:43.001725986 +0000 UTC m=+3.004555062 container died 0482b6e873df6258f9d1956e172e580c3c9a1128d4947345dc6e93da828fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD) 2024-09-05T12:51:43.708 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:51:43 smithi174 podman[74489]: 2024-09-05 12:51:43.423139446 +0000 UTC m=+3.425968506 container cleanup 0482b6e873df6258f9d1956e172e580c3c9a1128d4947345dc6e93da828fe34f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-05T12:51:43.708 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 05 12:51:43 smithi174 bash[74489]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-c 2024-09-05T12:51:43.836 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.c.service' 2024-09-05T12:51:43.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:43.871 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-05T12:51:43.871 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-05T12:51:43.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.f 2024-09-05T12:51:44.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:51:43 smithi174 systemd[1]: Stopping Ceph mds.f for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:44.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:51:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: 2024-09-05T12:51:44.200+0000 7fcdd816d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:44.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:51:44 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f[63793]: 2024-09-05T12:51:44.200+0000 7fcdd816d640 -1 mds.f *** got signal Terminated *** 2024-09-05T12:51:47.260 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:51:46 smithi174 podman[74611]: 2024-09-05 12:51:46.987966757 +0000 UTC m=+2.890333859 container died 7d7208420c38c0b9710b9ec94b81f9fa3b1f5657b9acb76649af89a3c433ba26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-05T12:51:47.653 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:51:47 smithi174 podman[74611]: 2024-09-05 12:51:47.443011681 +0000 UTC m=+3.345378774 container cleanup 7d7208420c38c0b9710b9ec94b81f9fa3b1f5657b9acb76649af89a3c433ba26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b) 2024-09-05T12:51:47.654 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 05 12:51:47 smithi174 bash[74611]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-f 2024-09-05T12:51:47.863 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.f.service' 2024-09-05T12:51:47.896 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:47.897 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-05T12:51:47.897 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-05T12:51:47.897 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.i 2024-09-05T12:51:48.214 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:47 smithi174 systemd[1]: Stopping Ceph mds.i for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:48.510 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i[64397]: 2024-09-05T12:51:48.211+0000 7faf33ed3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:48.510 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:48 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i[64397]: 2024-09-05T12:51:48.211+0000 7faf33ed3640 -1 mds.i *** got signal Terminated *** 2024-09-05T12:51:51.260 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:50 smithi174 podman[74731]: 2024-09-05 12:51:50.991665197 +0000 UTC m=+2.882776389 container died 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-05T12:51:51.540 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:51 smithi174 podman[74731]: 2024-09-05 12:51:51.338108136 +0000 UTC m=+3.229219336 container cleanup 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-05T12:51:51.540 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:51 smithi174 bash[74731]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i 2024-09-05T12:51:51.769 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.i.service' 2024-09-05T12:51:51.790 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:51 smithi174 podman[74743]: 2024-09-05 12:51:51.539394159 +0000 UTC m=+0.542489207 container remove 6f51e998a793d4e774d67a0530fd2ca319198b50577275bff5f4931f99e903bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-i, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-05T12:51:51.791 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:51 smithi174 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-05T12:51:51.791 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:51 smithi174 systemd[1]: Stopped Ceph mds.i for 8c9ae41c-6b80-11ef-bcd6-c7b262605968. 2024-09-05T12:51:51.791 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 05 12:51:51 smithi174 systemd[1]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.i.service: Consumed 3min 21.657s CPU time. 2024-09-05T12:51:51.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:51.802 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-05T12:51:51.803 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-05T12:51:51.803 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.l 2024-09-05T12:51:52.126 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:51:51 smithi174 systemd[1]: Stopping Ceph mds.l for 8c9ae41c-6b80-11ef-bcd6-c7b262605968... 2024-09-05T12:51:52.510 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:51:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l[63481]: 2024-09-05T12:51:52.124+0000 7f208ee6c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-05T12:51:52.510 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:51:52 smithi174 ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l[63481]: 2024-09-05T12:51:52.124+0000 7f208ee6c640 -1 mds.l *** got signal Terminated *** 2024-09-05T12:51:55.260 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:51:55 smithi174 podman[74849]: 2024-09-05 12:51:55.005698492 +0000 UTC m=+2.985230952 container died 4708f31af45f58a5cd8bfd0151df6544c3994c0df8847ea1335b29d192ae7db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-05T12:51:55.510 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:51:55 smithi174 podman[74849]: 2024-09-05 12:51:55.333051457 +0000 UTC m=+3.312583906 container cleanup 4708f31af45f58a5cd8bfd0151df6544c3994c0df8847ea1335b29d192ae7db9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a8d119a7bc9b781809e472dee68e518e59e6f64fda4c70bd4af602122e8dd945, name=ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-bharath6-testing-2024-09-05-1413-squid-e856d6b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-05T12:51:55.510 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 05 12:51:55 smithi174 bash[74849]: ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968-mds-l 2024-09-05T12:51:55.754 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-8c9ae41c-6b80-11ef-bcd6-c7b262605968@mds.l.service' 2024-09-05T12:51:55.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-05T12:51:55.787 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-05T12:51:55.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T12:51:56.058 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:51:59.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T12:52:00.194 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:52:02.732 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --force --keep-logs 2024-09-05T12:52:02.931 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:52:05.320 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:52:05.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:52:05.417 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-05T12:52:05.444 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-05T12:52:05.447 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi071/crash 2024-09-05T12:52:05.447 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T12:52:05.522 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T12:52:05.523 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-09-05T12:52:05.526 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi123/crash 2024-09-05T12:52:05.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T12:52:05.574 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T12:52:05.575 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-09-05T12:52:05.576 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi174/crash 2024-09-05T12:52:05.577 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash -- . 2024-09-05T12:52:05.613 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/crash: Cannot open: No such file or directory 2024-09-05T12:52:05.613 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-09-05T12:52:05.614 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-05T12:52:05.615 DEBUG:teuthology.orchestra.run.smithi071:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-05T12:52:05.689 INFO:tasks.cephadm:Compressing logs... 2024-09-05T12:52:05.690 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T12:52:05.694 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T12:52:05.696 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-05T12:52:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T12:52:05.726 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T12:52:05.727 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T12:52:05.727 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.i.log 2024-09-05T12:52:05.727 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.c.log 2024-09-05T12:52:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log: /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T12:52:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.11.log 2024-09-05T12:52:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.2.log 2024-09-05T12:52:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.5.log 2024-09-05T12:52:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T12:52:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.l.log 2024-09-05T12:52:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.8.log 2024-09-05T12:52:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.z.log 2024-09-05T12:52:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.c.log 2024-09-05T12:52:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.f.log 2024-09-05T12:52:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T12:52:05.733 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-05T12:52:05.734 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.10.log 2024-09-05T12:52:05.735 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.1.log 2024-09-05T12:52:05.736 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.7.log 2024-09-05T12:52:05.736 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T12:52:05.736 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.b.log 2024-09-05T12:52:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.k.log 2024-09-05T12:52:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.4.log 2024-09-05T12:52:05.737 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T12:52:05.740 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T12:52:05.740 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.y.log 2024-09-05T12:52:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T12:52:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.h.log 2024-09-05T12:52:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.b.log 2024-09-05T12:52:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.e.log 2024-09-05T12:52:05.741 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T12:52:05.744 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log: No such file or directory 2024-09-05T12:52:05.744 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-05T12:52:05.744 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.0.log 2024-09-05T12:52:05.745 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.9.log 2024-09-05T12:52:05.746 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log 2024-09-05T12:52:05.747 INFO:teuthology.orchestra.run.smithi071.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-05T12:52:05.747 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log 2024-09-05T12:52:05.748 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.a.log 2024-09-05T12:52:05.749 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log 2024-09-05T12:52:05.751 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.a.log 2024-09-05T12:52:05.752 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log: gzip 90.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log.gz -5 2024-09-05T12:52:05.752 INFO:teuthology.orchestra.run.smithi071.stderr: --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.g.log 2024-09-05T12:52:05.752 INFO:teuthology.orchestra.run.smithi071.stderr: 84.8% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T12:52:05.753 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.6.log 2024-09-05T12:52:05.754 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log 2024-09-05T12:52:05.757 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.6.log: /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.d.log 2024-09-05T12:52:05.761 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.x.log 2024-09-05T12:52:05.761 INFO:teuthology.orchestra.run.smithi071.stderr: 91.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T12:52:05.762 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.j.log 2024-09-05T12:52:05.767 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.3.log 2024-09-05T12:52:05.767 INFO:teuthology.orchestra.run.smithi071.stderr: 86.9% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T12:52:05.767 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-05T12:52:05.778 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.d.log.gz 2024-09-05T12:52:05.781 INFO:teuthology.orchestra.run.smithi071.stderr: 94.5% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.j.log.gz 2024-09-05T12:52:05.792 INFO:teuthology.orchestra.run.smithi071.stderr: 94.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T12:52:05.856 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T12:52:05.903 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.y.log.gz 2024-09-05T12:52:05.908 INFO:teuthology.orchestra.run.smithi123.stderr: 82.6% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T12:52:05.932 INFO:teuthology.orchestra.run.smithi123.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T12:52:05.948 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-volume.log.gz 2024-09-05T12:52:05.985 INFO:teuthology.orchestra.run.smithi174.stderr: 82.5% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.cephadm.log.gz 2024-09-05T12:52:06.124 INFO:teuthology.orchestra.run.smithi123.stderr: 90.4% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T12:52:06.292 INFO:teuthology.orchestra.run.smithi123.stderr: 87.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T12:52:06.339 INFO:teuthology.orchestra.run.smithi071.stderr: 90.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.x.log.gz 2024-09-05T12:52:06.577 INFO:teuthology.orchestra.run.smithi174.stderr: 90.5% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.audit.log.gz 2024-09-05T12:52:06.585 INFO:teuthology.orchestra.run.smithi174.stderr: 87.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph.log.gz 2024-09-05T12:52:06.586 INFO:teuthology.orchestra.run.smithi174.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-05T12:52:07.027 INFO:teuthology.orchestra.run.smithi174.stderr: 92.6% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mgr.z.log.gz 2024-09-05T12:52:07.866 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.b.log.gz 2024-09-05T12:52:08.550 INFO:teuthology.orchestra.run.smithi071.stderr: 91.5% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.a.log.gz 2024-09-05T12:52:09.119 INFO:teuthology.orchestra.run.smithi071.stderr: 95.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.a.log.gz 2024-09-05T12:52:10.286 INFO:teuthology.orchestra.run.smithi174.stderr: 91.7% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mon.c.log.gz 2024-09-05T12:52:12.964 INFO:teuthology.orchestra.run.smithi174.stderr: 95.1% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.f.log.gz 2024-09-05T12:52:25.136 INFO:teuthology.orchestra.run.smithi123.stderr: 95.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.e.log.gz 2024-09-05T12:52:30.483 INFO:teuthology.orchestra.run.smithi174.stderr: 95.1% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.l.log.gz 2024-09-05T12:52:32.197 INFO:teuthology.orchestra.run.smithi123.stderr: 95.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.k.log.gz 2024-09-05T12:52:36.533 INFO:teuthology.orchestra.run.smithi071.stderr: 91.4% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.0.log.gz 2024-09-05T12:52:45.953 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.9.log.gz 2024-09-05T12:52:50.337 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.6.log.gz 2024-09-05T12:52:51.955 INFO:teuthology.orchestra.run.smithi071.stderr: 91.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.3.log.gz 2024-09-05T12:52:52.153 INFO:teuthology.orchestra.run.smithi174.stderr: 91.8% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.11.log.gz 2024-09-05T12:53:10.746 INFO:teuthology.orchestra.run.smithi071.stderr: 93.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.g.log.gz 2024-09-05T12:53:10.750 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-09-05T12:53:10.750 INFO:teuthology.orchestra.run.smithi071.stderr:real 1m5.026s 2024-09-05T12:53:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:user 3m36.471s 2024-09-05T12:53:10.751 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m16.487s 2024-09-05T12:53:13.097 INFO:teuthology.orchestra.run.smithi174.stderr: 93.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.i.log.gz 2024-09-05T12:53:23.913 INFO:teuthology.orchestra.run.smithi123.stderr: 91.6% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.1.log.gz 2024-09-05T12:53:25.931 INFO:teuthology.orchestra.run.smithi174.stderr: 91.4% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.2.log.gz 2024-09-05T12:53:34.827 INFO:teuthology.orchestra.run.smithi123.stderr: 91.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.10.log.gz 2024-09-05T12:53:45.225 INFO:teuthology.orchestra.run.smithi123.stderr: 91.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.4.log.gz 2024-09-05T12:53:49.536 INFO:teuthology.orchestra.run.smithi123.stderr: 93.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.b.log.gz 2024-09-05T12:53:49.714 INFO:teuthology.orchestra.run.smithi174.stderr: 91.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.8.log.gz 2024-09-05T12:53:50.284 INFO:teuthology.orchestra.run.smithi123.stderr: 91.2% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.7.log.gz 2024-09-05T12:53:59.759 INFO:teuthology.orchestra.run.smithi174.stderr: 91.1% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-osd.5.log.gz 2024-09-05T12:54:04.071 INFO:teuthology.orchestra.run.smithi174.stderr: 93.3% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.c.log.gz 2024-09-05T12:54:04.073 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-09-05T12:54:04.074 INFO:teuthology.orchestra.run.smithi174.stderr:real 1m58.362s 2024-09-05T12:54:04.074 INFO:teuthology.orchestra.run.smithi174.stderr:user 4m24.361s 2024-09-05T12:54:04.074 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m9.330s 2024-09-05T12:54:06.042 INFO:teuthology.orchestra.run.smithi123.stderr: 93.0% -- replaced with /var/log/ceph/8c9ae41c-6b80-11ef-bcd6-c7b262605968/ceph-mds.h.log.gz 2024-09-05T12:54:06.044 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-05T12:54:06.045 INFO:teuthology.orchestra.run.smithi123.stderr:real 2m0.333s 2024-09-05T12:54:06.045 INFO:teuthology.orchestra.run.smithi123.stderr:user 4m36.619s 2024-09-05T12:54:06.045 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m9.497s 2024-09-05T12:54:06.046 INFO:tasks.cephadm:Archiving logs... 2024-09-05T12:54:06.046 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi071/log 2024-09-05T12:54:06.048 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T12:54:38.760 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi123/log 2024-09-05T12:54:38.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T12:55:21.707 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi174/log 2024-09-05T12:55:21.709 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-05T12:56:06.692 INFO:tasks.cephadm:Removing cluster... 2024-09-05T12:56:06.693 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --force 2024-09-05T12:56:06.964 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:56:08.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --force 2024-09-05T12:56:08.395 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:56:08.980 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 8c9ae41c-6b80-11ef-bcd6-c7b262605968 --force 2024-09-05T12:56:09.182 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 8c9ae41c-6b80-11ef-bcd6-c7b262605968 2024-09-05T12:56:09.785 INFO:tasks.cephadm:Removing cephadm ... 2024-09-05T12:56:09.786 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T12:56:09.825 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T12:56:09.844 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-05T12:56:09.860 INFO:tasks.cephadm:Teardown complete 2024-09-05T12:56:09.860 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-05T12:56:09.871 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-05T12:56:09.871 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T12:56:09.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T12:56:09.886 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-05T12:56:10.019 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-05T12:56:10.029 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-05T12:56:10.030 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:56:10.034 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:56:10.036 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-05T12:56:10.049 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:56:10.051 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:56:10.064 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-09-05T12:56:10.065 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:56:10.066 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-05T12:56:10.066 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 +335us[ +348us] +/- 54ms 2024-09-05T12:56:10.066 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -487us[ -487us] +/- 74ms 2024-09-05T12:56:10.066 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 16 +36us[ +36us] +/- 64ms 2024-09-05T12:56:10.066 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:56:10.068 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:56:10.068 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-05T12:56:10.068 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 18 +319us[ +326us] +/- 54ms 2024-09-05T12:56:10.069 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 -497us[ -490us] +/- 74ms 2024-09-05T12:56:10.069 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 17 +56us[ +56us] +/- 64ms 2024-09-05T12:56:10.069 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:56:10.071 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-05T12:56:10.072 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-09-05T12:56:10.072 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +340us[ +358us] +/- 54ms 2024-09-05T12:56:10.072 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -478us[ -460us] +/- 74ms 2024-09-05T12:56:10.072 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 25 +45us[ +63us] +/- 63ms 2024-09-05T12:56:10.072 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-05T12:56:10.073 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-05T12:56:10.094 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-05T12:56:10.095 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-05T12:56:10.105 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T12:56:10.246 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T12:56:10.373 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-05T12:56:10.502 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:56:10.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:56:10.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:56:10.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:56:10.593 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-05T12:56:10.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:56:10.622 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-05T12:56:10.634 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-05T12:56:10.645 INFO:teuthology.task.internal:Duration was 2824.023567 seconds 2024-09-05T12:56:10.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-05T12:56:10.656 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-05T12:56:10.656 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T12:56:10.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T12:56:10.663 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-05T12:56:10.727 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:56:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:56:10.743 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-05T12:56:11.060 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-05T12:56:11.060 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-09-05T12:56:11.060 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T12:56:11.110 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-05T12:56:11.110 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T12:56:11.140 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-09-05T12:56:11.140 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-05T12:56:11.175 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-05T12:56:11.175 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T12:56:11.182 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T12:56:11.185 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-05T12:56:11.244 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-05T12:56:11.245 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T12:56:11.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T12:56:11.255 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-05T12:56:12.388 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-05T12:56:12.401 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-05T12:56:12.401 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T12:56:12.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T12:56:12.508 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-05T12:56:12.548 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-05T12:56:12.559 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:56:12.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:56:12.566 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:56:12.594 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-05T12:56:12.630 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-09-05T12:56:12.635 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-09-05T12:56:12.659 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:56:12.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:56:12.693 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:56:12.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:56:12.708 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-05T12:56:12.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-05T12:56:12.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-05T12:56:12.739 INFO:teuthology.task.internal:Transferring archived files... 2024-09-05T12:56:12.741 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi071 2024-09-05T12:56:12.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T12:56:12.832 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi123 2024-09-05T12:56:12.832 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T12:56:12.870 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-05_11:20:21-fs:workload-wip-bharath6-testing-2024-09-05-1413-squid-testing-default-smithi/7890431/remote/smithi174 2024-09-05T12:56:12.870 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-05T12:56:12.912 INFO:teuthology.task.internal:Removing archive directory... 2024-09-05T12:56:12.912 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T12:56:12.917 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T12:56:12.919 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-05T12:56:12.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-05T12:56:12.978 INFO:teuthology.task.internal:Not uploading archives. 2024-09-05T12:56:12.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-05T12:56:13.009 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-05T12:56:13.009 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T12:56:13.013 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T12:56:13.016 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-05T12:56:13.028 INFO:teuthology.orchestra.run.smithi123.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 12:56 /home/ubuntu/cephtest 2024-09-05T12:56:13.032 INFO:teuthology.orchestra.run.smithi174.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 12:56 /home/ubuntu/cephtest 2024-09-05T12:56:13.044 INFO:teuthology.orchestra.run.smithi071.stdout: 393253 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 5 12:56 /home/ubuntu/cephtest 2024-09-05T12:56:13.046 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-05T12:56:13.058 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-05T12:56:13.086 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8ad8b8b160987a14eb199bb67cc00e23a08d43f6 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2824.0235674381256 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi071 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e856d6b2462509a377ef39149a05475ed006ebac TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_hyelloji@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=74a2ce9eec454e6bae926a672758bbab status: fail success: false 2024-09-05T12:56:13.087 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-05T12:56:13.161 INFO:teuthology.run:FAIL